Unmanned aerial vehicle autonomous navigation method and system based on YOLO visual neural network

Abstract

The invention discloses an unmanned aerial vehicle autonomous navigation method and system based on a YOLO visual neural network. The method comprises the steps that image data and environment sensing data of a photovoltaic power station are collected in real time; performing scene type judgment according to the environment sensing data acquired in real time and scheduling the YOLO model matched with the current scene type; thermally loading the matched YOLO model, inputting real-time acquired image data of the photovoltaic power station into the matched YOLO model for target detection, and obtaining initial target positioning; performing post-processing on a detection result output by the YOLO to obtain final positioning of the target; judging whether a track generated according to the sequence of the final positioning of the currently detected photovoltaic module and the final positioning of the photovoltaic module detected at the previous moment accords with an expected track or not; if yes, continuing to navigate according to the expected track; otherwise, re-planning to obtain the flight path of the unmanned aerial vehicle and navigating. According to the invention, target detection can be rapidly and accurately carried out on a resource-limited unmanned aerial vehicle platform, and the autonomous navigation capability is improved.

Description

Technical Field

[0001] The present application relates to the technical field of drone navigation and target detection, and specifically to a drone autonomous navigation method and system based on the YOLO visual neural network. Background Art

[0002] With the rapid development of science and technology, the field of autonomous navigation of drones has made significant progress. Computer vision-based target detection and tracking technologies have been widely used, enabling drones to identify and track specific targets in complex environments.

[0003] In the field of autonomous drone navigation, traditional methods have employed a variety of approaches to address target detection and navigation. For example, relying on convolutional neural networks (CNNs) for image recognition and target detection is a common approach. CNNs extract and learn image features through multiple convolutional layers, performing exceptionally well in image classification tasks. They can identify target objects in images, determine their current location, and guide drone flight based on pre-set routes and rules, enabling a degree of autonomous navigation. While CNNs excel in image classification, their processing speed and accuracy often struggle to meet demanding real-time requirements. Existing technologies utilize the YOLO model algorithm for image recognition, which can, to a certain extent, enhance drones' real-time target detection and tracking capabilities in dynamic environments. However, their high computational resource consumption limits their application on resource-constrained drone platforms.

[0004] Therefore, how to achieve accurate and efficient positioning and navigation on resource-constrained drone application platforms has become a technical problem that needs to be solved urgently. Summary of the Invention

[0005] In order to achieve fast and accurate target detection on resource-constrained drone platforms and improve the timeliness and accuracy of drone autonomous navigation; this application provides a drone autonomous navigation method and system based on the YOLO visual neural network.

[0006] In a first aspect, the present application provides a method for autonomous navigation of a drone based on a YOLO visual neural network, comprising: Real-time collection of image data of photovoltaic power plants as well as illumination, occlusion, and height / angle data; The scene type is determined based on real-time collected illumination, occlusion, and height / angle data. A YOLO model matching the current scene type is retrieved from the YOLO visual neural network model library. Each YOLO model uses the same Backbone and is trained using a differentiated prediction head based on the PV power plant image data corresponding to the scene type, and undergoes lightweight processing. Hot-load the acquired YOLO model that matches the current scene type, input the real-time image data of the photovoltaic power station into the YOLO model that matches the current scene type for target detection, and obtain the initial location of the currently detected photovoltaic module; post-process the detection results output by YOLO, including positioning correction based on prior knowledge of the topological relationship of photovoltaic modules in the photovoltaic power station, to obtain the final location of the currently detected photovoltaic module; Determine whether the trajectory generated based on the final positioning of the currently detected PV module and the final positioning of the PV module detected at the previous moment meets the expected trajectory; if it meets the expected trajectory, continue to navigate according to the expected trajectory; otherwise, count the final positioning of all PV modules obtained by detection, determine the undetected PV modules, and use the final positioning of the currently detected PV module as the starting point to replan the UAV flight trajectory and navigate according to the optimal path algorithm.

[0007] By adopting the above solution, scene type judgment is performed based on real-time collected illumination, occlusion, and height / angle data, and a lightweight YOLO model is scheduled and matched, thereby improving the pertinence and efficiency of photovoltaic module target detection in different scenarios. The detection results are corrected by using prior knowledge of topological relationships, thereby improving the accuracy of photovoltaic module positioning. The flight trajectory is flexibly adjusted according to whether the positioning trajectory meets expectations, thereby enhancing the flexibility and intelligence of the drone's autonomous navigation.

[0008] Preferably, it also includes: When the scene type is determined based on the real-time collected illumination, occlusion, and height / angle data, the determined scene type is not a single scene type, and all single scene types are obtained; Design a hierarchical trigger mechanism and determine the priority trigger judgment order for each scene type. Set priority trigger conditions for multiple single scene types corresponding to the lighting, occlusion, and height / angle data ranges. Determine and obtain the trigger scene type priority based on the collected lighting, occlusion, and height / angle data. According to the priority order of the triggered scene types, the YOLO model corresponding to the single scene type is switched and hot-loaded in sequence; the initial positioning of the photovoltaic component currently detected by the target detection output of the YOLO model matching the single scene type is obtained respectively, and positioning fusion is performed according to the preset fusion positioning engine rules to obtain the initial positioning of the fused photovoltaic component; the preset fusion positioning engine rules include weighted fusion rules, and the higher the priority order, the higher the corresponding weighted weight.

[0009] By adopting the above solution, when the scene type is not a single scene type, a hierarchical trigger mechanism is set to determine the priority order, and the corresponding YOLO model is hot-loaded in the priority order and positioning fusion is performed, thereby improving the positioning accuracy and target detection performance of the UAV for photovoltaic modules in complex environments, and enhancing the adaptability and reliability of the UAV's autonomous navigation.

[0010] Preferably, it also includes: Monitor the illumination, occlusion and height / angle data collected in real time. If the fluctuation values ​​of the illumination, occlusion and height / angle perception data collected in consecutive preset time periods are all less than the corresponding preset fluctuation thresholds, it is judged that they belong to the same scene type. There is no need to switch the YOLO model for hot loading. Instead, a dynamic batch processing method is selected to input the image data of the photovoltaic power station corresponding to the same scene type into the YOLO model matching the current scene type for target detection, so as to replace the input of the real-time image data of the photovoltaic power station into the YOLO model matching the current scene type for target detection. The dynamic batch processing method includes: setting a basic batch processing size; monitoring the processing performance during the target detection process for the image data of the photovoltaic power station, including: processing time, memory occupancy and CPU / GPU utilization; dynamically increasing or decreasing the batch size based on the basic batch processing size according to whether the monitored processing performance is in the preset processing performance range; The initial positioning of several frames of photovoltaic components output by batch processing is obtained and the detection results output by YOLO are post-processed to obtain the final positioning of several frames of photovoltaic components output by batch processing. It is judged whether the trajectories generated according to the final positioning of several frames of photovoltaic components output by different batch processing in the batch processing order and the final positioning of several frames of photovoltaic components output by the same batch processing in the corresponding frame acquisition order meet the expected trajectory, instead of judging whether the trajectory generated according to the final positioning of the photovoltaic component currently detected and the final positioning of the photovoltaic component detected at the previous moment meet the expected trajectory.

[0011] By adopting the above solution, the perception data can be monitored in real time and determined to be in a stable state. There is no need to switch the YOLO model for hot loading. Dynamic batch processing is used to perform target detection on multiple frames of image data of the same scene type, reducing the resource consumption and time cost caused by real-time hot loading and model detection. The batch size is dynamically adjusted according to the processing performance, which can fully utilize system resources and improve processing efficiency.

[0012] Preferably, it also includes: Before inputting the real-time image data of the photovoltaic power station into the YOLO model matching the current scene type for target detection, the real-time image data of the photovoltaic power station is pre-inputted into the CNN convolution model to obtain the background complexity of the currently collected image data of the photovoltaic power station; After retrieving the YOLO model that matches the current scene type from the YOLO visual neural network model library according to the judged scene type, each YOLO model is designed to include a YOLO model with a PAN structure and a YOLO model without a PAN structure, and then continue to match the YOLO model with or without a PAN structure according to the background complexity of the currently collected photovoltaic power station image data, including matching the YOLO model without a PAN structure when the background complexity of the currently collected photovoltaic power station image data is greater than the preset background complexity.

[0013] By adopting the above scheme, the actual complexity of the image background is considered to select whether to match the YOLO model with a PNA structure, so that images with a single background can adopt the PAN structure, strengthen the fusion of underlying detail features and high-level semantic features, improve the discrimination of similar targets, and ensure detection accuracy.

[0014] Preferably, it also includes: determining that the background complexity of the currently acquired photovoltaic power station image data exceeds the preset background complexity, and introducing an acceleration engine based on matching the YOLO model without a PAN structure to assist in accelerating detection.

[0015] By adopting the above solution, considering that complex backgrounds will increase the processing burden of the YOLO model, the introduction of an acceleration engine to assist detection can increase the target detection speed in complex backgrounds and further improve the timeliness and accuracy of drone autonomous navigation.

[0016] Preferably, it also includes: Collect meteorological data in real time and analyze whether the collected meteorological data contains preset extreme meteorological data; If so, while collecting the RGB image of the photovoltaic power station in real time, the depth image data and thermal imaging data are also collected synchronously; the simultaneously collected RGB image, depth image and thermal imaging data are synchronously input into the YOLO model that matches the current scene type for target detection instead of inputting the real-time collected image data of the photovoltaic power station into the YOLO model that matches the current scene type for target detection.

[0017] By adopting the above solution and combining RGB images, depth maps and thermal imaging data, the target recognition accuracy of drones in complex environments can be improved, especially under preset complex weather conditions, while maintaining stable detection performance.

[0018] Preferably, it also includes: When determining the undetected PV modules and taking the final positioning of the currently detected PV modules as the starting point, the flight trajectory of the UAV is re-planned according to the optimal path algorithm. The flight trajectory adjustment plan including the flight altitude and flight speed is dynamically matched in combination with the real-time collected illumination, occlusion and altitude / angle data; different illumination, occlusion and altitude / angle data combinations are pre-matched with flight trajectory adjustment plans including flight altitude and flight speed. The specific matched flight trajectory adjustment plan is determined by analyzing and statistically analyzing the flight altitude and flight speed of the UAV whose target detection and positioning accuracy is greater than the preset accuracy under different historical illumination, occlusion and altitude / angle data combinations; based on the generated flight trajectory, the flight altitude and flight speed are adjusted according to the matched aircraft trajectory adjustment plan.

[0019] By adopting the above solution, when replanning the UAV flight trajectory according to the optimal path algorithm, the flight trajectory adjustment plan is dynamically matched with the real-time collected illumination, occlusion and altitude / angle data, so that the UAV's flight altitude and flight speed can adapt to different environments, thereby improving the accuracy and safety of the UAV's autonomous navigation.

[0020] In a second aspect, the present application provides a UAV autonomous navigation system based on the YOLO visual neural network, comprising: Data acquisition module, used to collect real-time image data of the photovoltaic power station as well as illumination, occlusion, and height / angle data; The YOLO model matching module determines the scene type based on real-time collected illumination, occlusion, and height / angle data. It retrieves a YOLO model that matches the current scene type from the YOLO visual neural network model library. Each YOLO model uses the same Backbone and is trained using a differentiated prediction head based on the PV power plant image data corresponding to the scene type, and undergoes lightweight processing. The target detection and positioning module is used to hot-load the acquired YOLO model that matches the current scene type, input the real-time image data of the photovoltaic power station into the YOLO model that matches the current scene type for target detection, and obtain the initial location of the currently detected photovoltaic module; the detection results output by YOLO are post-processed, including positioning correction based on prior knowledge of the topological relationship of photovoltaic modules in the photovoltaic power station, to obtain the final location of the currently detected photovoltaic module; The positioning verification and navigation module is used to determine whether the trajectory generated based on the final positioning of the currently detected photovoltaic module and the final positioning of the photovoltaic module detected at the previous moment meets the expected trajectory; if it meets the expected trajectory, navigation continues according to the expected trajectory; otherwise, the final positioning of all photovoltaic modules obtained by statistical analysis is used to determine the undetected photovoltaic modules, and the final positioning of the currently detected photovoltaic module is used as the starting point to re-plan the drone's flight trajectory and navigate using the optimal path algorithm.

[0021] By adopting the above solution, real-time data on photovoltaic power plants is collected and a lightweight YOLO model is matched to the perception data. This enables adaptive target detection in different scenarios, improving detection efficiency and accuracy. Detection results are corrected using the topological relationship of photovoltaic modules to improve positioning accuracy. Verifying the trajectory generated by positioning and replanning the flight trajectory as needed ensures the efficiency and accuracy of autonomous navigation of drones.

[0022] In a third aspect, the present application provides a computer-readable storage medium, which includes a stored computer program, wherein when the computer program is running, the device where the computer-readable storage medium is located is controlled to execute the method as described above.

[0023] In a fourth aspect, the present application provides a computer device, which includes a memory, a processor, and a program stored and executable on the memory, and the program implements the steps of the above method when executed by the processor.

[0024] In summary, this application has the following beneficial effects: 1. Real-time collection of photovoltaic power station perception data to determine scene types, matching the YOLO model, and achieving adaptive target detection for different scenarios, improving detection efficiency and accuracy. A lightweight YOLO model is used for hot loading, enabling fast and accurate target detection on resource-constrained drone platforms, verifying the accuracy of the trajectory generated by positioning, and replanning the flight trajectory as needed, assisting in more efficient autonomous navigation. 2. Combining real-time perception data, a dynamic batch processing method is used to perform target detection on multiple frames of image data of the same scene type. This optimizes the target detection algorithm on resource-constrained UAV platforms, reduces computing resource consumption, and improves the overall operating efficiency of the system. 3. Pre-input the real-time collected photovoltaic power station image data into the CNN convolution model to obtain background complexity and understand the complexity of the image; based on the complexity of the collected image, match the YOLO variant model with a PAN structure or a YOLO variant model with an acceleration engine. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] Figure 1Flowchart of the autonomous navigation method of the UAV based on the YOLO visual neural network described in the specific embodiment; Figure 2 Schematic diagram of the structure of the UAV autonomous navigation system based on the YOLO visual neural network described in the specific embodiment. DETAILED DESCRIPTION

[0026] In order to make the purpose, technical solutions and advantages of this application more clear, the following further describes this application in detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain this application and are not intended to limit this application.

[0027] like Figure 1 As shown, the embodiment of the present application discloses a UAV autonomous navigation method based on the YOLO visual neural network. In order to achieve fast and accurate target detection on a resource-constrained UAV platform, a lightweight YOLO visual neural network model library is integrated, and a suitable YOLO model is called in combination with dynamic and complex scene perception data. Target detection and positioning are performed efficiently and accurately in different scenarios, assisting UAV photovoltaic inspection navigation. The specific steps include: S1. Real-time collection of image data and perception data of photovoltaic power stations.

[0028] Use a high-definition camera installed on a drone platform to collect image data of the photovoltaic power station; Utilize sensors installed on the drone platform to collect perception data such as illumination, occlusion, and altitude / angle in real time. Specifically, use illumination sensors to collect illumination intensity, and collect illumination intensity corresponding to different times (such as dawn and dusk) and weather conditions (such as sunny and rainy). Use occlusion sensors (such as lidar) to collect information about target object occlusion. Use altitude sensors and angle sensors to collect the current altitude and angle of the drone.

[0029] S2. Determine the scene type based on the real-time perception data collected, and schedule a YOLO model that matches the current scene type from the YOLO visual neural network model library.

[0030] Specifically, the scene type is determined using a machine learning classification model (such as a decision tree algorithm or a support vector machine algorithm) stored in the drone's computing processing center. This model is trained using historically collected perception data and annotated scene types. In this embodiment, specific scenes include high-altitude scenes (the quantitative indicator is the altitude value, with scenes greater than 30 meters considered high-altitude), strong reflective scenes (the quantitative indicator is the image grayscale entropy value, with scenes greater than 7.2 considered strong reflective), severely occluded scenes (the quantitative indicator is the occlusion area / image area, with scenes less than 0.35 considered severely occluded), and other scenes.

[0031] In order to better cope with the detection of targets in complex scene conditions (such as photovoltaic modules in photovoltaic power stations), the adaptive selection focuses on the YOLO model for target detection under corresponding scene conditions. Accordingly, the UAV platform integrates and stores a YOLO visual neural network model library. Each YOLO model uses the same Backbone. Backbone is a network structure used for feature extraction, with Darknet being a common example. This effectively extracts image feature information. Each YOLO model's prediction head is trained using differentiated data from photovoltaic power plant images corresponding to the scene type, enabling more effective object detection in specific scenarios. Each YOLO model undergoes lightweight processing, using techniques such as model quantization and pruning to reduce the number of model parameters and computational complexity, making it suitable for resource-constrained drone platforms.

[0032] In this embodiment, the YOLO visual neural network model library includes the YOLO-FAST model that matches high-altitude scenes, and can use low-resolution samples as training data; the YOLO-Reflect model that matches strong reflective scenes, and can use 80% strong reflective samples + 20% regular lighting samples as training data, and add a channel attention module to enhance the reflective edges for fine-tuning the head structure; the YOLO-Occlusion model that matches severe occlusion scenes, and can use 60% strong occlusion samples + 40% regular non-severe occlusion samples as training data, and add void convolution to expand the receptive field to penetrate the occlusion for fine-tuning the head structure; and the YOLO-Precision model that matches other scenes, etc.

[0033] Furthermore, after completing the scene type judgment based on the real-time collected lighting, occlusion and height / angle data, the YOLO model that matches the current scene type is retrieved from the YOLO visual neural network model library for the judged scene type.

[0034] In addition, in order to improve detection accuracy, the image preprocessing model stored in the drone's computing processing center is used to perform image preprocessing on the collected photovoltaic image data, including: adaptive histogram equalization to enhance local contrast, targeted color space conversion to process lighting and shadows, or real-time deblurring, etc.

[0035] S3: Hot load and apply the YOLO model that matches the current scene type for target detection to obtain the initial positioning of the currently detected photovoltaic module.

[0036] Specifically, a YOLO model matching the current scene type is pre-hot loaded, similar to loading program code from the hard disk into memory for fast execution when a computer starts a program. Then, the real-time acquired photovoltaic power station image data (or photovoltaic power station image data after image preprocessing) is input into the model for target detection to obtain the initial positioning of the currently detected photovoltaic module.

[0037] Among them, the matching YOLO model is used to detect photovoltaic modules, and the bounding box coordinates, the pixel coordinates of the photovoltaic module center point, the corner coordinates of the photovoltaic module photovoltaic panel, and the confidence level of the photovoltaic module ID are output.

[0038] S4. Post-process the obtained initial position of the photovoltaic component currently being detected to obtain the final position of the photovoltaic component currently being detected.

[0039] In order to solve the problem of positioning errors caused by similar photovoltaic panel components, the candidate component set obtained by YOLO detection is combined with topological map matching to obtain the optimal ID allocation and realize positioning coordinate optimization.

[0040] Specifically, post-processing of the YOLO detection results involves pre-digital mapping of the PV plant's PV array, topological modeling, and storage in each YOLO model. Bounding box corrections are then performed based on prior knowledge of the PV plant's topological relationships (e.g., topological constraints such as constant horizontal spacing between panels in a row, vertical alignment of panels in a column, and consistent orientation of panels). This final positioning of the currently detected PV panel is achieved. For example, if the bounding box of a detected PV panel overlaps with that of an adjacent panel or the spacing is unreasonably large, adjustments are made using topological relationships.

[0041] It also includes: converting the topological constraints into a graph optimization problem, solving the optimal position by minimizing the following error function, including: setting an objective function that minimizes the geometric deviation between observation and topology, solving the objective function to obtain the optimal photovoltaic ID, etc.

[0042] S5. Determine whether the yaw occurs based on the final positioning of the currently detected photovoltaic module, and perform automatic navigation of the UAV based on the result of whether the yaw occurs or not.

[0043] Specifically, based on the final positioning of the photovoltaic component currently being detected, in order to verify whether the photovoltaic component currently being detected is yawed, it is necessary to combine the final positioning of the photovoltaic component detected at the previous moment, and generate a trajectory according to the final positioning of the photovoltaic component currently being detected and the final positioning of the photovoltaic component detected at the previous moment, such as: using the center pixel coordinates in the final positioning of the photovoltaic component detected at each previous moment as the waypoint, and the center pixel coordinates in the final positioning of the photovoltaic component currently being detected as the sequential waypoints, and then generating the trajectory in the order of acquisition. In addition, if there are many photovoltaic components detected at each moment, that is, there are several photovoltaic components currently being detected, considering the inspection of a large number of photovoltaic components, in order to fully detect the photovoltaic components, the waypoint is generally set at the center position of the photovoltaic component to be detected. When the drone passes through the waypoint, it will cover and collect several photovoltaic components, and then select the photovoltaic component at the center position of the collected image as the target photovoltaic component, and use the center position of the target photovoltaic component as the waypoint.

[0044] Determine whether the trajectory generated based on the final positioning of the currently detected PV module and the final positioning of the PV module detected at the previous moment meets the expected trajectory; the expected trajectory is the flight path generated by the preset route planning algorithm, including the starting point, end point and waypoints; if it meets the expected trajectory, continue to navigate according to the expected trajectory; otherwise, count the final positioning of all PV modules obtained by detection, determine the undetected PV modules, and use the final positioning of the currently detected PV module as the starting point to re-plan the UAV flight trajectory according to the optimal path algorithm and navigate.

[0045] In addition, considering the influence of meteorological data, adjustments need to be made during the re-planning of the UAV flight trajectory to collect clearer photovoltaic module image data. The method further includes: After identifying uninspected PV modules and using the final positioning of the currently inspected PV modules as a starting point, the drone's flight trajectory is replanned according to the optimal path algorithm. A flight trajectory adjustment plan, including flight altitude and speed, is dynamically matched with real-time collected illumination, occlusion, and altitude / angle data. Based on the generated flight trajectory, the altitude and speed are adjusted according to the matched aircraft trajectory adjustment plan. Different combinations of illumination, occlusion, and altitude / angle data (i.e., combinations of illumination, occlusion, and altitude / angle perception data) are pre-matched with flight trajectory adjustment plans, including flight altitude and speed. The specific matched flight trajectory adjustment plan can be determined by analyzing and statistically analyzing the drone's flight altitude and speed, which exhibit greater accuracy than a preset accuracy under different historical combinations of illumination, occlusion, and altitude / angle data.

[0046] In a specific embodiment, considering that the currently identified scene type is not a single scene type, a dynamic scheduling strategy is designed to improve the positioning accuracy and target detection performance of the UAV for photovoltaic modules in complex environments, thereby enhancing the adaptability and reliability of the UAV's autonomous navigation, and further improving the efficiency and stability of the UAV's autonomous navigation in various complex scenarios. The method also includes: When the scene type is determined based on the real-time collected illumination, occlusion, and height / angle data, the determined scene type is not a single scene type, and all single scene types are obtained; for example: the current detection is in a strong reflective scene, a severely occluded scene, and a high-altitude scene; Design a hierarchical trigger mechanism and determine the priority trigger judgment order of each scene type, and set the priority trigger conditions for the corresponding perception data range of each single scene type; specifically, according to the characteristics of different photovoltaic power stations, the frequency of scene types in historical inspections is counted to set the hierarchical trigger order of different scene types. The ones with higher frequencies can be judged first whether to trigger the priority, and then determine whether to trigger the priority conditions based on the range of perception data corresponding to the scene type.

[0047] In this embodiment, the trigger condition judgment level of a single scene type is set preferentially, and the levels from high to low are: severe occlusion scene type, strong reflection scene type, high altitude scene type and other scene types; priority trigger conditions are set for the corresponding perception data range (i.e., the numerical range of quantitative indicators) of each scene type, and trigger conditions are not set separately for other scene types. Only when the severe occlusion scene type, strong reflection scene type and high altitude scene type do not meet the priority trigger conditions, they are determined to have the highest priority; for example: priority is given to judging whether the ratio of the occlusion area / image area in the severe occlusion scene is greater than the first occlusion area / image area threshold (e.g., 0.4); if it is greater, the priority trigger condition of the current scene type is met, and the severe occlusion scene type has the highest priority; otherwise, continue to judge the strong reflection scene type. Whether the image grayscale entropy value in the light scene is greater than the first image grayscale entropy value threshold (such as 0.8), if so, the current scene type priority trigger condition is met, and the strong reflective scene has the highest priority; otherwise, continue to judge whether the height value in the high-altitude scene is greater than the first height threshold (such as 35), if so, the current scene type priority trigger condition is met, and the high-altitude scene type has the highest priority; otherwise, other scene types have the highest priority; and after the current highest priority is determined, the scene type corresponding to the current highest priority is excluded, and priority judgment is continued according to the level. The highest level of subsequent judgments defaults to the secondary priority of the highest level of the previous judgment, and when no scene type triggers the priority judgment in subsequent judgments, they are randomly sorted to finally obtain the scene type priority.

[0048] Based on the collected perception data and the corresponding perception data range of each single scene type, the priority of the triggered scene type is determined and obtained; According to the priority order of the triggered scene types, the YOLO model corresponding to the single scene type matching is switched and hot-loaded in sequence; the initial positioning of the photovoltaic component currently detected by the target detection output of the YOLO model matching the single scene type is obtained respectively, and positioning fusion is performed according to the preset fusion positioning engine rules to obtain the initial positioning of the fused photovoltaic component; the preset fusion positioning engine rules include weighted fusion rules or clustering fusion rules, and the higher the priority order, the higher the corresponding weighted weight.

[0049] In a specific embodiment, in order to avoid unnecessary consumption of computing resources and to ensure timeliness to a certain extent, the method further includes: Monitor the real-time illumination, occlusion, and height / angle data. If the fluctuation values ​​of the illumination, occlusion, and height / angle perception data collected during consecutive preset time periods are all less than the corresponding preset fluctuation thresholds, it is determined that they belong to the same scene type and there is no need to switch the YOLO model for hot loading. In order to ensure timeliness, the preset time period should not be too long. Choose 2t-3t, where t is the sampling time.

[0050] A dynamic batch processing method is selected to input the image data of multiple frames of photovoltaic power stations corresponding to the same scene type into the YOLO model that matches the current scene type for target detection to replace the input of the real-time image data of the photovoltaic power station into the YOLO model that matches the current scene type for target detection; the dynamic batch processing method includes: setting a basic batch processing size; monitoring the processing performance during the target detection process for the image data of multiple frames of photovoltaic power stations, including: processing time, memory occupancy and CPU / GPU utilization; dynamically increasing or decreasing the batch processing size based on the basic batch processing size according to the preset processing performance range of the monitored processing performance; in this embodiment, when the preset processing performance range of the monitored processing performance is the first preset processing performance range, the batch size is dynamically reduced based on the basic batch processing size. , such as: the processing time delay is greater than 100ms, the memory usage is greater than 80%, and the CPU / GPU utilization is greater than 80%; or when the monitored processing performance is within the preset processing performance range of the second preset processing performance range, the batch size is dynamically increased based on the basic batch size, such as: the processing time delay is less than 1ms, the memory usage is less than 50%, and the CPU / GPU utilization is less than 60%.

[0051] The initial positioning of several frames of photovoltaic components output by batch processing is obtained and the detection results output by YOLO are post-processed to obtain the final positioning of several frames of photovoltaic components output by batch processing. It is judged whether the flight trajectory jointly generated according to the final positioning of several frames of photovoltaic components output by different batch processing in the batch processing order and the final positioning of several frames of photovoltaic components output by the same batch processing in the corresponding frame acquisition order meets the expected trajectory, instead of judging whether the trajectory generated according to the final positioning of the photovoltaic component currently detected and the final positioning of the photovoltaic component detected at the previous moment meets the expected trajectory.

[0052] In a specific embodiment, a suitable model is selected for target detection based on the actual complexity of the image background to improve the accuracy and efficiency of detection, thereby helping to enhance the target detection performance of drones during photovoltaic power station inspections, thereby improving the real-time and safety of autonomous navigation. The method further includes: Before inputting the real-time image data of the photovoltaic power station into the YOLO model matching the current scene type for target detection, the real-time image data of the photovoltaic power station is pre-input into the CNN convolution model pre-existing in the computing processing center of the drone platform to obtain the background complexity of the currently collected photovoltaic power station image data; After retrieving the YOLO model that matches the current scene type from the YOLO visual neural network model library based on the judged scene type, each YOLO model is designed to include a YOLO model with a PAN structure and a YOLO model without a PAN structure. The YOLO model with or without a PAN structure is then matched based on the background complexity of the currently acquired photovoltaic power station image data. This includes matching the YOLO model without a PAN structure when the background complexity of the currently acquired photovoltaic power station image data is greater than the preset background complexity. When the background complexity of the image data is high, it helps to accurately identify the current photovoltaic module by combining background features. When the background complexity of the image data is not high, it is necessary to design a YOLO model with a PAN structure. That is, based on YOLO's original FPN (Feature Pyramid Network), a PAN (Path Aggregation Network) structure is added to strengthen the fusion of underlying detail features (such as the texture of the photovoltaic panel module) and high-level semantic features (such as the overall shape of the photovoltaic panel module), thereby improving the discrimination of similar targets.

[0053] In addition, considering that if the image contains multiple elements, textures or a background with rich color changes, such as trees, buildings, and other equipment, this may increase the processing burden of the YOLO model, resulting in a decrease in detection speed, the inference speed of the YOLO model is accelerated by the acceleration engine. The method also includes: for the result of determining that the background complexity of the currently acquired photovoltaic power station image data exceeds the preset background complexity, and choosing to introduce an acceleration engine on the basis of matching the YOLO model without a PAN structure to assist in accelerating detection.

[0054] In a specific embodiment, in order to enhance the target detection capability of the UAV under adverse weather conditions, improve the accuracy and stability of target detection, enable the UAV to effectively perform autonomous navigation in complex weather environments, and expand the application scenarios and working conditions of the UAV, the method further includes: Collect meteorological data in real time and analyze whether the collected meteorological data contains preset extreme meteorological data, including: preset extreme meteorological data exceeding the preset precipitation and snowfall threshold data; for example: precipitation and snowfall exceeding the preset precipitation and snowfall threshold data; If available, while collecting the RGB image of the photovoltaic power station in real time, simultaneously use the thermal imaging camera installed on the drone platform to collect thermal imaging data and the binocular camera installed on the drone platform to collect depth image data; The simultaneously collected RGB images, depth images, and thermal imaging data are synchronously input into the YOLO model that matches the current scene type for target detection. Target detection is performed by fusing feature data instead of inputting the real-time image data of the photovoltaic power station into the YOLO model that matches the current scene type for target detection.

[0055] like Figure 2 As shown, the embodiment of the present application discloses an autonomous navigation system for a drone based on a YOLO visual neural network, specifically comprising: The data acquisition module 101 is used to collect image data of the photovoltaic power station as well as illumination, shading and height / angle data in real time; YOLO model matching module 102 is used to determine the scene type based on the real-time collected illumination, occlusion, and height / angle data. Based on the determined scene type, a YOLO model matching the current scene type is retrieved from the YOLO visual neural network model library. Each YOLO model uses the same Backbone and is trained using a differentiated prediction head based on the photovoltaic power station image data corresponding to the scene type, and undergoes lightweight processing. The target detection and positioning module 103 is used to hot-load the acquired YOLO model that matches the current scene type, input the real-time image data of the photovoltaic power station into the YOLO model that matches the current scene type to perform target detection, and obtain the initial positioning of the photovoltaic module currently being detected; post-process the detection results output by YOLO, including performing positioning correction based on prior knowledge of the topological relationship of photovoltaic modules in the photovoltaic power station, and obtain the final positioning of the photovoltaic module currently being detected; The positioning verification and navigation module 104 is used to determine whether the trajectory generated based on the final positioning of the currently detected photovoltaic component and the final positioning of the photovoltaic component detected at the previous moment meets the expected trajectory; if it meets the expected trajectory, navigation continues according to the expected trajectory; otherwise, the final positioning of all photovoltaic components obtained by statistical analysis is determined, and the undetected photovoltaic components are used as the starting point. The optimal path algorithm is used to re-plan the flight trajectory of the drone and perform navigation.

[0056] In a specific embodiment, the target detection and positioning module 103 in the system is also used to complete scene type judgment based on the illumination, occlusion and height / angle data collected in real time, and the judged scene type is not a single scene type, and all single scene types are obtained; a hierarchical trigger mechanism is designed and the priority trigger judgment order of each scene type is determined, and priority trigger conditions are set for multiple single scene types corresponding to the illumination, occlusion and height / angle data ranges; according to the collected illumination, occlusion and height / angle data, the priority of the triggered scene type is judged and obtained; according to the priority order of the triggered scene type, the YOLO model corresponding to the single scene type is sequentially switched and hot-loaded; the initial positioning of the photovoltaic component currently detected by the YOLO model matching the single scene type is respectively obtained for target detection output, and positioning fusion is performed according to the preset fusion positioning engine rules to obtain the initial positioning of the fused photovoltaic component; the preset fusion positioning engine rules include weighted fusion rules, and the higher the priority order, the higher the weighted weight.

[0057] In a specific embodiment, the target detection and positioning module 103 in the system is further used to monitor the illumination, occlusion and height / angle data collected in real time. If the fluctuation values ​​of the illumination, occlusion and height / angle perception data collected in consecutive preset time periods are all less than the corresponding preset fluctuation thresholds, it is determined that they belong to the same scene type, then there is no need to switch the YOLO model for hot loading, and a dynamic batch processing method is selected to input the image data of the photovoltaic power station corresponding to the same scene type into the YOLO model matching the current scene type for target detection to replace the input of the image data of the photovoltaic power station collected in real time into the YOLO model matching the current scene type for target detection; the dynamic batch processing method includes: setting a basic batch size; during the target detection process for the image data of the photovoltaic power station, monitoring the processing performance, including: processing time, memory occupancy and CPU / GPU utilization; dynamically increasing or decreasing the batch size based on the basic batch size according to whether the monitored processing performance is in the preset processing performance range; The positioning verification and navigation module 104 is further used to obtain the initial positioning of several frames of photovoltaic components output by batch processing and post-process the detection results output by YOLO, obtain the final positioning of several frames of photovoltaic components output by batch processing, and judge whether the trajectory generated according to the final positioning of several frames of photovoltaic components output by different batch processing in the batch processing order and the final positioning of several frames of photovoltaic components output by the same batch processing in the corresponding frame acquisition order conforms to the expected trajectory, instead of judging whether the trajectory generated according to the final positioning of the photovoltaic component currently detected and the final positioning sequence of the photovoltaic component detected at the previous moment conforms to the expected trajectory.

[0058] In a specific embodiment, the YOLO model matching module 102 in the system is further configured to input the real-time collected image data of the photovoltaic power station into the CNN convolutional model before inputting the real-time collected image data of the photovoltaic power station into the YOLO model matching the current scene type for target detection, so as to obtain the background complexity of the currently collected image data of the photovoltaic power station; after retrieving the YOLO model matching the current scene type from the YOLO visual neural network model library based on the judged scene type, design each YOLO model to include a YOLO model with a PAN structure and a YOLO model without a PAN structure, and continue to match the YOLO model with or without a PAN structure based on the background complexity of the currently collected image data of the photovoltaic power station, including matching the YOLO model without a PAN structure when the background complexity of the currently collected image data of the photovoltaic power station is greater than a preset background complexity; and further configured to, when it is determined that the background complexity of the currently collected image data of the photovoltaic power station exceeds the preset background complexity, select to introduce an acceleration engine based on matching the YOLO model without a PAN structure to assist in accelerated detection.

[0059] In a specific embodiment, the data acquisition module 101 in the system is further configured to collect meteorological data in real time and analyze whether the collected meteorological data contains preset extreme meteorological data; if so, while collecting the RGB image of the photovoltaic power station in real time, it also simultaneously collects depth image data and thermal imaging data; The YOLO model matching module 102 is further configured to synchronously input the simultaneously acquired RGB image, depth image, and thermal imaging data into a YOLO model that matches the current scene type for target detection, instead of inputting the real-time acquired photovoltaic power station image data into a YOLO model that matches the current scene type for target detection.

[0060] In a specific embodiment, the positioning verification and navigation module 104 in the system is also used to dynamically match a flight trajectory adjustment scheme including flight altitude and flight speed in combination with the real-time collected illumination, occlusion and altitude / angle data in the process of determining undetected photovoltaic modules and replanning the drone flight trajectory based on the optimal path algorithm with the final positioning of the currently detected photovoltaic modules as the starting point; different illumination, occlusion and altitude / angle data combinations are pre-matched with flight trajectory adjustment schemes including flight altitude and flight speed, and the specific matched flight trajectory adjustment scheme is determined by analyzing and statistically analyzing the drone flight altitude and flight speed whose target detection and positioning accuracy is greater than a preset accuracy under different historical illumination, occlusion and altitude / angle data combinations; based on the generated flight trajectory, the flight altitude and flight speed are adjusted according to the matched aircraft trajectory adjustment scheme.

[0061] The embodiment of the present application also discloses a computer-readable storage medium.

[0062] Specifically, the computer-readable storage medium stores a computer program that can be loaded by a processor and execute the above-mentioned drone autonomous navigation method based on the YOLO visual neural network. The computer-readable storage medium includes, for example, various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0063] The embodiment of the present application also discloses a computer device.

[0064] Specifically, the computer device includes a memory and a processor, and the memory stores a computer program that can be loaded by the processor and execute the above-mentioned drone autonomous navigation method based on the YOLO visual neural network.

[0065] The above are all preferred embodiments of the present application and are not intended to limit the scope of protection of this application. Unless otherwise stated, any feature disclosed in this specification (including the abstract and drawings) may be replaced by other equivalent or similar features. In other words, unless otherwise stated, each feature is merely an example of a series of equivalent or similar features.

Claims

1. A UAV autonomous navigation method based on YOLO visual neural network, characterized in that: include: Real-time collection of image data of photovoltaic power plants as well as illumination, occlusion, and height / angle data; The scene type is determined based on real-time collected illumination, occlusion, and height / angle data. A YOLO model matching the current scene type is retrieved from the YOLO visual neural network model library. Each YOLO model uses the same Backbone and is trained using a differentiated prediction head based on the PV power plant image data corresponding to the scene type, and undergoes lightweight processing. Hot-load the acquired YOLO model that matches the current scene type, input the real-time image data of the photovoltaic power station into the YOLO model that matches the current scene type for target detection, and obtain the initial location of the currently detected photovoltaic module; post-process the detection results output by YOLO, including positioning correction based on prior knowledge of the topological relationship of photovoltaic modules in the photovoltaic power station, to obtain the final location of the currently detected photovoltaic module; Determine whether the trajectory generated based on the final positioning of the currently detected PV module and the final positioning of the PV module detected at the previous moment meets the expected trajectory; if it meets the expected trajectory, continue to navigate according to the expected trajectory; otherwise, count the final positioning of all PV modules obtained by detection, determine the undetected PV modules, and use the final positioning of the currently detected PV module as the starting point to replan the UAV flight trajectory and navigate according to the optimal path algorithm.

2. The autonomous navigation method for unmanned aerial vehicles based on the YOLO visual neural network according to claim 1, characterized in that: Also includes: When the scene type is determined based on the real-time collected illumination, occlusion, and height / angle data, the determined scene type is not a single scene type, and all single scene types are obtained; Design a hierarchical trigger mechanism and determine the priority trigger judgment order for each scene type. Set priority trigger conditions for multiple single scene types corresponding to the lighting, occlusion, and height / angle data ranges. Determine and obtain the trigger scene type priority based on the collected lighting, occlusion, and height / angle data. According to the priority order of the triggered scene types, the YOLO model corresponding to the single scene type is switched and hot-loaded in sequence; the initial positioning of the photovoltaic module currently detected is obtained by the target detection output of the YOLO model matching the single scene type, and the positioning is fused according to the preset fusion positioning engine rules to obtain the fused initial positioning of the photovoltaic module; The preset fusion positioning engine rules include weighted fusion rules, and a higher priority order corresponds to a higher weighted weight.

3. The autonomous navigation method for unmanned aerial vehicles based on the YOLO visual neural network according to claim 1, characterized in that: Also includes: Monitor the real-time collected illumination, occlusion, and height / angle data. If the fluctuation values ​​of the illumination, occlusion, and height / angle perception data collected during consecutive preset time periods are all less than the corresponding preset fluctuation thresholds, it is determined that they belong to the same scene type. There is no need to switch the YOLO model for hot loading. Instead, a dynamic batch processing method is selected to input multiple frames of photovoltaic power station image data corresponding to the same scene type into the YOLO model that matches the current scene type for target detection, replacing the real-time collected photovoltaic power station image data input into the YOLO model that matches the current scene type for target detection; The dynamic batch processing method includes: setting a basic batch size; monitoring processing performance, including processing time, memory occupancy, and CPU / GPU utilization, during target detection of multiple frames of photovoltaic power station image data; and dynamically increasing or decreasing the batch size based on the basic batch size according to whether the monitored processing performance falls within a preset processing performance range; The initial positioning of several frames of photovoltaic components output by batch processing is obtained and the detection results output by YOLO are post-processed to obtain the final positioning of several frames of photovoltaic components output by batch processing. It is judged whether the trajectories generated according to the final positioning of several frames of photovoltaic components output by different batch processing in the batch processing order and the final positioning of several frames of photovoltaic components output by the same batch processing in the corresponding frame acquisition order meet the expected trajectory, instead of judging whether the trajectory generated according to the final positioning of the photovoltaic component currently detected and the final positioning of the photovoltaic component detected at the previous moment meet the expected trajectory.

4. The autonomous navigation method for unmanned aerial vehicles based on the YOLO visual neural network according to claim 1, characterized in that: Also includes: Before inputting the real-time image data of the photovoltaic power station into the YOLO model matching the current scene type for target detection, the real-time image data of the photovoltaic power station is pre-inputted into the CNN convolution model to obtain the background complexity of the currently collected image data of the photovoltaic power station; After retrieving the YOLO model that matches the current scene type from the YOLO visual neural network model library according to the judged scene type, each YOLO model is designed to include a YOLO model with a PAN structure and a YOLO model without a PAN structure, and then continue to match the YOLO model with or without a PAN structure according to the background complexity of the currently collected photovoltaic power station image data, including matching the YOLO model without a PAN structure when the background complexity of the currently collected photovoltaic power station image data is greater than the preset background complexity.

5. The autonomous navigation method for unmanned aerial vehicles based on the YOLO visual neural network according to claim 4, characterized in that: Also includes: In response to the result that the background complexity of the currently acquired photovoltaic power station image data exceeds the preset background complexity, an acceleration engine is introduced on the basis of matching the YOLO model without a PAN structure to assist in accelerated detection.

6. The autonomous navigation method for unmanned aerial vehicles based on the YOLO visual neural network according to claim 1, characterized in that: Also includes: Collect meteorological data in real time and analyze whether the collected meteorological data contains preset extreme meteorological data; If so, while collecting the RGB image of the photovoltaic power station in real time, the depth image data and thermal imaging data are also collected synchronously; the simultaneously collected RGB image, depth image and thermal imaging data are synchronously input into the YOLO model that matches the current scene type for target detection instead of inputting the real-time collected image data of the photovoltaic power station into the YOLO model that matches the current scene type for target detection.

7. The autonomous navigation method for unmanned aerial vehicles based on the YOLO visual neural network according to claim 1, characterized in that: Also includes: When determining the undetected PV modules and taking the final positioning of the currently detected PV modules as the starting point, the flight trajectory of the UAV is re-planned according to the optimal path algorithm. The flight trajectory adjustment plan including the flight altitude and flight speed is dynamically matched in combination with the real-time collected illumination, occlusion and altitude / angle data; different illumination, occlusion and altitude / angle data combinations are pre-matched with flight trajectory adjustment plans including flight altitude and flight speed. The specific matched flight trajectory adjustment plan is determined by analyzing and statistically analyzing the flight altitude and flight speed of the UAV whose target detection and positioning accuracy is greater than the preset accuracy under different historical illumination, occlusion and altitude / angle data combinations; based on the generated flight trajectory, the flight altitude and flight speed are adjusted according to the matched aircraft trajectory adjustment plan.

8. An autonomous navigation system for unmanned aerial vehicles based on the YOLO visual neural network, characterized in that: include: Data acquisition module, used to collect real-time image data of the photovoltaic power station as well as illumination, occlusion, and height / angle data; The YOLO model matching module determines the scene type based on real-time collected illumination, occlusion, and height / angle data. It retrieves a YOLO model that matches the current scene type from the YOLO visual neural network model library. Each YOLO model uses the same Backbone and is trained using a differentiated prediction head based on the PV power plant image data corresponding to the scene type, and undergoes lightweight processing. The target detection and positioning module is used to hot-load the acquired YOLO model that matches the current scene type, input the real-time image data of the photovoltaic power station into the YOLO model that matches the current scene type for target detection, and obtain the initial location of the currently detected photovoltaic module; the detection results output by YOLO are post-processed, including positioning correction based on prior knowledge of the topological relationship of photovoltaic modules in the photovoltaic power station, to obtain the final location of the currently detected photovoltaic module; The positioning verification and navigation module is used to determine whether the trajectory generated based on the final positioning of the currently detected photovoltaic module and the final positioning of the photovoltaic module detected at the previous moment meets the expected trajectory; if it meets the expected trajectory, navigation continues according to the expected trajectory; otherwise, the final positioning of all photovoltaic modules obtained by statistical analysis is used to determine the undetected photovoltaic modules, and the final positioning of the currently detected photovoltaic module is used as the starting point to re-plan the drone's flight trajectory and navigate using the optimal path algorithm.

9. A computer-readable storage medium, characterized in that The computer-readable storage medium includes a stored computer program, wherein when the computer program is executed, the device where the computer-readable storage medium is located is controlled to execute the method according to any one of claims 1 to 7.

10. A computer device, characterized in that: The computer device includes a memory, a processor, and a program stored and executable on the memory, and when the program is executed by the processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images