Autonomous Navigation Method and System for Unmanned Aerial Vehicles Based on YOLO Visual Neural Network

Abstract

This application discloses an autonomous navigation method and system for unmanned aerial vehicles (UAVs) based on a YOLO visual neural network. The method includes: real-time acquisition of image data and environmental perception data from a photovoltaic power station; scene type determination based on the real-time acquired environmental perception data and scheduling a YOLO model matching the current scene type; hot-loading the matched YOLO model, inputting the real-time acquired image data from the photovoltaic power station into the matched YOLO model for target detection, and obtaining the initial target location; post-processing the detection results output by the YOLO model to obtain the final target location; determining whether the trajectory generated based on the final location of the currently detected photovoltaic module and the final location of the photovoltaic module detected in the previous time step matches the expected trajectory; if it matches, navigation continues according to the expected trajectory; otherwise, the UAV flight trajectory is replanned and navigation is performed. This application enables rapid and accurate target detection on resource-constrained UAV platforms, improving autonomous navigation capabilities.

Description

Technical Field

[0001] This application relates to the technical field of unmanned aerial vehicle (UAV) navigation and target detection, specifically to an autonomous navigation method and system for UAVs based on the YOLO visual neural network. Background Technology

[0002] Currently, with the rapid development of technology, significant progress has been made in the field of autonomous navigation for unmanned aerial vehicles (UAVs). Target detection and tracking technologies based on computer vision have been widely applied, enabling UAVs to identify and track specific targets in complex environments.

[0003] In the field of autonomous navigation for unmanned aerial vehicles (UAVs), traditional methods employ various approaches to address target detection and navigation issues. For example, relying on convolutional neural networks (CNNs) for image recognition and target detection is a commonly used method. CNNs extract and learn image features through multiple convolutional layers, performing well in image classification tasks. They can identify target objects in images, determine their current location, and guide the UAV's flight based on pre-defined routes and rules, achieving autonomous navigation to a certain extent. However, despite their excellent performance in image classification, CNNs often struggle to meet the demands of real-time scenarios. Existing technologies utilize the YOLO model algorithm for image recognition, which can improve the real-time target detection and tracking capabilities of UAVs in dynamic environments to some extent. However, its high computational resource consumption limits its application on resource-constrained UAV platforms.

[0004] Therefore, how to achieve accurate and efficient positioning and navigation on resource-constrained drone application platforms has become an urgent technical problem to be solved. Summary of the Invention

[0005] To achieve rapid and accurate target detection on resource-constrained UAV platforms and improve the timeliness and accuracy of UAV autonomous navigation, this application provides a UAV autonomous navigation method and system based on YOLO visual neural networks.

[0006] Firstly, this application provides an autonomous navigation method for unmanned aerial vehicles (UAVs) based on a YOLO visual neural network, including:

[0007] Real-time acquisition of image data, as well as data on illumination, shading, and height / angle of photovoltaic power plants;

[0008] The scene type is determined based on the real-time collected illumination, occlusion and height / angle data. For the determined scene type, the YOLO model matching the current scene type is retrieved from the YOLO visual neural network model library. Each YOLO model uses the same backbone and is trained using the differential prediction head of the photovoltaic power station image data corresponding to the scene type, and undergoes lightweight processing.

[0009] The YOLO model obtained by hot loading matches the current scene type. The real-time image data of the photovoltaic power station is input into the YOLO model that matches the current scene type for target detection to obtain the initial location of the detected photovoltaic module. The detection results output by YOLO are post-processed, including the location correction based on the prior knowledge of the topological relationship of photovoltaic modules in the photovoltaic power station, to obtain the final location of the detected photovoltaic module.

[0010] Determine whether the trajectory generated based on the final location of the currently detected photovoltaic module matches the final location of the photovoltaic modules detected in the previous time step. If it matches the expected trajectory, continue navigation according to the expected trajectory. Otherwise, count the final locations of all detected photovoltaic modules, identify the undetected photovoltaic modules, and use the final location of the currently detected photovoltaic module as the starting point to re-plan and obtain the UAV flight trajectory according to the optimal path algorithm and perform navigation.

[0011] By adopting the above scheme, scene type is determined based on real-time collected data on illumination, occlusion, and altitude / angle, and a lightweight YOLO model is scheduled and matched, which improves the targeting and efficiency of photovoltaic module target detection in different scenarios; the detection results are corrected by using prior knowledge of topological relationships, which improves the accuracy of photovoltaic module positioning; and the flight trajectory is flexibly adjusted according to whether the positioning trajectory meets expectations, which enhances the flexibility and intelligence of UAV autonomous navigation.

[0012] Preferred options also include:

[0013] When the scene type is determined based on the real-time collected lighting, occlusion, and height / angle data, if the determined scene type is not a single scene type, all single scene types are obtained.

[0014] Design a hierarchical triggering mechanism and determine the priority triggering order for each scene type. Set priority triggering conditions for multiple single scene types corresponding to illumination, occlusion, and height / angle data ranges. Determine and obtain the priority of the triggered scene type based on the collected illumination, occlusion, and height / angle data.

[0015] According to the priority order of the triggered scene types, the YOLO models corresponding to the single scene type are switched and hot-loaded in sequence; the initial positioning of the currently detected photovoltaic module is obtained by the target detection output of the YOLO model matching the single scene type; the positioning is fused according to the preset fusion positioning engine rules to obtain the initial positioning of the photovoltaic module after fusion; the preset fusion positioning engine rules include weighted fusion rules, and the higher the priority order, the higher the corresponding weight.

[0016] By adopting the above scheme, when the scene type is not a single scene type, a hierarchical triggering mechanism is set to determine the priority order, and the corresponding YOLO model is hot-loaded and the positioning fusion is performed according to the priority order. This improves the positioning accuracy and target detection performance of UAVs for photovoltaic modules in complex environments, and enhances the adaptability and reliability of UAV autonomous navigation.

[0017] Preferred options also include:

[0018] The system monitors real-time collected data on illumination, occlusion, and height / angle. If the fluctuation values ​​of each sensing data point (illuminance, occlusion, and height / angle) collected over a consecutive preset time period are all less than the corresponding preset fluctuation threshold, and the data is determined to belong to the same scene type, then there is no need to switch the YOLO model for hot reloading. Instead, a dynamic batch processing method is adopted. This method inputs multiple frames of photovoltaic power station image data corresponding to the same scene type into a YOLO model that matches the current scene type for target detection, replacing the method of inputting real-time collected photovoltaic power station image data into a YOLO model that matches the current scene type for target detection. The dynamic batch processing method includes: setting a basic batch size; monitoring processing performance during target detection of multiple frames of photovoltaic power station image data, including processing time, memory occupancy, and CPU / GPU utilization; and dynamically increasing or decreasing the batch size based on the preset processing performance range corresponding to the monitored processing performance.

[0019] The system acquires several frames of initial photovoltaic module positioning from the batch output and performs post-processing on the detection results output by YOLO. It then acquires several frames of final photovoltaic module positioning from the batch output and determines whether the trajectory generated based on the final positioning of several frames of photovoltaic modules from different batches in the batch order and the final positioning of several frames of photovoltaic modules from the same batch in the corresponding frame acquisition order matches the expected trajectory. This is used as an alternative to determining whether the trajectory generated based on the final positioning of the currently detected photovoltaic module matches the final positioning order of the photovoltaic modules detected in the previous time step.

[0020] By adopting the above scheme, real-time monitoring of the sensing data and determination that the sensing data is in a stable state can eliminate the need to switch the YOLO model for hot loading. The dynamic batch processing method can be used to perform target detection on multiple frames of image data of the same scene type, reducing the resource consumption and time cost caused by real-time hot loading and model detection. Furthermore, the batch size can be dynamically adjusted according to the processing performance, making full use of system resources and improving processing efficiency.

[0021] Preferred options also include:

[0022] Before inputting the real-time acquired image data of the photovoltaic power station into the YOLO model that matches the current scene type for target detection, the real-time acquired image data of the photovoltaic power station is pre-input into the CNN convolutional model to obtain the background complexity of the currently acquired image data of the photovoltaic power station;

[0023] After retrieving a YOLO model from the YOLO visual neural network model library that matches the current scene type based on the determined scene type, each YOLO model is designed to include both a YOLO model with a PAN structure and a YOLO model without a PAN structure. The matching process continues based on the background complexity of the currently acquired photovoltaic power station image data, including matching a YOLO model without a PAN structure if the background complexity of the currently acquired photovoltaic power station image data is greater than a preset background complexity.

[0024] By adopting the above scheme, the actual complexity of the image background is considered when selecting whether to design a YOLO model with a PNA structure for matching. This allows images with a simple background to use the PAN structure, which strengthens the fusion of low-level detail features and high-level semantic features, improves the distinguishability of similar targets, and ensures detection accuracy.

[0025] Preferably, it also includes: for the result that the background complexity of the currently acquired photovoltaic power station image data exceeds the preset background complexity, and selects to introduce an acceleration engine to assist in accelerating detection based on matching the YOLO model without a PAN structure.

[0026] By adopting the above scheme, considering that complex backgrounds will increase the processing burden of the YOLO model, an acceleration engine is introduced to assist in detection, which can improve the target detection speed in complex backgrounds and further enhance the timeliness and accuracy of UAV autonomous navigation.

[0027] Preferred options also include:

[0028] Real-time collection of meteorological data and analysis of whether the collected meteorological data contains preset extreme weather data;

[0029] If applicable, while acquiring RGB images of the photovoltaic power station in real time, depth image data and thermal imaging data are simultaneously acquired; the simultaneously acquired RGB images, depth images, and thermal imaging data are then input into a YOLO model that matches the current scene type for target detection, instead of inputting the real-time acquired photovoltaic power station image data into a YOLO model that matches the current scene type for target detection.

[0030] By adopting the above scheme and combining RGB images, depth maps and thermal imaging data, the target recognition accuracy of UAVs in complex environments can be improved, especially under pre-set complex weather conditions, it can still maintain stable detection performance.

[0031] Preferred options also include:

[0032] In the process of identifying undetected photovoltaic modules and using the final location of the currently detected photovoltaic modules as a starting point, the drone's flight trajectory is replanned and obtained according to the optimal path algorithm. This involves dynamically matching flight trajectory adjustment schemes, including flight altitude and speed, based on real-time collected data on illumination, occlusion, and altitude / angle. Different combinations of illumination, occlusion, and altitude / angle data are pre-matched with flight trajectory adjustment schemes, including flight altitude and speed. The specific matched flight trajectory adjustment scheme is determined based on historical data combinations of illumination, occlusion, and altitude / angle, analyzing and statistically determining the drone's flight altitude and speed where the drone's target detection and positioning accuracy is greater than the preset accuracy. Based on the generated flight trajectory, the flight altitude and speed are adjusted according to the matched flight trajectory adjustment scheme.

[0033] By adopting the above scheme, when replanning the UAV flight trajectory according to the optimal path algorithm, the flight trajectory adjustment scheme is dynamically matched by combining real-time collected lighting, occlusion and altitude / angle data. This allows the UAV's flight altitude and speed to adapt to different environments, improving the accuracy and safety of the UAV's autonomous navigation.

[0034] Secondly, this application provides an autonomous navigation system for unmanned aerial vehicles (UAVs) based on a YOLO visual neural network, comprising:

[0035] The data acquisition module is used to collect image data, as well as data on illumination, shading, and height / angle of the photovoltaic power station in real time.

[0036] The YOLO model matching module is used to determine the scene type based on the real-time collected illumination, occlusion and height / angle data. For the determined scene type, it retrieves the YOLO model that matches the current scene type from the YOLO visual neural network model library. Each YOLO model uses the same backbone and is trained using the differential prediction head of the photovoltaic power station image data corresponding to the matched scene type, and undergoes lightweight processing.

[0037] The target detection and localization module is used to hot-load the YOLO model that matches the current scene type. It inputs real-time acquired image data of the photovoltaic power station into the YOLO model that matches the current scene type to perform target detection and obtain the initial localization of the detected photovoltaic module. It then performs post-processing on the detection results output by YOLO, including localization correction based on prior knowledge of the topological relationship of photovoltaic modules in the photovoltaic power station, to obtain the final localization of the detected photovoltaic module.

[0038] The positioning verification and navigation module is used to determine whether the trajectory generated based on the final positioning of the currently detected photovoltaic module and the final positioning of the photovoltaic modules detected in the previous time step matches the expected trajectory. If it matches the expected trajectory, navigation continues according to the expected trajectory. Otherwise, based on the final positioning of all detected photovoltaic modules, the module identifies the undetected photovoltaic modules, uses the final positioning of the currently detected photovoltaic module as the starting point, and re-plans the drone's flight trajectory using the optimal path algorithm for navigation.

[0039] By adopting the above scheme, real-time collection of relevant data from photovoltaic power plants and matching of lightweight YOLO models with the perceived data can achieve adaptive target detection in different scenarios, improving detection efficiency and accuracy; using the topological relationship of photovoltaic modules to correct detection results can improve positioning accuracy; verifying the trajectory generated by positioning and replanning the flight trajectory as needed can ensure the efficiency and accuracy of UAV autonomous navigation.

[0040] Thirdly, this application provides a computer-readable storage medium including a stored computer program, wherein the computer program, when running, controls the device where the computer-readable storage medium is located to perform the method described above.

[0041] Fourthly, this application provides a computer device, the computer device including a memory, a processor and a program stored in the memory and executable thereon, the program being executed by the processor to implement the steps of the method described above.

[0042] In summary, this application has the following beneficial effects:

[0043] 1. Real-time acquisition of photovoltaic power station perception data determines the scene type, matches the YOLO model, and achieves adaptive target detection for different scenes, improving detection efficiency and accuracy; and adopts a lightweight YOLO model for hot loading, enabling fast and accurate target detection on resource-constrained UAV platforms, verifying the accuracy of the positioning-generated trajectory and replanning the flight trajectory as needed, assisting in achieving more efficient autonomous navigation;

[0044] 2. By combining real-time perception data, a dynamic batch processing method is adopted to perform target detection on multiple frames of image data of the same scene type. The target detection algorithm is optimized on the resource-constrained UAV platform, reducing the consumption of computing resources and improving the overall operating efficiency of the system.

[0045] 3. Input the real-time acquired photovoltaic power station image data into the CNN convolutional model in advance to obtain the background complexity and understand the complexity of the image; combine the complexity of the acquired image to match whether the YOLO variant model has a PAN structure or whether the YOLO variant model introduces an acceleration engine. Attached Figure Description

[0046] Figure 1 This is a flowchart of the UAV autonomous navigation method based on YOLO visual neural network described in a specific embodiment;

[0047] Figure 2 This is a schematic diagram of the structure of the UAV autonomous navigation system based on the YOLO visual neural network described in a specific embodiment. Detailed Implementation

[0048] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0049] like Figure 1 As shown in the figure, this application discloses an autonomous navigation method for unmanned aerial vehicles (UAVs) based on YOLO visual neural networks. To achieve fast and accurate target detection on resource-constrained UAV platforms, a lightweight YOLO visual neural network model library is integrated. Suitable YOLO models are called in conjunction with dynamic and complex scene perception data to efficiently and accurately perform target detection and localization in different scenarios, assisting UAV photovoltaic inspection navigation. Specific steps include:

[0050] S1. Real-time acquisition of image and sensing data from photovoltaic power plants.

[0051] High-definition cameras installed on drone platforms are used to collect image data of photovoltaic power plants;

[0052] Using sensors installed on the drone platform, real-time sensing data such as illumination, occlusion, and altitude / angle are collected. Specifically, illumination sensors are used to collect illumination intensity, with different illumination intensities collected at different times (such as dawn and dusk) and in different weather conditions (such as sunny and rainy). Occlusion sensors (such as lidar) are used to collect the occlusion status of the target object. Altitude and angle sensors are used to collect the current altitude and angle of the drone.

[0053] S2. Determine the scene type based on the real-time collected perception data, and schedule a YOLO model that matches the current scene type from the YOLO visual neural network model library.

[0054] Specifically, the machine learning classification model (such as decision tree algorithm or support vector machine algorithm) stored in the drone's computing processing center is used to determine the scene type, and the model is trained using historically collected perception data and labeled scene types. In this embodiment, specific scenes include high-altitude scenes (quantified by altitude value, greater than 30m is considered a high-altitude scene), highly reflective scenes (quantified by image grayscale entropy value, greater than 7.2 is considered a highly reflective scene), severely occluded scenes (quantified by occlusion area / image area, less than 0.35 is considered a severely occluded scene), and other scenes.

[0055] To better handle target detection in complex scenarios (such as photovoltaic modules in a photovoltaic power station), adaptive selection focuses on YOLO models suited to the specific scenario conditions. Accordingly, a YOLO visual neural network model library is integrated and stored on the UAV platform, with each YOLO model using the same backbone.

[0056] Backbone networks, such as Darknet, are used for feature extraction, effectively extracting image features. Furthermore, each YOLO model's prediction head is trained using photovoltaic power station image data corresponding to its specific scene type, achieving more effective target detection in those scenes. Each YOLO model also undergoes lightweighting processes, such as model quantization and pruning, to reduce the number of parameters and computational cost, adapting to resource-constrained drone platforms.

[0057] In this embodiment, the YOLO visual neural network model library includes the YOLO-FAST model, which is matched with high-altitude scenes and can use low-resolution samples as training data; the YOLO-Reflect model, which is matched with highly reflective scenes and can use 80% highly reflective samples + 20% normal lighting samples as training data, and adds a channel attention module to strengthen the reflective edges for fine-tuning of the head structure; the YOLO-Occlusion model, which is matched with severely occluded scenes and can use 60% strongly occluded samples + 40% normal non-severely occluded samples as training data, and adds dilated convolution to expand the receptive field to penetrate occlusion for fine-tuning of the head structure; and the YOLO-Precision model, which is matched with other scenes.

[0058] Furthermore, after determining the scene type based on the real-time collected illumination, occlusion, and height / angle data, the system retrieves the YOLO model that matches the current scene type from the YOLO visual neural network model library.

[0059] In addition, to improve detection accuracy, the image preprocessing model stored in the UAV's computing processing center is used to perform image preprocessing on the collected photovoltaic image data, including: adaptive histogram equalization to enhance local contrast, targeted color space conversion to process illumination and shadows, or real-time deblurring, etc.

[0060] S3. Hot-load and apply the YOLO model that matches the current scene type to perform target detection and obtain the initial location of the currently detected photovoltaic module.

[0061] Specifically, a YOLO model matching the current scene type is pre-loaded, which is similar to loading program code from the hard drive into memory when a computer starts a program so that it can run quickly; then, real-time collected photovoltaic power station image data (or photovoltaic power station image data after image preprocessing) is input into the model for target detection to obtain the initial location of the currently detected photovoltaic module.

[0062] The photovoltaic module detection is performed using a matched YOLO model, and the output includes the bounding box coordinates, the center pixel coordinates of the photovoltaic module, the corner coordinates of the photovoltaic panel, and the confidence score of the photovoltaic module ID.

[0063] S4. Post-process the obtained initial position of the currently detected photovoltaic module to obtain the final position of the currently detected photovoltaic module.

[0064] To address issues such as positioning errors caused by similar photovoltaic panel components, topology graph matching is combined with the candidate component set obtained by YOLO detection to obtain the optimal ID allocation and optimize the positioning coordinates.

[0065] Specifically, post-processing of the YOLO output detection results includes: pre-determining the digital mapping of the photovoltaic array in the photovoltaic power station, completing topological relationship modeling and storing it in each YOLO model, and correcting the bounding boxes based on prior knowledge of the topological relationships of photovoltaic modules in the photovoltaic power station (e.g., constant horizontal spacing between modules in the same row, vertical alignment of modules in the same column, and consistent orientation of modules, etc.) to obtain the final location of the currently detected photovoltaic module. For example, if the bounding box of the current photovoltaic module overlaps with the bounding boxes of its adjacent modules or the spacing is unreasonable, adjustments are made using topological relationships.

[0066] It also includes: transforming topological constraints into a graph optimization problem, and solving for the optimal location by minimizing the following error functions, including: setting an objective function to minimize the geometric deviation between the observation and the topology, and solving the objective function to obtain the optimal photovoltaic ID, etc.

[0067] S5. Determine whether the drone is veerging from its course based on the final positioning of the photovoltaic modules being detected, and then perform automatic navigation for the drone based on the result of whether it is veerging from its course.

[0068] Specifically, based on the final location of the currently detected photovoltaic (PV) module, to verify whether the currently detected PV module has deviated from its course, it is necessary to combine the final locations of the PV modules detected at previous times. A trajectory is generated according to the order of the final locations of the currently detected PV module and the final locations of the PV modules detected at previous times. For example, the center pixel coordinates of the final locations of the PV modules detected at each previous time step are used as waypoints, and the center pixel coordinates of the final location of the currently detected PV module are used as sequential waypoints, thus generating the trajectory in the acquisition order. Furthermore, if there are many PV modules detected at each time step, i.e., several currently detected PV modules, considering the inspection of a large number of PV modules, to ensure comprehensive detection, waypoints are generally set at the center of the PV modules to be detected. When the drone passes through the waypoint, it will cover and collect images of several PV modules. Then, the PV module at the center of the collected image is selected as the target PV module, and the center of the target PV module is used as the waypoint.

[0069] The system determines whether the trajectory generated based on the final location of the currently detected photovoltaic module matches the final location of the photovoltaic modules detected in the previous time step. The expected trajectory is a flight path generated according to a preset route planning algorithm, including the start point, end point, and waypoints. If it matches the expected trajectory, navigation continues according to the expected trajectory. Otherwise, the system counts the final locations of all detected photovoltaic modules, identifies the undetected photovoltaic modules, and uses the final location of the currently detected photovoltaic module as the starting point to re-plan and obtain the UAV flight trajectory according to the optimal path algorithm for navigation.

[0070] Furthermore, considering the impact of meteorological data, adjustments are needed during the replanning of drone flight paths to collect clearer photovoltaic module image data. The method also includes:

[0071] In the process of identifying undetected photovoltaic modules and using the final location of the currently detected photovoltaic modules as a starting point, the drone's flight trajectory is replanned and obtained according to the optimal path algorithm. This involves dynamically matching flight trajectory adjustment schemes, including flight altitude and speed, with real-time collected data on illumination, occlusion, and altitude / angle. Based on the generated flight trajectory, the flight altitude and speed are adjusted according to the matched flight trajectory adjustment scheme. Different combinations of illumination, occlusion, and altitude / angle data (i.e., various combinations of sensing data such as illumination, occlusion, and altitude / angle) are pre-matched with flight trajectory adjustment schemes, including flight altitude and speed. The specific matched flight trajectory adjustment scheme is determined by analyzing and statistically analyzing different historical combinations of illumination, occlusion, and altitude / angle data, identifying drone flight altitudes and speeds where the drone's target detection and positioning accuracy exceeds the preset accuracy.

[0072] In a specific embodiment, considering that the currently identified scene type is not a single scene type, a dynamic scheduling strategy is designed to improve the accuracy of UAV positioning of photovoltaic modules and target detection performance in complex environments, enhance the adaptability and reliability of UAV autonomous navigation, and further improve the efficiency and stability of UAV autonomous navigation in various complex scenarios; the method also includes:

[0073] When scene type determination is completed based on real-time collected illumination, occlusion, and height / angle data, the determined scene type is not a single scene type; all single scene types are obtained. For example, the current detection is in a strong reflective scene, a severely occluded scene, and a high-altitude scene.

[0074] Design a tiered triggering mechanism and determine the priority triggering order for each scenario type. Set priority triggering conditions for the sensing data range corresponding to each single scenario type. Specifically, based on the characteristics of different photovoltaic power stations, statistically analyze the frequency of the scenario type in historical inspections and set the tiered triggering order for different scenario types. The scenario with higher frequency can be prioritized for priority triggering. Then, determine whether to trigger the priority condition based on the range of sensing data corresponding to the scenario type.

[0075] In this embodiment, the trigger condition judgment level for a single scene type is prioritized, with the levels from high to low as follows: severely occluded scene type, strong reflective scene type, high-altitude scene type, and other scene types. Priority trigger conditions are set for the perceived data range (i.e., the range of quantified index values) corresponding to each scene type. Other scene types are not set with separate trigger conditions; they are only considered to have the highest priority when the severely occluded scene type, strong reflective scene type, and high-altitude scene type all fail to meet their priority trigger conditions. For example, it is prioritized to determine whether the ratio of occlusion area to image area in a severely occluded scene is greater than the first occlusion area / image area threshold (e.g., 0.4). If it is greater, the priority trigger condition for the current scene type is met, and the severely occluded scene type has the highest priority; otherwise, the strong reflective scene type is further judged. If the grayscale entropy value of the image in the light scene is greater than the first image grayscale entropy value threshold (e.g., 0.8), then the current scene type priority trigger condition is met, and the strong reflective scene has the highest priority. Otherwise, it continues to judge whether the height value in the high-altitude scene is greater than the first height threshold (e.g., 35). If it is greater, then the current scene type priority trigger condition is met, and the high-altitude scene type has the highest priority. Otherwise, other scene types have the highest priority. After the current highest priority is determined, the scene type corresponding to the current highest priority is excluded, and priority judgment is continued according to level. The highest level of subsequent judgments is the second highest priority of the previous judgment. When no scene type triggers priority judgment in subsequent judgments, the scene type is randomly sorted to finally obtain the scene type priority.

[0076] Based on the collected perception data and the perception data range corresponding to each single scene type, the priority of the triggered scene type is determined and obtained.

[0077] According to the priority order of the triggered scene types, the YOLO models corresponding to the single scene type are switched and hot-loaded in sequence; the initial location of the currently detected photovoltaic module is obtained by the target detection output of the YOLO model matching the single scene type; the location is fused according to the preset fusion location engine rules to obtain the initial location of the fused photovoltaic module; the preset fusion location engine rules include weighted fusion rules or clustering fusion rules, and the higher the priority order, the higher the weighting weight.

[0078] In one specific embodiment, to avoid unnecessary consumption of computing resources and to ensure timeliness to a certain extent, the method further includes:

[0079] Monitor the real-time collected data on illumination, occlusion, and height / angle. If the fluctuation values ​​of each sensing data point collected over a consecutive preset time period are all less than the corresponding preset fluctuation threshold, it is determined that they belong to the same scene type, and there is no need to switch the YOLO model for hot reloading. In order to ensure timeliness, the preset time period should not be too long, and 2t-3t should be selected, where t is the sampling time.

[0080] A dynamic batch processing method is adopted, in which multiple frames of photovoltaic power station image data corresponding to the same scene type are input into a YOLO model matching the current scene type for target detection, instead of inputting real-time acquired photovoltaic power station image data into a YOLO model matching the current scene type for target detection. The dynamic batch processing method includes: setting a base batch size; monitoring processing performance during target detection of multiple frames of photovoltaic power station image data, including processing time, memory occupancy, and CPU / GPU utilization; dynamically increasing or decreasing the batch size based on the preset processing performance range corresponding to the monitored processing performance; in this embodiment, when the monitored processing performance falls within the first preset processing performance range, the batch size is dynamically reduced based on the base batch size.

[0081] For example: the processing time delay is greater than 100ms, the memory usage is greater than 80%, and the CPU / GPU utilization is greater than 80%; or when the monitored processing performance is within the second preset processing performance range, the batch size is dynamically increased based on the basic batch size, such as: the processing time delay is less than 1ms, the memory usage is less than 50%, and the CPU / GPU utilization is less than 60%.

[0082] The system acquires several frames of initial photovoltaic module positioning from the batch output and performs post-processing on the detection results output by YOLO. It then acquires several frames of final photovoltaic module positioning from the batch output and determines whether the flight trajectory generated by combining the final positioning of several frames of photovoltaic modules from different batches according to the batch processing order and the final positioning of several frames of photovoltaic modules from the same batch according to the corresponding frame acquisition order conforms to the expected trajectory. This is used as a substitute for determining whether the trajectory generated by combining the final positioning of the currently detected photovoltaic module with the final positioning order of the photovoltaic modules detected in the previous time step conforms to the expected trajectory.

[0083] In one specific embodiment, a suitable model is selected for target detection based on the actual complexity of the image background, improving the accuracy and efficiency of detection. This helps enhance the target detection performance of UAVs during photovoltaic power station inspections, thereby improving the real-time performance and safety of autonomous navigation. The method also includes:

[0084] Before inputting the real-time acquired image data of the photovoltaic power station into the YOLO model that matches the current scene type for target detection, the real-time acquired image data of the photovoltaic power station is pre-input into the CNN convolutional model stored in the computing processing center of the UAV platform to obtain the background complexity of the currently acquired image data of the photovoltaic power station.

[0085] After retrieving a YOLO model from the YOLO visual neural network model library that matches the current scene type, each YOLO model is designed to include both a YOLO model with a PAN structure and a YOLO model without a PAN structure. The matching process continues based on the background complexity of the acquired photovoltaic power station image data. This includes matching a YOLO model without a PAN structure when the background complexity of the acquired photovoltaic power station image data is greater than a preset background complexity. When the background complexity of the image data is high, it helps to accurately locate the current photovoltaic module by combining background features. When the background complexity of the image data is low, a YOLO model with a PAN structure needs to be designed. That is, based on the original YOLO FPN (Feature Pyramid Network), a PAN (Path Aggregation Network) structure is added to strengthen the fusion of low-level detailed features (such as the texture of the photovoltaic module) and high-level semantic features (such as the overall shape of the photovoltaic module), thereby improving the distinguishability of similar targets.

[0086] Furthermore, considering that if the image contains a background with multiple elements, rich textures or color variations, such as trees, buildings, and other equipment, this may increase the processing burden of the YOLO model and lead to a decrease in detection speed, the method also includes accelerating the inference speed of the YOLO model by using an acceleration engine: for the result that the background complexity of the currently acquired photovoltaic power station image data exceeds the preset background complexity, an acceleration engine is introduced to assist in accelerating detection based on matching a YOLO model without a PAN structure.

[0087] In a specific embodiment, to enhance the target detection capability of UAVs under adverse weather conditions, improve the accuracy and stability of target detection, and enable UAVs to effectively perform autonomous navigation in complex weather environments, thereby expanding the application scenarios and working conditions of UAVs, the method further includes:

[0088] Real-time collection of meteorological data and analysis of whether the collected meteorological data contains preset extreme meteorological data, including: preset extreme meteorological data exceeding preset rainfall and snowfall thresholds; such as: rainfall and snowfall exceeding preset rainfall and snowfall thresholds;

[0089] If present, while acquiring RGB images of the photovoltaic power station in real time, thermal imaging data is simultaneously acquired using a thermal imaging camera installed on the drone platform, and depth image data is acquired using a binocular camera installed on the drone platform.

[0090] To perform target detection, the YOLO model that matches the current scene type is simultaneously input into the RGB image, depth image, and thermal imaging data acquired at the same time. Instead of inputting real-time acquired photovoltaic power station image data into the YOLO model that matches the current scene type, target detection is performed by fusing feature data.

[0091] like Figure 2 As shown in the figure, this application discloses an autonomous navigation system for unmanned aerial vehicles (UAVs) based on a YOLO visual neural network, specifically including:

[0092] Data acquisition module 101 is used to acquire image data of photovoltaic power station, as well as data on illumination, shading, and height / angle in real time;

[0093] YOLO model matching module 102 is used to determine the scene type based on the real-time collected illumination, occlusion and height / angle data. For the determined scene type, it retrieves the YOLO model that matches the current scene type from the YOLO visual neural network model library. Each YOLO model uses the same backbone and is trained using the differential prediction head of the photovoltaic power station image data corresponding to the scene type it matches, and undergoes lightweight processing.

[0094] The target detection and localization module 103 is used to hot-load the YOLO model that matches the current scene type. It inputs the real-time acquired image data of the photovoltaic power station into the YOLO model that matches the current scene type to perform target detection and obtain the initial localization of the detected photovoltaic module. It performs post-processing on the detection results output by YOLO, including localization correction based on prior knowledge of the topological relationship of photovoltaic modules in the photovoltaic power station, and obtains the final localization of the detected photovoltaic module.

[0095] The positioning verification and navigation module 104 is used to determine whether the trajectory generated based on the final positioning of the currently detected photovoltaic module and the final positioning of the photovoltaic modules detected in the previous time step matches the expected trajectory. If it matches the expected trajectory, navigation continues according to the expected trajectory. Otherwise, based on the final positioning of all detected photovoltaic modules, the undetected photovoltaic modules are identified, and the final positioning of the currently detected photovoltaic module is used as the starting point to re-plan the drone's flight trajectory using the optimal path algorithm and then navigation is performed.

[0096] In one specific embodiment, the target detection and localization module 103 in the system is further configured to: when scene type determination is completed based on real-time collected illumination, occlusion, and height / angle data, and the determined scene type is not a single scene type, acquire all single scene types; design a hierarchical triggering mechanism and determine the priority triggering order of each scene type, and set priority triggering conditions for the illumination, occlusion, and height / angle data ranges corresponding to multiple single scene types; determine and acquire the priority of the triggered scene type based on the collected illumination, occlusion, and height / angle data; sequentially switch and hot-load the YOLO model matching the corresponding single scene type according to the priority order of the triggered scene types; acquire the initial localization of the currently detected photovoltaic module output by the target detection using the YOLO model matching the single scene type, and perform localization fusion according to the preset fusion localization engine rules to obtain the fused initial localization of the photovoltaic module; the preset fusion localization engine rules include weighted fusion rules, where higher priority order corresponds to higher weighting.

[0097] In one specific embodiment, the target detection and localization module 103 in the system is also used to monitor the real-time collected illumination, occlusion, and height / angle data. If the fluctuation values ​​of each sensing data of illumination, occlusion, and height / angle collected over a consecutive preset time period are all less than the corresponding preset fluctuation threshold, and it is determined that they belong to the same scene type, then there is no need to switch the YOLO model for hot loading. Instead, a dynamic batch processing method is selected, in which multiple frames of photovoltaic power station image data corresponding to the same scene type are input into the YOLO model that matches the current scene type for target detection, instead of inputting the real-time collected photovoltaic power station image data into the YOLO model that matches the current scene type for target detection. The dynamic batch processing method includes: setting a basic batch processing size; monitoring the processing performance during the target detection process for multiple frames of photovoltaic power station image data, including: processing time, memory occupancy, and CPU / GPU utilization; and dynamically increasing or decreasing the batch processing size based on the preset processing performance range corresponding to the monitored processing performance.

[0098] The positioning verification and navigation module 104 is further configured to acquire the initial positioning of several frames of photovoltaic modules output by batch processing and post-process the detection results output by YOLO, acquire the final positioning of several frames of photovoltaic modules output by batch processing, and determine whether the trajectory generated according to the final positioning of several frames of photovoltaic modules output by different batch processing according to the batch processing order and the trajectory generated according to the corresponding frame acquisition order of several frames of photovoltaic modules output by the same batch processing conforms to the expected trajectory instead of determining whether the trajectory generated according to the final positioning of the currently detected photovoltaic module and the final positioning order of the photovoltaic modules detected in the previous time step conforms to the expected trajectory.

[0099] In one specific embodiment, the YOLO model matching module 102 in the system is further configured to: input the real-time acquired photovoltaic power station image data into a CNN convolutional model before inputting the real-time acquired photovoltaic power station image data into a YOLO model matching the current scene type for target detection; obtain the background complexity of the currently acquired photovoltaic power station image data; after retrieving a YOLO model matching the current scene type from the YOLO visual neural network model library for the determined scene type, design each YOLO model to include a YOLO model with a PAN structure and a YOLO model without a PAN structure; continue to match whether a YOLO model with a PAN structure is used based on the background complexity of the acquired real-time acquired photovoltaic power station image data, including matching a YOLO model without a PAN structure if the background complexity of the acquired real-time acquired photovoltaic power station image data is greater than a preset background complexity; and further configured to: if the background complexity of the acquired real-time acquired photovoltaic power station image data exceeds a preset background complexity, select to introduce an acceleration engine to assist in accelerating detection based on matching a YOLO model without a PAN structure.

[0100] In one specific embodiment, the data acquisition module 101 in the system is also used to acquire meteorological data in real time and analyze whether there is preset extreme meteorological data in the acquired meteorological data; if so, while acquiring the RGB image of the photovoltaic power station in real time, depth image data and thermal imaging data are acquired simultaneously.

[0101] The YOLO model matching module 102 is also used to synchronously input the YOLO model that matches the current scene type into the simultaneously acquired RGB image, depth image and thermal imaging data for target detection, instead of inputting the real-time acquired photovoltaic power station image data into the YOLO model that matches the current scene type for target detection.

[0102] In one specific embodiment, the positioning verification and navigation module 104 in the system is further configured to, during the process of identifying undetected photovoltaic modules and using the final positioning of the currently detected photovoltaic modules as the starting point, dynamically match a flight trajectory adjustment scheme including flight altitude and flight speed by combining real-time collected illumination, occlusion, and altitude / angle data to re-plan and obtain the UAV flight trajectory according to the optimal path algorithm; different combinations of illumination, occlusion, and altitude / angle data are pre-matched with flight trajectory adjustment schemes including flight altitude and flight speed, and the specific matched flight trajectory adjustment scheme is determined based on the analysis and statistics of different historical combinations of illumination, occlusion, and altitude / angle data, where the accuracy of UAV target detection and positioning is greater than the preset accuracy of UAV flight altitude and flight speed; based on the generated flight trajectory, the flight altitude and flight speed are adjusted according to the matched flight trajectory adjustment scheme.

[0103] This application also discloses a computer-readable storage medium.

[0104] Specifically, the computer-readable storage medium stores a computer program that can be loaded by a processor and executed, such as the above-described UAV autonomous navigation method based on the YOLO visual neural network. The computer-readable storage medium includes, for example, various media that can store program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0105] This application also discloses a computer device.

[0106] Specifically, the computer device includes a memory and a processor. The memory stores a computer program that can be loaded and executed by the processor to implement the aforementioned autonomous navigation method for unmanned aerial vehicles based on the YOLO visual neural network.

[0107] The above are all preferred embodiments of this application and are not intended to limit the scope of protection of this application. Any feature disclosed in this specification (including the abstract and drawings) may be replaced by other equivalent or similar features unless specifically stated otherwise. That is, unless specifically stated otherwise, each feature is only one example of a series of equivalent or similar features.

Claims

1. A method for autonomous navigation of a UAV based on YOLO visual neural network, characterized in that, include: Real-time acquisition of image data of photovoltaic power plants, as well as data on illumination, shading, and height, or data on illumination, shading, and angle; The scene type is determined based on real-time collected illumination, occlusion, and height data or illumination, occlusion, and angle data. For the determined scene type, the YOLO model matching the current scene type is retrieved from the YOLO visual neural network model library. Each YOLO model uses the same backbone and is trained using the differentiated prediction head of the photovoltaic power station image data corresponding to the scene type it matches, and undergoes lightweight processing. The YOLO model obtained by hot loading matches the current scene type. The real-time image data of the photovoltaic power station is input into the YOLO model that matches the current scene type for target detection to obtain the initial location of the detected photovoltaic module. The detection results output by YOLO are post-processed, including the location correction based on the prior knowledge of the topological relationship of photovoltaic modules in the photovoltaic power station, to obtain the final location of the detected photovoltaic module. The system determines whether the trajectory generated based on the final location of the currently detected photovoltaic (PV) module matches the expected trajectory from the final locations of PV modules detected in previous time steps. If it matches, navigation continues according to the expected trajectory. Otherwise, it calculates the final locations of all detected PV modules, identifies undetected PV modules, and uses the final location of the currently detected PV module as the starting point to re-plan and obtain the UAV's flight trajectory based on the optimal path algorithm for navigation. This also includes: When the scene type is determined based on the real-time collected illumination, occlusion, and height data or illumination, occlusion, and angle data, and the determined scene type is not a single scene type, all single scene types are obtained. Design a hierarchical triggering mechanism and determine the priority triggering order for each scene type. Set priority triggering conditions for multiple single scene types corresponding to illumination, occlusion, and height data or illumination, occlusion, and angle data ranges. Determine and obtain the priority of the triggered scene type based on the collected illumination, occlusion, and height data or illumination, occlusion, and angle data. According to the priority order of the triggered scene type, the YOLO model corresponding to the single scene type is switched and hot-loaded in sequence; the initial location of the currently detected photovoltaic module is obtained by the target detection output of the YOLO model using the single scene type matching; the location is fused according to the preset fusion location engine rules to obtain the initial location of the fused photovoltaic module. The preset fusion positioning engine rules include weighted fusion rules, with higher priority order corresponding to higher weighting. 2.The method of claim 1, wherein, Also includes: If the fluctuation values ​​of the light, occlusion and height data or light, occlusion and angle data collected in real time are all less than the corresponding preset fluctuation thresholds for a continuous preset time period, it is determined that they belong to the same scene type. In this case, there is no need to switch the YOLO model for hot loading. Instead, a dynamic batch processing method is selected. The image data of multiple frames of photovoltaic power stations corresponding to the same scene type are input into the YOLO model that matches the current scene type for target detection, instead of inputting the real-time collected image data of photovoltaic power stations into the YOLO model that matches the current scene type for target detection. The dynamic batch processing method includes: setting a basic batch size; monitoring processing performance during target detection of multi-frame photovoltaic power station image data, including processing time, memory occupancy, and CPU / GPU utilization; and dynamically increasing or decreasing the batch size based on the preset processing performance range corresponding to the monitored processing performance. The system acquires several frames of initial photovoltaic module positioning from the batch output and performs post-processing on the detection results output by YOLO. It then acquires several frames of final photovoltaic module positioning from the batch output and determines whether the trajectory generated based on the final positioning of several frames of photovoltaic modules from different batches in the batch order and the final positioning of several frames of photovoltaic modules from the same batch in the corresponding frame acquisition order matches the expected trajectory. This is used as an alternative to determining whether the trajectory generated based on the final positioning of the currently detected photovoltaic module matches the final positioning order of the photovoltaic modules detected in the previous time step.

3. The UAV autonomous navigation method based on YOLO visual neural network according to claim 1, characterized in that, Also includes: Before inputting the real-time acquired image data of the photovoltaic power station into the YOLO model that matches the current scene type for target detection, the real-time acquired image data of the photovoltaic power station is pre-input into the CNN convolutional model to obtain the background complexity of the currently acquired image data of the photovoltaic power station; After retrieving a YOLO model from the YOLO visual neural network model library that matches the current scene type based on the determined scene type, each YOLO model is designed to include both a YOLO model with a PAN structure and a YOLO model without a PAN structure. The matching process continues based on the background complexity of the currently acquired photovoltaic power station image data, including matching a YOLO model without a PAN structure if the background complexity of the currently acquired photovoltaic power station image data is greater than a preset background complexity.

4. The UAV autonomous navigation method based on YOLO visual neural network according to claim 3, characterized in that, Also includes: For cases where the background complexity of the currently acquired photovoltaic power station image data exceeds the preset background complexity, an acceleration engine is introduced to assist in accelerating detection, based on the matching of the YOLO model without a PAN structure.

5. The UAV autonomous navigation method based on YOLO visual neural network according to claim 1, characterized in that, Also includes: Real-time collection of meteorological data and analysis of whether the collected meteorological data contains preset extreme weather data; If applicable, while acquiring RGB images of the photovoltaic power station in real time, depth image data and thermal imaging data are simultaneously acquired; the simultaneously acquired RGB images, depth images, and thermal imaging data are then input into a YOLO model that matches the current scene type for target detection, instead of inputting the real-time acquired photovoltaic power station image data into a YOLO model that matches the current scene type for target detection.

6. The UAV autonomous navigation method based on YOLO visual neural network according to claim 1, characterized in that, Also includes: In the process of identifying undetected photovoltaic modules and using the final location of the currently detected photovoltaic modules as a starting point, the drone's flight trajectory is replanned and obtained according to the optimal path algorithm. This involves dynamically matching flight trajectory adjustment schemes, including flight altitude and speed, based on real-time collected illumination, occlusion, and altitude data or illumination, occlusion, and angle data. Different combinations of illumination, occlusion, and altitude data or illumination, occlusion, and angle data are pre-matched with flight trajectory adjustment schemes, including flight altitude and speed. The specific matched flight trajectory adjustment scheme is determined based on historical combinations of illumination, occlusion, and altitude data or illumination, occlusion, and angle data, analyzing and statistically determining the drone's flight altitude and speed where the drone's target detection and positioning accuracy is greater than the preset accuracy. Based on the generated flight trajectory, the flight altitude and speed are adjusted according to the matched flight trajectory adjustment scheme.

7. An autonomous navigation system for unmanned aerial vehicles (UAVs) based on a YOLO visual neural network, characterized in that, include: The data acquisition module is used to collect real-time image data of photovoltaic power plants, as well as data on illumination, shading, and height, or data on illumination, shading, and angle. The YOLO model matching module is used to determine the scene type based on real-time collected illumination, occlusion, and height data or illumination, occlusion, and angle data. For the determined scene type, it retrieves the YOLO model that matches the current scene type from the YOLO visual neural network model library. Each YOLO model uses the same backbone and is trained using the differential prediction head of the photovoltaic power station image data corresponding to the matched scene type, and undergoes lightweight processing. The target detection and localization module is used to hot-load the YOLO model that matches the current scene type. It inputs real-time acquired image data of the photovoltaic power station into the YOLO model that matches the current scene type to perform target detection and obtain the initial localization of the detected photovoltaic module. It then performs post-processing on the detection results output by YOLO, including localization correction based on prior knowledge of the topological relationship of photovoltaic modules in the photovoltaic power station, to obtain the final localization of the detected photovoltaic module. It is also used to obtain all single scene types when the scene type is not determined by real-time collected illumination, occlusion and height data or illumination, occlusion and angle data; Design a hierarchical triggering mechanism and determine the priority triggering order for each scene type. Set priority triggering conditions for multiple single scene types corresponding to illumination, occlusion, and height data or illumination, occlusion, and angle data ranges. Determine and obtain the priority of the triggered scene type based on the collected illumination, occlusion, and height data or illumination, occlusion, and angle data. According to the priority order of the triggered scene type, the YOLO model corresponding to the single scene type is switched and hot-loaded in sequence; the initial location of the currently detected photovoltaic module is obtained by the target detection output of the YOLO model using the single scene type matching; the location is fused according to the preset fusion location engine rules to obtain the initial location of the fused photovoltaic module. The preset fusion positioning engine rules include weighted fusion rules, where higher priority order corresponds to higher weighting. The positioning verification and navigation module is used to determine whether the trajectory generated based on the final positioning of the currently detected photovoltaic module and the final positioning of the photovoltaic modules detected in the previous time step matches the expected trajectory. If it matches the expected trajectory, navigation continues according to the expected trajectory. Otherwise, based on the final positioning of all detected photovoltaic modules, the module identifies the undetected photovoltaic modules, uses the final positioning of the currently detected photovoltaic module as the starting point, and re-plans the drone's flight trajectory using the optimal path algorithm for navigation.

8. A computer-readable storage medium, characterized in that, The computer-readable storage medium includes a stored computer program, wherein, when the computer program is executed, it controls the device on which the computer-readable storage medium is located to perform the method as described in any one of claims 1 to 6.

9. A computer device, characterized in that, The computer device includes a memory, a processor, and a program stored in and executable on the memory, the program being executed by the processor to implement the steps of the method as described in any one of claims 1 to 6.

Images