A maneuverability unmanned aerial vehicle path planning method based on feature point calculation

Abstract

The present application relates to unmanned aerial vehicle path planning, in particular to a kind of maneuverability unmanned aerial vehicle path planning method based on feature point calculation, obtain the remote sensing image of the area to be inspected, the color feature value of the color of photovoltaic module composition in image is calculated, and binary image is obtained based on color feature value conversion;Based on color feature value, the connected region in binary image is extracted, and the first target connected region of corresponding photovoltaic module is obtained according to geometric feature;First target connected region is morphologically processed, and the second target connected region corresponding to photovoltaic module string is obtained;The edge contour of second target connected region is sampled and calculated, the straight line information of photovoltaic module string edge is obtained, and the preset inspection path of unmanned aerial vehicle is determined according to straight line information;The technical scheme provided by the present application can effectively overcome the defects that the inspection efficiency of prior art is lower, and the photovoltaic module with defect cannot be effectively focused on inspection.

Description

Technical Field

[0001] This invention relates to unmanned aerial vehicle (UAV) path planning, and more specifically to a method for path planning of maneuverable UAVs based on feature point calculation. Background Technology

[0002] With the continuous development of the photovoltaic industry, the importance of photovoltaic operation and maintenance (O&M) work is increasing. Inspecting photovoltaic modules in photovoltaic power plants is a demanding task. To meet the requirements of remote centralized control and regional O&M, drones are increasingly being used to perform photovoltaic module inspections. Drones have advantages such as simple operation, rapid response, rich payload capacity, wide range of applications, and high mobility. They have widely replaced O&M personnel in inspections, reducing the labor intensity of O&M personnel, ensuring their personal safety, and significantly reducing production costs.

[0003] When drones perform photovoltaic (PV) module inspection tasks, they often follow a pre-set fixed inspection path, collecting point cloud images to inspect the PV power station. However, this fixed-path inspection method suffers from problems such as missing some PV modules, redundant inspections, and low inspection efficiency. Furthermore, this method focuses on all PV modules with essentially the same level of attention, failing to effectively target defective modules and accurately acquire their image information. Summary of the Invention

[0004] (a) Technical problems to be solved

[0005] In view of the above-mentioned shortcomings of the existing technology, the present invention provides a path planning method for a mobile UAV based on feature point calculation, which can effectively overcome the shortcomings of the existing technology, such as low inspection efficiency and inability to effectively focus on the inspection of defective photovoltaic modules.

[0006] (II) Technical Solution

[0007] To achieve the above objectives, the present invention provides the following technical solution:

[0008] A path planning method for a maneuverable unmanned aerial vehicle (UAV) based on feature point calculation includes the following steps:

[0009] S1. Obtain remote sensing images of the area to be inspected, calculate the color feature values ​​of the colors composed of photovoltaic modules in the image, and convert them into binary images based on the color feature values;

[0010] S2. Extract connected regions from the binary image based on color feature values, and obtain the first target connected region of the corresponding photovoltaic module based on geometric features;

[0011] S3. Perform morphological processing on the first target connected region to obtain the second target connected region corresponding to the photovoltaic module string;

[0012] S4. Sample and calculate the feature points of the edge contour of the second target connected region to obtain the straight line information of the edge of the photovoltaic module string, and determine the preset inspection path of the UAV based on the straight line information.

[0013] S5. Use the photovoltaic module defect identification model to identify surface defects of photovoltaic modules in remote sensing images to obtain the first photovoltaic module surface defect identification result;

[0014] S6. By performing image analysis and processing on the remote sensing images, the surface defect identification results of the second photovoltaic module are obtained;

[0015] S7. Combine the surface defect identification results of the first photovoltaic module and the surface defect identification results of the second photovoltaic module to determine the defective photovoltaic module, and optimize the preset inspection path of the UAV based on the geographical location information of the defective photovoltaic module to obtain the optimized inspection path.

[0016] Preferably, in S1, a remote sensing image of the area to be inspected is acquired, and the color feature values ​​of the colors composed of photovoltaic modules in the image are calculated, including:

[0017] Acquire remote sensing images of the area to be inspected, and calculate the color feature values ​​of the photovoltaic modules in the image based on the different distribution characteristics of the colors of the photovoltaic modules in the RGB histogram.

[0018] The colors of photovoltaic modules include the colors of the photovoltaic panels and the photovoltaic support structures.

[0019] Preferably, in step S3, morphological processing is performed on the first target connected region to obtain the second target connected region corresponding to the photovoltaic module string, including:

[0020] The first target connected region is subjected to expansion and corrosion treatment to eliminate the gaps between different first target connected regions, thereby obtaining the second target connected region corresponding to the photovoltaic module string.

[0021] Preferably, in step S4, feature points are sampled and calculated on the edge contour of the second target connected region to obtain straight line information of the photovoltaic module string edge, and the preset inspection path of the UAV is determined based on the straight line information, including:

[0022] Multiple feature points on the edge contour of the second target connected region are obtained and calculated to obtain the straight line information of the edge of the photovoltaic module string, and to determine the multiple line segments corresponding to the edge of the photovoltaic module string.

[0023] Based on a preset coordinate system, determine whether multiple line segments are collinear, and determine the preset inspection path of the UAV based on these line segments.

[0024] Preferably, step S4 involves sampling and calculating feature points on the edge contour of the second target connected region to obtain straight line information of the photovoltaic module string edge, and determining the preset inspection path of the UAV based on the straight line information. The step S4 also includes:

[0025] During actual inspection, when the ratio of the sum of pixel values ​​of all pixels in the binary image of the photovoltaic module string to the total sum of pixels in the image frame acquired by the drone is less than a preset threshold, it is determined that the drone is located at the boundary of the area to be inspected, and the drone is controlled to perform a turning action according to the preset inspection path.

[0026] Preferably, the photovoltaic module defect identification model includes a feature extraction layer, a model encoder, a model decoder, and a linear layer connected in sequence;

[0027] The model encoder processes the feature maps extracted by the feature extraction layer based on a multi-scale deformable attention mechanism to obtain multi-scale feature maps;

[0028] The model decoder processes multi-scale feature maps based on self-attention and cross-attention mechanisms, and inputs the resulting decoded features and reference points into a linear layer.

[0029] Preferably, the model encoder performs dimensionality compression on the multi-scale feature map, adds it to the positional encoding to obtain serialized data, and then adds the serialized data to the scale-level embedding;

[0030] The model encoder uses a position encoding method based on sine and cosine functions.

[0031] Preferably, the model decoder learns the two-dimensional coordinates of the reference point for each target feature pixel using a linear layer and an activation function, and then performs coordinate regression using a cross-attention mechanism.

[0032] Preferably, in step S6, image analysis processing of the remote sensing image is performed to obtain the surface defect identification result of the second photovoltaic module, including:

[0033] The remote sensing image was converted into an HSV image, and the outline of the photovoltaic module in the HSV image was initially extracted by adjusting the threshold.

[0034] The adjusted HSV image is converted into a binary image and then morphological processing is performed.

[0035] An edge detection algorithm is used to extract the outline of the photovoltaic module to obtain an image of the photovoltaic module;

[0036] Defects in photovoltaic module images were detected using the HSV threshold method, and the surface defect identification results of the second photovoltaic module were obtained.

[0037] Preferably, in step S7, the defective photovoltaic module is determined by combining the surface defect identification results of the first photovoltaic module and the surface defect identification results of the second photovoltaic module, and the preset inspection path of the UAV is optimized based on the geographical location information of the defective photovoltaic module to obtain an optimized inspection path, including:

[0038] By combining the surface defect identification results of the first photovoltaic module and the surface defect identification results of the second photovoltaic module, the defective photovoltaic module is identified, and the geographical location information of the defective photovoltaic module is obtained;

[0039] Based on the geographical location information of the defective photovoltaic module, the inspection route, number of detours, and detour time at the photovoltaic module string to which the defective photovoltaic module belongs in the preset inspection path are optimized to obtain the optimized inspection path.

[0040] (III) Beneficial Effects

[0041] Compared with existing technologies, the path planning method for maneuverable unmanned aerial vehicles based on feature point calculation provided by this invention has the following advantages:

[0042] 1) Acquire remote sensing images of the area to be inspected, calculate the color feature values ​​of the colors of the photovoltaic modules in the image, and convert them into binary images based on the color feature values. Extract connected regions from the binary images based on the color feature values, and obtain the first target connected region of the corresponding photovoltaic module according to the geometric features. Perform morphological processing on the first target connected region to obtain the second target connected region corresponding to the photovoltaic module string. Sample and calculate the feature points of the edge contour of the second target connected region to obtain the straight line information of the edge of the photovoltaic module string. Determine the preset inspection path of the UAV based on the straight line information. By determining the second target connected region corresponding to the photovoltaic module string in the remote sensing image and obtaining the straight line information of the edge of the photovoltaic module string, the automatic adaptive setting of the preset inspection path based on the photovoltaic module string can be realized, which can effectively improve the inspection efficiency and ensure the inspection effect.

[0043] 2) The photovoltaic module defect identification model is used to identify surface defects of photovoltaic modules in remote sensing images to obtain the first photovoltaic module surface defect identification result. By performing image analysis and processing on the remote sensing images, the second photovoltaic module surface defect identification result is obtained. The defective photovoltaic modules are identified by combining the first and second photovoltaic module surface defect identification results. Based on the geographical location information of the defective photovoltaic modules, the preset inspection path of the UAV is optimized to obtain the optimized inspection path. By initially identifying the defective photovoltaic modules and optimizing the preset inspection path based on the defective photovoltaic modules, the inspection of defective photovoltaic modules can be effectively focused on, so as to accurately obtain the image information of the defective photovoltaic modules and facilitate the maintenance personnel to complete the photovoltaic maintenance work efficiently. Attached Figure Description

[0044] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0045] Figure 1 This is a schematic diagram of the process of the present invention;

[0046] Figure 2 This is a flowchart illustrating the process of determining the preset inspection path of the UAV in this invention.

[0047] Figure 3 This is a schematic diagram illustrating the process of optimizing the preset inspection path of a drone based on the geographical location information of defective photovoltaic modules in this invention. Detailed Implementation

[0048] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative effort are within the scope of protection of the present invention.

[0049] A path planning method for maneuverable unmanned aerial vehicles based on feature point calculation, such as Figure 1 and Figure 2 As shown, ① obtain remote sensing images of the area to be inspected, calculate the color feature values ​​of the colors composed of photovoltaic modules in the image, and convert them into binary images based on the color feature values.

[0050] Specifically, remote sensing images of the area to be inspected are acquired, and color feature values ​​of the colors composed of photovoltaic modules in the images are calculated, including:

[0051] Acquire remote sensing images of the area to be inspected, and calculate the color feature values ​​of the photovoltaic modules in the image based on the different distribution characteristics of the colors of the photovoltaic modules in the RGB histogram.

[0052] The colors of photovoltaic modules include the colors of the photovoltaic panels and the photovoltaic support structures.

[0053] ② Extract connected regions from binary images based on color feature values, and obtain the first target connected region of the corresponding photovoltaic module based on geometric features.

[0054] ③ Perform morphological processing on the first target connected region to obtain the second target connected region corresponding to the photovoltaic module string, specifically including:

[0055] The first target connected region is subjected to expansion and corrosion treatment to eliminate the gaps between different first target connected regions, thereby obtaining the second target connected region corresponding to the photovoltaic module string.

[0056] ④ Sampling and calculating feature points on the edge contour of the second target connected region to obtain the straight line information of the photovoltaic module string edge, and determining the preset inspection path of the UAV based on the straight line information, specifically including:

[0057] Multiple feature points on the edge contour of the second target connected region are obtained and calculated to obtain the straight line information of the edge of the photovoltaic module string, and to determine the multiple line segments corresponding to the edge of the photovoltaic module string.

[0058] Based on a preset coordinate system, determine whether multiple line segments are collinear, and determine the preset inspection path of the UAV based on these line segments.

[0059] Specifically, feature points are sampled and calculated on the edge contour of the second target connected region to obtain straight line information of the photovoltaic module string edge, and the preset inspection path of the UAV is determined based on the straight line information. This also includes:

[0060] During actual inspection, when the ratio of the sum of pixel values ​​of all pixels in the binary image of the photovoltaic module string to the total sum of pixels in the image frame acquired by the drone is less than a preset threshold, it is determined that the drone is located at the boundary of the area to be inspected, and the drone is controlled to perform a turning action according to the preset inspection path.

[0061] The above technical solution acquires remote sensing images of the area to be inspected, calculates the color feature values ​​of the photovoltaic modules in the image, and converts them into a binary image. Connected regions are extracted from the binary image based on the color feature values, and a first target connected region corresponding to the photovoltaic module is obtained based on geometric features. Morphological processing is performed on the first target connected region to obtain a second target connected region corresponding to the photovoltaic module string. Feature points are sampled and calculated on the edge contour of the second target connected region to obtain the straight line information of the photovoltaic module string's edge. A preset inspection path for the UAV is determined based on this straight line information. By determining the second target connected region corresponding to the photovoltaic module string in the remote sensing image and obtaining the straight line information of the photovoltaic module string's edge, automated adaptive setting of the preset inspection path based on the photovoltaic module string is achieved, effectively improving inspection efficiency while ensuring inspection results.

[0062] like Figure 1 and Figure 3 As shown in Figure 5, the photovoltaic module defect identification model is used to identify surface defects of the photovoltaic module in the remote sensing image, and the first photovoltaic module surface defect identification result is obtained.

[0063] In the technical solution of this application, the photovoltaic module defect identification model includes a feature extraction layer, a model encoder, a model decoder and a linear layer connected in sequence.

[0064] The model encoder processes the feature maps extracted by the feature extraction layer based on a multi-scale deformable attention mechanism to obtain multi-scale feature maps;

[0065] The model decoder processes multi-scale feature maps based on self-attention and cross-attention mechanisms, and inputs the resulting decoded features and reference points into a linear layer.

[0066] 1) The model encoder compresses the multi-scale feature map in dimensions, adds it to the positional encoding to obtain serialized data, and then adds the serialized data to the scale-level embedding;

[0067] The model encoder uses a position encoding method based on sine and cosine functions.

[0068] 2) For each target feature pixel, the model decoder learns the two-dimensional coordinates of the reference point through a linear layer and activation function, and then uses a cross-attention mechanism to perform coordinate regression.

[0069] ⑥ By performing image analysis and processing on the remote sensing images, the surface defect identification results of the second photovoltaic module are obtained, specifically including:

[0070] The remote sensing image was converted into an HSV image, and the outline of the photovoltaic module in the HSV image was initially extracted by adjusting the threshold.

[0071] The adjusted HSV image is converted into a binary image and then morphological processing is performed.

[0072] An edge detection algorithm is used to extract the outline of the photovoltaic module to obtain an image of the photovoltaic module;

[0073] Defects in photovoltaic module images were detected using the HSV threshold method, and the surface defect identification results of the second photovoltaic module were obtained.

[0074] ⑦ Combining the surface defect identification results of the first photovoltaic module and the surface defect identification results of the second photovoltaic module, the defective photovoltaic module is identified. Based on the geographical location information of the defective photovoltaic module, the preset inspection path of the UAV is optimized to obtain the optimized inspection path, which specifically includes:

[0075] By combining the surface defect identification results of the first photovoltaic module and the surface defect identification results of the second photovoltaic module, the defective photovoltaic module is identified, and the geographical location information of the defective photovoltaic module is obtained;

[0076] Based on the geographical location information of the defective photovoltaic module, the inspection route, number of detours, and detour time at the photovoltaic module string to which the defective photovoltaic module belongs in the preset inspection path are optimized to obtain the optimized inspection path.

[0077] The above technical solution utilizes a photovoltaic module defect identification model to identify surface defects of photovoltaic modules in remote sensing images, obtaining a first photovoltaic module surface defect identification result. Through image analysis and processing of the remote sensing images, a second photovoltaic module surface defect identification result is obtained. Combining the first and second photovoltaic module surface defect identification results, defective photovoltaic modules are identified. Based on the geographical location information of the defective photovoltaic modules, the preset inspection path of the UAV is optimized to obtain an optimized inspection path. By initially identifying defective photovoltaic modules and optimizing the preset inspection path based on these defective modules, effective focused inspection of defective photovoltaic modules can be performed, accurately acquiring image information of the defective photovoltaic modules and facilitating efficient photovoltaic operation and maintenance work by maintenance personnel.

[0078] The above embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions will not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A path planning method for a maneuverable unmanned aerial vehicle (UAV) based on feature point calculation, characterized in that: Includes the following steps: S1. Obtain remote sensing images of the area to be inspected, calculate the color feature values ​​of the colors composed of photovoltaic modules in the image, and convert them into binary images based on the color feature values; S2. Extract connected regions from the binary image based on color feature values, and obtain the first target connected region of the corresponding photovoltaic module based on geometric features; S3. Perform morphological processing on the first target connected region to obtain the second target connected region corresponding to the photovoltaic module string; S4. Sample and calculate the feature points of the edge contour of the second target connected region to obtain the straight line information of the edge of the photovoltaic module string, and determine the preset inspection path of the UAV based on the straight line information. S5. Use the photovoltaic module defect identification model to identify surface defects of photovoltaic modules in remote sensing images to obtain the first photovoltaic module surface defect identification result; S6. By performing image analysis and processing on the remote sensing image, the surface defect identification results of the second photovoltaic module are obtained, including: The remote sensing image was converted into an HSV image, and the outline of the photovoltaic module in the HSV image was initially extracted by adjusting the threshold. The adjusted HSV image is converted into a binary image and then morphological processing is performed. An edge detection algorithm is used to extract the outline of the photovoltaic module to obtain an image of the photovoltaic module; Defects in photovoltaic module images were detected using the HSV threshold method, and the surface defect identification results of the second photovoltaic module were obtained. S7. Combine the surface defect identification results of the first photovoltaic module and the surface defect identification results of the second photovoltaic module to determine the defective photovoltaic module, and optimize the preset inspection path of the UAV based on the geographical location information of the defective photovoltaic module to obtain the optimized inspection path; The photovoltaic module defect identification model includes a feature extraction layer, a model encoder, a model decoder, and a linear layer connected in sequence. The model encoder processes the feature maps extracted by the feature extraction layer based on a multi-scale deformable attention mechanism to obtain multi-scale feature maps; The model decoder processes multi-scale feature maps based on self-attention and cross-attention mechanisms, and inputs the obtained decoded features and reference points into the linear layer; The model encoder performs dimensionality compression on the multi-scale feature map, adds it to the positional encoding to obtain serialized data, and then adds the serialized data to the scale-level embedding. The model encoder adopts a position encoding method based on sine and cosine functions; The model decoder learns the two-dimensional coordinates of the reference point for each target feature pixel using a linear layer and activation function, and then performs coordinate regression using a cross-attention mechanism.

2. The path planning method for maneuverable unmanned aerial vehicles based on feature point calculation according to claim 1, characterized in that: In S1, remote sensing images of the area to be inspected are acquired, and color feature values ​​of the colors composed of photovoltaic modules in the image are calculated, including: Acquire remote sensing images of the area to be inspected, and calculate the color feature values ​​of the photovoltaic modules in the image based on the different distribution characteristics of the colors of the photovoltaic modules in the RGB histogram. The colors of photovoltaic modules include the colors of the photovoltaic panels and the photovoltaic support structures.

3. The path planning method for maneuverable unmanned aerial vehicles based on feature point calculation according to claim 2, characterized in that: In S3, morphological processing is performed on the first target connected region to obtain the second target connected region corresponding to the photovoltaic module string, including: The first target connected region is subjected to expansion and corrosion treatment to eliminate the gaps between different first target connected regions, thereby obtaining the second target connected region corresponding to the photovoltaic module string.

4. The path planning method for maneuverable unmanned aerial vehicles based on feature point calculation according to claim 3, characterized in that: In S4, feature points are sampled and calculated on the edge contour of the second target connected region to obtain the straight line information of the photovoltaic module string edge. Based on the straight line information, the preset inspection path of the UAV is determined, including: Multiple feature points on the edge contour of the second target connected region are obtained and calculated to obtain the straight line information of the edge of the photovoltaic module string, and to determine the multiple line segments corresponding to the edge of the photovoltaic module string. Based on a preset coordinate system, determine whether multiple line segments are collinear, and determine the preset inspection path of the UAV based on these line segments.

5. The path planning method for maneuverable unmanned aerial vehicles based on feature point calculation according to claim 4, characterized in that: S4 involves sampling and calculating feature points on the edge contour of the second target's connected region to obtain straight line information of the photovoltaic module string edges, and determining the UAV's preset inspection path based on this straight line information. It also includes: During actual inspection, when the ratio of the sum of pixel values ​​of all pixels in the binary image of the photovoltaic module string to the total sum of pixels in the image frame acquired by the drone is less than a preset threshold, it is determined that the drone is located at the boundary of the area to be inspected, and the drone is controlled to perform a turning action according to the preset inspection path.

6. The path planning method for maneuverable unmanned aerial vehicles based on feature point calculation according to claim 1, characterized in that: In S7, the defective photovoltaic module is identified by combining the surface defect identification results of the first and second photovoltaic modules. Based on the geographical location information of the defective photovoltaic module, the preset inspection path of the UAV is optimized to obtain the optimized inspection path, which includes: By combining the surface defect identification results of the first photovoltaic module and the surface defect identification results of the second photovoltaic module, the defective photovoltaic module is identified, and the geographical location information of the defective photovoltaic module is obtained; Based on the geographical location information of the defective photovoltaic module, the inspection route, number of detours, and detour time at the photovoltaic module string to which the defective photovoltaic module belongs in the preset inspection path are optimized to obtain the optimized inspection path.

Images