Component segmentation method and device based on unmanned aerial vehicle infrared photovoltaic image

Abstract

The invention relates to the technical field of computer vision and image processing, in particular to a component segmentation method, device and equipment based on an unmanned aerial vehicle infrared photovoltaic image, and a storage medium, which effectively suppress background noise of the unmanned aerial vehicle infrared photovoltaic image in combination with sub-pixel structure feature reconstruction and a multi-modal gradient edge attention mechanism, and improve the detection accuracy of the unmanned aerial vehicle infrared photovoltaic image. According to the method, the background misjudgment rate is reduced, and the accuracy of identifying the sub-pixel-level structure is improved, so that the component edge segmentation precision of the unmanned aerial vehicle infrared photovoltaic image is improved, the problems that accurate component segmentation is difficult to perform on the unmanned aerial vehicle infrared photovoltaic image and missing detection and false detection are easy to occur in a complex illumination and shielding scene are effectively solved, and industrial detection requirements are met.

Description

Technical Field

[0001] The present invention relates to the field of computer vision and image processing technology, and in particular to a component segmentation method, device, computer equipment and storage medium based on unmanned aerial vehicle infrared photovoltaic images. Background Art

[0002] With the rapid development of the photovoltaic industry, efficient and accurate monitoring of photovoltaic panels is crucial. Photovoltaic imaging captured by drones equipped with infrared equipment can quickly and extensively inspect photovoltaic panels, identifying potential faults and defects.

[0003] At present, the use of deep learning models for photovoltaic component segmentation has become the mainstream method, but there are still problems such as complex background interference, difficulty in detecting small targets, high data labeling requirements, large computing resource consumption, insufficient model generalization ability, the influence of lighting changes and occlusions, insufficient real-time performance, and post-processing complexity. It is difficult to accurately segment components in drone infrared photovoltaic images, and missed detections and false detections are prone to occur. Summary of the Invention

[0004] Based on this, the purpose of the present invention is to provide a component segmentation method, device, equipment and storage medium based on drone infrared photovoltaic images, combined with sub-pixel structure feature reconstruction and multimodal gradient edge attention mechanism, to effectively suppress the background noise of drone infrared photovoltaic images, reduce the background misjudgment rate, and improve the accuracy of identifying sub-pixel level structures, thereby improving the component edge segmentation accuracy of drone infrared photovoltaic images, effectively overcoming the problem of difficult to accurately segment drone infrared photovoltaic images in complex lighting and occlusion scenes, and prone to missed detection and false detection, thereby meeting industrial detection needs.

[0005] In a first aspect, an embodiment of the present application provides a component segmentation method based on drone infrared photovoltaic images, comprising the following steps:

[0006] Obtaining a UAV infrared photovoltaic image to be segmented and a preset photovoltaic image segmentation model, wherein the photovoltaic image segmentation model includes a feature extraction module, a feature reconstruction module, a feature enhancement module, a component segmentation module, and a post-processing module;

[0007] Inputting the infrared photovoltaic image of the drone to be segmented into the feature extraction module to perform multi-scale feature extraction to obtain feature extraction images of several scales;

[0008] Inputting the feature extraction images at the plurality of scales into the feature reconstruction module to perform sub-pixel structure feature reconstruction to obtain a feature reconstruction image;

[0009] Inputting the feature reconstruction map into the feature enhancement module, performing edge feature enhancement according to the multimodal gradient edge attention mechanism, and obtaining an edge feature enhancement map;

[0010] The edge feature enhancement map is input into the component segmentation module to perform component segmentation, and a component segmentation map is obtained as the component segmentation result of the infrared photovoltaic image of the drone to be segmented.

[0011] In a second aspect, an embodiment of the present application provides a component segmentation device based on drone infrared photovoltaic imaging, comprising:

[0012] A data acquisition module is used to obtain the UAV infrared photovoltaic image to be segmented and a preset photovoltaic image segmentation model, wherein the photovoltaic image segmentation model includes a feature extraction module, a feature reconstruction module, a feature enhancement module, a component segmentation module and a post-processing module;

[0013] A multi-scale feature extraction module is used to input the infrared photovoltaic image of the drone to be segmented into the feature extraction module to perform multi-scale feature extraction and obtain feature extraction images of several scales;

[0014] A sub-pixel structure feature reconstruction module, configured to input the feature extraction images at the plurality of scales into the feature reconstruction module to perform sub-pixel structure feature reconstruction to obtain a feature reconstruction image;

[0015] An edge feature enhancement module is used to input the feature reconstruction map into the feature enhancement module, perform edge feature enhancement according to a multimodal gradient edge attention mechanism, and obtain an edge feature enhancement map;

[0016] The image component segmentation module is used to input the edge feature enhancement map into the component segmentation module for component segmentation, and obtain a component segmentation map as the component segmentation result of the drone infrared photovoltaic image to be segmented.

[0017] In a third aspect, an embodiment of the present application provides a computer device comprising: a processor, a memory, and a computer program stored on the memory and executable on the processor; when the computer program is executed by the processor, the steps of the component segmentation method based on drone infrared photovoltaic imaging as described in the first aspect are implemented.

[0018] In a fourth aspect, an embodiment of the present application provides a storage medium storing a computer program, which, when executed by a processor, implements the steps of the component segmentation method based on drone infrared photovoltaic imaging as described in the first aspect.

[0019] In an embodiment of the present application, a component segmentation method, apparatus, computer equipment and storage medium based on drone infrared photovoltaic images are provided, which, combined with sub-pixel structure feature reconstruction and multimodal gradient edge attention mechanism, effectively suppress the background noise of drone infrared photovoltaic images, reduce the background misjudgment rate, and improve the accuracy of identifying sub-pixel structures, thereby improving the component edge segmentation accuracy of drone infrared photovoltaic images, effectively overcoming the problem of difficult to accurately segment drone infrared photovoltaic images in complex lighting and occlusion scenes, and prone to missed detection and false detection, thereby meeting industrial detection needs.

[0020] For better understanding and implementation, the present invention is described in detail below with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] Figure 1 A schematic flow chart of a component segmentation method based on drone infrared photovoltaic imaging provided in one embodiment of the present application;

[0022] Figure 2 A schematic diagram of the process of step S2 in the component segmentation method based on drone infrared photovoltaic imaging provided in one embodiment of the present application;

[0023] Figure 3 A schematic diagram of the process of step S2 in a component segmentation method based on drone infrared photovoltaic imaging provided in another embodiment of the present application;

[0024] Figure 4 A schematic diagram of the process of step S3 in the component segmentation method based on drone infrared photovoltaic imaging provided in one embodiment of the present application;

[0025] Figure 5 A schematic diagram of the process of step S4 in the component segmentation method based on drone infrared photovoltaic imaging provided in one embodiment of the present application;

[0026] Figure 6 A flowchart of a component segmentation method based on drone infrared photovoltaic imaging provided in yet another embodiment of the present application;

[0027] Figure 7 A schematic diagram of the process of S6 in the component segmentation method based on drone infrared photovoltaic imaging provided in one embodiment of the present application;

[0028] Figure 8 A schematic diagram of the structure of a component segmentation device based on drone infrared photovoltaic imaging provided in one embodiment of the present application;

[0029] Figure 9 A schematic diagram of the structure of a computer device provided in one embodiment of the present application. DETAILED DESCRIPTION

[0030] Exemplary embodiments will be described in detail herein, with examples illustrated in the accompanying drawings. In the following description, when referring to the drawings, identical numerals in different figures represent identical or similar elements, unless otherwise indicated. The embodiments described in the following exemplary embodiments are not intended to represent all embodiments consistent with the present application. Rather, they are merely examples of apparatus and methods consistent with certain aspects of the present application, as detailed in the appended claims.

[0031] The terms used in this application are for the purpose of describing specific embodiments only and are not intended to limit this application. As used in this application and the appended claims, the singular forms "a," "an," "the," and "the" are intended to include the plural forms, unless the context clearly indicates otherwise. It should also be understood that the term "and / or" as used herein refers to and encompasses any and all possible combinations of one or more of the associated listed items.

[0032] It should be understood that although the terms first, second, third, etc. may be used in this application to describe various information, such information should not be limited to these terms. These terms are only used to distinguish information of the same type from each other. For example, without departing from the scope of this application, first information may also be referred to as second information, and similarly, second information may also be referred to as first information. Depending on the context, the words "if" / "if" as used herein may be interpreted as "at the time of" or "when" or "in response to a determination."

[0033] See also Figure 1 , Figure 1 A schematic flow chart of a component segmentation method based on drone infrared photovoltaic imaging provided in one embodiment of the present application, the method comprising the following steps:

[0034] S1: Obtain the UAV infrared photovoltaic image to be segmented and the preset photovoltaic image segmentation model.

[0035] The method for segmenting components based on drone infrared photovoltaic images is performed by a segmentation device for the method (hereinafter referred to as the segmentation device). In an optional embodiment, the segmentation device can be a computer device, a server, or a server cluster composed of multiple computer devices.

[0036] In this embodiment, the segmentation device can obtain the drone infrared photovoltaic image to be segmented input by the user, or obtain the drone infrared photovoltaic image to be segmented from a preset database. Specifically, the drone infrared photovoltaic image is a large-area aerial image of the photovoltaic panel collected by a drone equipped with infrared equipment flying over the photovoltaic panel area, which includes all areas where defects or abnormalities may exist.

[0037] The segmentation device obtains a preset photovoltaic image segmentation model, wherein the photovoltaic image segmentation model includes a feature extraction module, a feature reconstruction module, a feature enhancement module, a component segmentation module and a post-processing module.

[0038] S2: Inputting the infrared photovoltaic image of the drone to be segmented into the feature extraction module to perform multi-scale feature extraction to obtain feature extraction maps of several scales; performing feature fusion based on the feature extraction maps of several scales to obtain a feature reconstruction map.

[0039] In this embodiment, the segmentation device inputs the drone infrared photovoltaic image to be segmented into the feature extraction module to perform multi-scale feature extraction to obtain feature extraction maps of several scales; and performs feature fusion based on the feature extraction maps of several scales to obtain a feature reconstruction map.

[0040] The feature extraction module includes an encoding module and a multi-scale pyramid pooling module; the multi-scale pyramid pooling module includes a convolution layer and several pooling layers connected in sequence; please refer to Figure 2 , Figure 2 The flowchart of S2 in the component segmentation method based on drone infrared photovoltaic imagery provided in one embodiment of the present application includes steps S21 to S22, which are specifically as follows:

[0041] S21: Inputting the infrared photovoltaic image of the UAV into the encoding module for encoding processing to obtain a feature coding map.

[0042] The encoding module uses the Segment Anything Model 2 (SAM2) encoder, a next-generation image and video segmentation model from Meta. SAM2 is designed to enable real-time, predictable segmentation of objects in any image or video. It extends the original SAM model and has made significant progress in video segmentation, enabling it to handle dynamic video content and produce high-quality segmentation results.

[0043] In this embodiment, the segmentation device inputs the infrared photovoltaic image of the UAV into the SAM2 model for encoding processing to obtain a feature coding map.

[0044] S22: Input the feature coding map into the convolution layer of the multi-scale pyramid pooling module for convolution processing to obtain a convolution feature map, use the convolution feature map as the input feature map of the first pooling layer for pooling processing, and obtain the feature pooling map output by the first pooling layer; use the feature pooling map output by the first pooling layer as the input feature of the next pooling layer, repeatedly perform pooling processing until the feature pooling map output by the last pooling layer is obtained, and use the feature pooling map as the feature extraction map.

[0045] Although the SAM2 model has the ability to segment small samples, its single-scale feature extraction is difficult to adapt to the balance between global semantics and local details of UAV infrared photovoltaic images.

[0046] In order to overcome the above problems, in this embodiment, the segmentation device inputs the feature coding map into the convolution layer of the multi-scale pyramid pooling module (SPP) for convolution processing to obtain a convolution feature map, and uses the convolution feature map as the input feature map of the first pooling layer for pooling processing to obtain the feature pooling map output by the first pooling layer; the feature pooling map output by the first pooling layer is used as the input feature of the next pooling layer, and the pooling process is repeatedly performed until the feature pooling map output by the last pooling layer is obtained, and the feature pooling map is used as the feature extraction map to obtain feature extraction maps at different levels.

[0047] Specifically, the feature extraction maps of the several scales include feature extraction maps of the first scale to the fourth scale, which correspond to resolutions of 1 / 4, 1 / 8, 1 / 16 and 1 / 32 of the drone infrared photovoltaic image, respectively. The low-resolution feature extraction map can capture the global semantic information of the image and help identify the approximate position and category of the photovoltaic module in the image; the high-resolution feature extraction map retains rich details, such as the texture of the photovoltaic module surface, providing a basis for accurate segmentation.

[0048] See also Figure 3 , Figure 3 The flowchart of S2 in the component segmentation method based on drone infrared photovoltaic imaging provided in another embodiment of the present application includes steps S23 to S24, which are specifically as follows:

[0049] S23: Calculating the feature map pixel mean and the feature map pixel standard deviation for the feature extraction map at the last scale to obtain the feature map pixel mean and the feature map pixel standard deviation.

[0050] In this embodiment, the segmentation device calculates the feature map pixel mean and the feature map pixel standard deviation for the feature extraction map at the last scale, i.e., the feature extraction map with a resolution of 1 / 32 of the infrared photovoltaic image of the drone, to obtain the feature map pixel mean and the feature map pixel standard deviation, wherein the feature map pixel mean is:

[0051]

[0052] Wherein, μ is the pixel mean of the feature map, H is , W is , and F4(i, j) is the feature extraction map of the last scale, that is, the element of position (i, j) of the feature extraction map of the fourth scale.

[0053] The standard deviation of the feature map pixels is:

[0054]

[0055] Where σ is the standard deviation of feature map pixels.

[0056] S24: performing illumination correction on the feature extraction image at the last scale according to the feature image pixel mean and the feature image pixel standard deviation to obtain the processed feature extraction image at the last scale.

[0057] In this embodiment, the segmentation device performs illumination correction on the feature extraction map at the last scale according to the feature map pixel mean, the feature map pixel standard deviation, and a preset correction algorithm to obtain the processed feature extraction map at the last scale, wherein the correction algorithm is:

[0058]

[0059] Where, F 4corr is the feature extraction image of the last scale after processing, and F4 is the feature extraction image of the last scale.

[0060] Retinex theory is used to decompose the reflection component and illumination component of the UAV infrared photovoltaic image. The low-resolution feature extraction map, that is, the feature extraction map at the last scale, is subjected to illumination normalization to suppress interference from overexposed / underexposed areas. The high-resolution feature extraction map retains the details of the reflection component and enhances noise robustness.

[0061] S3: Inputting the feature extraction images of the plurality of scales into the feature reconstruction module to perform sub-pixel structure feature reconstruction to obtain a feature reconstruction image.

[0062] In this embodiment, the segmentation device inputs the feature extraction images of the plurality of scales into the feature reconstruction module to perform sub-pixel structure feature reconstruction to obtain a feature reconstruction image.

[0063] See also Figure 4 , Figure 4 The flowchart of S3 in the component segmentation method based on drone infrared photovoltaic imagery provided in one embodiment of the present application includes steps S31 to S32, which are specifically as follows:

[0064] S31: According to the feature extraction images of several scales, a step-by-step bilinear interpolation method is used to upsample and fuse the feature extraction image of the last scale upwards to obtain feature fusion images of several scales.

[0065] In this embodiment, the segmentation device uses a step-by-step bilinear interpolation method based on the feature extraction maps of several scales to upsample and fuse the feature extraction map from the last scale upwards layer by layer to obtain feature fusion maps of several scales, as follows:

[0066] F 3fusion =UPsample(F 4corr )+F3)

[0067] F 2fusion =UPsample(F 3fusion )+F2)

[0068] F 1fusion =UPsample(F 2fusion )+F1)

[0069] Where, F fusion 、F 2fusion 、F 3fusion are the feature fusion maps of the first scale, second scale and third scale respectively, F1, F2 and F3 are the feature extraction maps of the first scale, second scale and third scale respectively, and UPsample(·) is the upsampling function.

[0070] The low-resolution feature extraction map is upsampled using bilinear interpolation to make it consistent with the resolution of the high-resolution feature extraction map of the previous scale, and then fused using the bilinear interpolation method. Bilinear interpolation is a mature image scaling algorithm that calculates new pixel values ​​by linear interpolation of adjacent pixels. It can better maintain the smoothness and continuity of the image when enlarging the image, so that deep semantic information can be combined with shallow detail information to obtain a feature fusion map of the corresponding scale, thereby enhancing the accuracy of identifying photovoltaic components in drone infrared photovoltaic images.

[0071] S32: performing sub-pixel convolution on the feature fusion maps of several scales to obtain sub-pixel convolution feature maps of several scales; performing weighted fusion according to the sub-pixel convolution feature maps of several scales and adaptive weight parameters corresponding to preset scales to obtain a feature reconstruction map.

[0072] In this embodiment, the segmentation device performs sub-pixel convolution on the feature fusion images at several scales to reconstruct sub-pixel structural features (for example, the structural features of components with a size smaller than a single-pixel resolution such as the cell gaps and microcracks in photovoltaic modules, which are key defects), and obtains sub-pixel convolution feature maps at several scales, thereby improving the accuracy of photovoltaic module recognition.

[0073] The segmentation device performs weighted fusion based on the sub-pixel convolution feature maps of several scales, the adaptive weight parameters corresponding to the preset scales, and the preset weight fusion algorithm to optimize the contribution of the features at each scale and obtain a feature reconstruction map, wherein the adaptive weight parameters are normalized weights generated by 1×1 convolution and Softmax, reflecting the importance of the features at each scale. The weight fusion algorithm is:

[0074]

[0075] Where, F Refactoring is the feature reconstruction graph, α i is the adaptive weight parameter corresponding to the i-th scale, n is the number of scales, F ifusion is the feature fusion map of the i-th scale.

[0076] S4: Input the feature reconstruction map into the feature enhancement module, perform edge feature enhancement according to the multimodal gradient edge attention mechanism, and obtain an edge feature enhancement map.

[0077] In this embodiment, the segmentation device inputs the feature reconstruction map into the feature enhancement module, performs edge feature enhancement according to the multimodal gradient edge attention mechanism, and obtains an edge feature enhancement map.

[0078] See also Figure 5 , Figure 5 The flowchart of S4 in the component segmentation method based on drone infrared photovoltaic imagery provided in one embodiment of the present application includes steps S41 to S42, which are specifically as follows:

[0079] S41: Using the Sobel operator, a multimodal gradient map is calculated for the infrared photovoltaic image of the UAV to be segmented to obtain an intensity gradient map and a texture gradient map; the intensity gradient map and the texture gradient map are fused to obtain a composite edge response map.

[0080] In this embodiment, the segmentation device uses the Sobel operator to perform multimodal gradient map calculation on the feature reconstruction map to obtain an intensity gradient map and a texture gradient map.

[0081] Specifically, for the intensity gradient map, the segmentation device obtains the intensity gradient map according to the infrared photovoltaic image of the drone to be segmented, the Sobel operator, and a preset intensity gradient algorithm, wherein the intensity gradient algorithm is:

[0082]

[0083] Where, is the intensity gradient map, I is the infrared photovoltaic image of the UAV to be segmented, S x is the Sobel operator, * indicates a convolution operation.

[0084] For the texture gradient map, the segmentation device adopts the local binary pattern (LBP) method, which generates a binary pattern by comparing the grayscale values ​​of the central pixel and its neighboring pixels. The grayscale image is obtained according to the infrared photovoltaic image of the drone to be segmented and a preset grayscale algorithm, wherein the grayscale algorithm is:

[0085]

[0086] Where T(x,y) is the element at position (x,y) in the grayscale image T, δ(·) is the value function, That is, when the grayscale difference between the central pixel and its neighboring pixels is greater than 0, the value is 1, and when the difference is less than 0, the value is 0.

[0087] I(x,y) is the element at position (x,y) in the infrared photovoltaic image I of the drone to be segmented, k represents the index number of the neighborhood pixel, I(x k ,y k ) is the position (x, y) around the position (x, y) in the infrared photovoltaic image I of the UAV to be segmented. k ,y k ) elements.

[0088] A texture gradient map is obtained according to the grayscale map, the Sobel operator, and a preset texture gradient algorithm, wherein the texture gradient algorithm is:

[0089]

[0090] Where, is the texture gradient map.

[0091] The segmentation device fuses the intensity gradient map and the texture gradient map according to the intensity gradient map, the texture gradient map, and a preset gradient map fusion algorithm to obtain a composite edge response map, wherein the gradient map fusion algorithm is:

[0092]

[0093] Where, |ΔF fusion | is the composite edge response map, α and β are the preset first and second learnable parameters, respectively, initialized to 0.5. Through back-propagation optimization, the contribution of the two gradients to the sub-pixel edge is adaptively balanced.

[0094] S42: Calculate the attention weight according to the composite edge response map to obtain an attention weight parameter; multiply the feature reconstruction map and the attention weight parameter element by element to obtain an edge feature enhancement map.

[0095] In this embodiment, the segmentation device calculates the attention weight according to the composite edge response map and the preset attention weight algorithm, maps the composite edge response map to the attention weight space through a 1x1 convolution function, and normalizes the weight to [0, 1] through a Sigmoid function. The high-weight area (close to 1) corresponds to the edge of the photovoltaic panel, and the low-weight area (close to 0) suppresses background noise. The attention weight parameter is obtained, which can highlight the weight of the edge area of ​​the component. The attention weight algorithm is:

[0096] A edag =σ(Conv 1×1 (|ΔF fusion |))

[0097] Where A edag is the attention weight parameter, σ(·) is the Sigmoid function, Conv 1×1 (·) is a 1x1 convolution function.

[0098] The segmentation device multiplies the feature reconstruction map by the attention weight parameter element by element to strengthen the sub-pixel structure and obtain an edge feature enhancement map, wherein the edge feature enhancement map is:

[0099]

[0100] Where, F att is the edge feature enhancement map, is the element-wise multiplication symbol.

[0101] S5: Inputting the edge feature enhancement map into the component segmentation module to perform component segmentation, and obtaining a component segmentation map as a component segmentation result of the infrared photovoltaic image of the UAV to be segmented.

[0102] In this embodiment, the segmentation device inputs the edge feature enhancement map into the component segmentation module to perform component segmentation, and obtains a component segmentation map as the component segmentation result of the infrared photovoltaic image of the drone to be segmented.

[0103] Specifically, the component segmentation module adopts the decoder of the SAM2 model. The segmentation device inputs the edge feature enhancement map into the decoder, converts the mask prompt into a feature vector according to the edge feature enhancement map and the preset mask prompt, performs feature splicing with the edge feature enhancement map, and sequentially processes the feature splicing map through a multi-layer perceptron and a convolutional layer, maps the fused features to the probability space, and generates a binary mask M. pred , as the component segmentation map.

[0104] Combining sub-pixel structure feature reconstruction and multimodal gradient edge attention mechanism, the background noise of UAV infrared photovoltaic images can be effectively suppressed, the background misjudgment rate can be reduced, and the accuracy of identifying sub-pixel structures can be improved, thereby improving the component edge segmentation accuracy of UAV infrared photovoltaic images. It effectively overcomes the problem of difficult to accurately segment components of UAV infrared photovoltaic images in complex lighting and occlusion scenes, which is prone to missed detection and false detection, and meets the needs of industrial detection.

[0105] See also Figure 6 , Figure 6 A schematic flow chart of a component segmentation method based on drone infrared photovoltaic imaging provided in another embodiment of the present application includes step S6, which is specifically as follows:

[0106] S6: According to the component segmentation map, the infrared photovoltaic image of the UAV to be segmented, and the optimized post-processing, a component segmentation map after optimization is obtained as a component segmentation result of the infrared photovoltaic image of the UAV to be segmented.

[0107] In this embodiment, the segmentation device obtains the optimized component segmentation map according to the component segmentation map, the infrared photovoltaic image of the drone to be segmented, and the optimized post-processing as the component segmentation result of the infrared photovoltaic image of the drone to be segmented.

[0108] See also Figure 7 , Figure 7 A flow chart of S6 in a component segmentation method based on drone infrared photovoltaic imaging provided in another embodiment of the present application includes steps S61 to S64, which are specifically as follows:

[0109] S61: Obtain the noise pixel ratio and noise standard deviation of the component segmentation map, calculate the structure kernel size based on the noise pixel ratio and noise standard deviation, and obtain the structure kernel size parameter; perform dilation-erosion operation on the component segmentation map based on the structure kernel size parameter to obtain a first intermediate component segmentation map.

[0110] The dilation operation can expand the boundaries of the object and connect some previously separated small areas, while the erosion operation can remove some noise and small bumps. By first dilating and then closing the erosion operation, small holes in the segmentation result can be filled, making the segmented area more complete.

[0111] In this embodiment, the segmentation device obtains the noise pixel ratio of the component segmentation image, calculates the structure kernel size based on the noise pixel ratio, and obtains the structure kernel size parameter to ensure that small holes are filled while retaining sub-pixel gaps. The structure kernel size parameter is:

[0112] K size =[σ′·log(Nnoise )]

[0113] Where K size is the structural kernel size parameter, σ′ is the noise standard deviation, N noise is the noise pixel ratio.

[0114] The segmentation device performs a dilation-erosion operation on the component segmentation map according to the structural core size parameter to obtain a first intermediate component segmentation map, wherein the intermediate component segmentation map is:

[0115] M close =Erode(Dilate(M pred ))

[0116] Where M close is the intermediate component segmentation map, M pred is the component segmentation map, Erode(·) is the erosion function, and Dilate(·) is the dilation function.

[0117] S62: Obtain a one-dimensional array corresponding to the intensity gradient map and the length of the one-dimensional array, arrange the one-dimensional array in ascending order, and use linear interpolation to calculate a position index of the one-dimensional array after the ascending order according to the length of the one-dimensional array to obtain a position index.

[0118] In this embodiment, the segmentation device flattens the intensity gradient map to obtain a one-dimensional array corresponding to the intensity gradient map and the length of the one-dimensional array, sorts the one-dimensional array in ascending order, and uses linear interpolation to calculate a position index for the one-dimensional array after the ascending order according to the length of the one-dimensional array to obtain a position index, where the position index is:

[0119] z=(L-1)×0.95

[0120] Where z is the position index and L is the length of the one-dimensional array.

[0121] S63: If the position index is an integer, obtain the value of the position index corresponding to the one-dimensional array after ascending order as the first edge detection threshold; if the position index is not an integer, take the integer part of the position index as the first sub-position index and the decimal part of the position index as the second sub-position index, and perform edge detection threshold calculation based on the first sub-position index, the second sub-position index and the one-dimensional array after ascending order to obtain the first edge detection threshold; multiply the first edge detection threshold by a preset low threshold ratio to obtain the second edge detection threshold.

[0122] If the position index is an integer, in this embodiment, the segmentation device obtains the value of the position index corresponding to the one-dimensional array after the ascending order is arranged as the first edge detection threshold.

[0123] If the position index is not an integer, the segmentation device takes the integer part of the position index as the first sub-position index and the decimal part of the position index as the second sub-position index, and performs edge detection threshold calculation based on the first sub-position index, the second sub-position index, the one-dimensional array after ascending order, and a preset threshold algorithm to obtain a first edge detection threshold, wherein the threshold algorithm is:

[0124] G[K]=G[m]+f×(G[m+1]-G[m])

[0125] Where G[K] is the first edge detection threshold, m is the first sub-position index, G[m] is the value of the first sub-position index corresponding to the one-dimensional array after ascending order, and f is the second sub-position index.

[0126] S64: Perform edge detection on the drone infrared photovoltaic image to be segmented according to the first edge detection threshold and the second edge detection threshold to obtain a second intermediate component segmentation map; overlap the first intermediate component segmentation map and the second intermediate component segmentation map to obtain the optimized post-processing component segmentation map.

[0127] Setting a dual threshold effectively avoids false and missed edge detections. The low threshold is used to detect weaker edges, while the high threshold is used to detect stronger edges. Combining the two thresholds accurately extracts the true boundaries of photovoltaic modules. In this embodiment, the segmentation device performs edge detection on the drone infrared photovoltaic image to be segmented based on the first and second edge detection thresholds to obtain a second intermediate module segmentation map.

[0128] The segmentation device overlaps the first intermediate component segmentation map and the second intermediate component segmentation map to obtain the optimized component segmentation map, which can further refine the boundary and improve the quality of the segmentation result.

[0129] Specifically, the segmentation device uses a logical "AND" operation to overlap the first intermediate component segmentation graph and the second intermediate component segmentation graph to obtain the optimized component segmentation graph, as described below:

[0130]

[0131] Where M final To optimize the post-processed component segmentation map, Canny (I input ) is the second intermediate component segmentation diagram.

[0132] Please refer to Figure 8 , Figure 8 This is a schematic diagram of the structure of a component segmentation device based on drone infrared photovoltaic imaging provided by one embodiment of the present application. The device can implement all or part of the component segmentation device based on drone infrared photovoltaic imaging through software, hardware, or a combination of both. The device 8 includes:

[0133] A data acquisition module 81 is used to obtain the UAV infrared photovoltaic image to be segmented and a preset photovoltaic image segmentation model, wherein the photovoltaic image segmentation model includes a feature extraction module, a feature reconstruction module, a feature enhancement module, a component segmentation module, and a post-processing module;

[0134] A multi-scale feature extraction module 82 is used to input the infrared photovoltaic image of the drone to be segmented into the feature extraction module to perform multi-scale feature extraction and obtain feature extraction images of several scales;

[0135] A sub-pixel structure feature reconstruction module 83 is used to input the feature extraction images of the plurality of scales into the feature reconstruction module to perform sub-pixel structure feature reconstruction to obtain a feature reconstruction image;

[0136] An edge feature enhancement module 84 is configured to input the feature reconstruction map into the feature enhancement module, perform edge feature enhancement according to a multimodal gradient edge attention mechanism, and obtain an edge feature enhancement map;

[0137] The image component segmentation module 85 is used to input the edge feature enhancement map into the component segmentation module to perform component segmentation, and obtain a component segmentation map as the component segmentation result of the drone infrared photovoltaic image to be segmented.

[0138] In an embodiment of the present application, a data acquisition module is used to obtain a drone infrared photovoltaic image to be segmented and a preset photovoltaic image segmentation model, wherein the photovoltaic image segmentation model includes a feature extraction module, a feature reconstruction module, a feature enhancement module, a component segmentation module and a post-processing module; through a multi-scale feature extraction module, the drone infrared photovoltaic image to be segmented is input into the feature extraction module for multi-scale feature extraction to obtain feature extraction maps of several scales; through a sub-pixel structure feature reconstruction module, the feature extraction maps of the several scales are input into the feature reconstruction module for sub-pixel structure feature reconstruction to obtain a feature reconstruction map; through an edge feature enhancement module, the feature reconstruction map is input into the feature enhancement module, and edge feature enhancement is performed according to a multimodal gradient edge attention mechanism to obtain an edge feature enhancement map; through an image component segmentation module, the edge feature enhancement map is input into the component segmentation module for component segmentation to obtain a component segmentation map as the component segmentation result of the drone infrared photovoltaic image to be segmented. Combining sub-pixel structure feature reconstruction and multimodal gradient edge attention mechanism, the background noise of UAV infrared photovoltaic images can be effectively suppressed, the background misjudgment rate can be reduced, and the accuracy of identifying sub-pixel structures can be improved, thereby improving the component edge segmentation accuracy of UAV infrared photovoltaic images. It effectively overcomes the problem of difficult to accurately segment components of UAV infrared photovoltaic images in complex lighting and occlusion scenes, which is prone to missed detection and false detection, and meets the needs of industrial detection.

[0139] Please refer to Figure 9 , Figure 9 This is a schematic diagram of the structure of a computer device provided in one embodiment of the present application. The computer device 9 includes: a processor 91, a memory 92, and a computer program 93 stored in the memory 92 and executable on the processor 91; the computer device may store multiple instructions, which are suitable for being loaded and executed by the processor 91. Figures 1 to 7 The specific execution process can be found in the method steps shown in Figures 1 to 7 The specific description shown will not be repeated here.

[0140] The processor 91 may include one or more processing cores. The processor 91 utilizes various interfaces and circuits to connect to various components within the server. By running or executing instructions, programs, code sets, or instruction sets stored in the memory 92, as well as accessing data within the memory 92, the processor 91 performs various functions and processes data in the component segmentation device 8 based on the infrared photovoltaic image of a drone. Optionally, the processor 91 may be implemented in the form of at least one of a digital signal processing (DSP), a field-programmable gate array (FPGA), or a programmable logic array (PLA). The processor 91 may integrate one or a combination of a central processing unit (CPU), a graphics processing unit (GPU), and a modem. The CPU primarily processes the operating system, user interface, and application programs; the GPU is responsible for rendering and drawing the content displayed on the touch screen; and the modem handles wireless communications. It is understood that the modem may not be integrated into the processor 91 but implemented as a separate chip.

[0141] Among them, the memory 92 may include a random access memory 92 (Random Access Memory, RAM), and may also include a read-only memory 92 (Read-Only Memory). Optionally, the memory 92 includes a non-transitory computer-readable storage medium. The memory 92 can be used to store instructions, programs, codes, code sets or instruction sets. The memory 92 may include a program storage area and a data storage area, wherein the program storage area may store instructions for implementing an operating system, instructions for at least one function (such as touch instructions, etc.), instructions for implementing the above-mentioned various method embodiments, etc.; the data storage area may store data involved in the above-mentioned various method embodiments, etc. The memory 92 may also optionally be at least one storage device located away from the aforementioned processor 91.

[0142] The embodiment of the present application also provides a storage medium, which can store multiple instructions, which are suitable for the processor to load and execute the above Figures 1 to 7 The specific execution process can be found in the method steps shown in Figures 1 to 7 The specific description shown will not be repeated here.

[0143] Those skilled in the art can clearly understand that, for the convenience and brevity of description, only the division of the above-mentioned functional units and modules is used as an example for illustration. In actual applications, the above-mentioned functions can be distributed and completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiment can be integrated into one processing unit, or each unit can exist physically alone, or two or more units can be integrated into one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units. In addition, the specific names of the functional units and modules are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of this application. The specific working process of the units and modules in the above-mentioned system can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here.

[0144] In the above embodiments, the description of each embodiment has its own focus. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant description of other embodiments.

[0145] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraint algorithm of the technical solution. Professionals and technicians can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the present invention.

[0146] In the embodiments provided by the present invention, it should be understood that the disclosed devices / terminal equipment and methods can be implemented in other ways. For example, the device / terminal equipment embodiments described above are only schematic. For example, the division of the modules or units is only a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0147] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0148] In addition, the functional units in the various embodiments of the present invention may be integrated into a single processing unit, each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or software functional units.

[0149] If the integrated module / unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present invention implements all or part of the process in the above-mentioned embodiment method, and can also be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium. When the computer program is executed by a processor, it can implement the steps of the above-mentioned various method embodiments. The computer program includes computer program code, which can be in source code form, object code form, executable file or some intermediate form.

[0150] The present invention is not limited to the above-mentioned embodiments. If various changes or modifications of the present invention do not depart from the spirit and scope of the present invention, and if these changes and modifications fall within the scope of the claims of the present invention and equivalent technologies, the present invention is also intended to include these changes and modifications.

Claims

1. A component segmentation method based on UAV infrared photovoltaic images, characterized in that: The following steps are involved: Obtaining a UAV infrared photovoltaic image to be segmented and a preset photovoltaic image segmentation model, wherein the photovoltaic image segmentation model includes a feature extraction module, a feature reconstruction module, a feature enhancement module, and a component segmentation module; Inputting the infrared photovoltaic image of the drone to be segmented into the feature extraction module to perform multi-scale feature extraction to obtain feature extraction images of several scales; Inputting the feature extraction images at the plurality of scales into the feature reconstruction module to perform sub-pixel structure feature reconstruction to obtain a feature reconstruction image; Inputting the infrared photovoltaic image of the drone to be segmented and the feature reconstruction map into the feature enhancement module, performing edge feature enhancement according to the multimodal gradient edge attention mechanism, and obtaining an edge feature enhancement map; The edge feature enhancement map is input into the component segmentation module to perform component segmentation, and a component segmentation map is obtained as the component segmentation result of the infrared photovoltaic image of the drone to be segmented.

2. The component segmentation method based on UAV infrared photovoltaic imagery according to claim 1 is characterized in that: The feature extraction module includes a coding module and a multi-scale pyramid pooling module; the multi-scale pyramid pooling module includes a convolution layer and several pooling layers connected in sequence; The step of inputting the infrared photovoltaic image of the drone to be segmented into the feature extraction module to perform multi-scale feature extraction to obtain feature extraction images of several scales includes the following steps: Inputting the infrared photovoltaic image of the UAV into the encoding module for encoding processing to obtain a feature coding map; The feature coding map is input into the convolution layer of the multi-scale pyramid pooling module for convolution processing to obtain a convolution feature map, and the convolution feature map is used as the input feature map of the first pooling layer for pooling processing to obtain the feature pooling map output by the first pooling layer; the feature pooling map output by the first pooling layer is used as the input feature of the next pooling layer, and the pooling process is repeatedly performed until the feature pooling map output by the last pooling layer is obtained, and the feature pooling map is used as the feature extraction map.

3. The component segmentation method based on drone infrared photovoltaic images according to claim 2 is characterized in that: The step of inputting the to-be-segmented UAV infrared photovoltaic image into the feature extraction module for multi-scale feature extraction to obtain feature extraction images of several scales further includes the following steps: Calculating the feature map pixel mean and the feature map pixel standard deviation for the feature extraction map at the last scale to obtain the feature map pixel mean and the feature map pixel standard deviation; According to the feature map pixel mean and the feature map pixel standard deviation, illumination correction is performed on the feature extraction map of the last scale to obtain the processed feature extraction map of the last scale.

4. The component segmentation method based on drone infrared photovoltaic imagery according to claim 2 or 3, characterized in that: The step of inputting the feature extraction images at the plurality of scales into the feature reconstruction module to perform sub-pixel structural feature reconstruction to obtain a feature reconstruction image comprises the following steps: According to the feature extraction maps of several scales, a step-by-step bilinear interpolation method is used to upsample and fuse the feature extraction map of the last scale upwards to obtain feature fusion maps of several scales; Sub-pixel convolution is performed on the feature fusion maps of several scales to obtain sub-pixel convolution feature maps of several scales; weighted fusion is performed according to the sub-pixel convolution feature maps of several scales and adaptive weight parameters corresponding to preset scales to obtain a feature reconstruction map.

5. The component segmentation method based on drone infrared photovoltaic images according to claim 4 is characterized in that: The infrared photovoltaic image of the drone to be segmented and the feature reconstruction map are input into the feature enhancement module, and edge feature enhancement is performed according to the multimodal gradient edge attention mechanism to obtain an edge feature enhancement map, including the steps of: Using the Sobel operator, a multimodal gradient map is calculated for the infrared photovoltaic image of the UAV to be segmented to obtain an intensity gradient map and a texture gradient map, and the intensity gradient map and the texture gradient map are fused to obtain a composite edge response map; Attention weight calculation is performed according to the composite edge response map to obtain an attention weight parameter; the feature reconstruction map is element-wise multiplied by the attention weight parameter to obtain an edge feature enhancement map.

6. The component segmentation method based on drone infrared photovoltaic images according to claim 5 is characterized in that: Also includes the steps: According to the component segmentation map, the infrared photovoltaic image of the drone to be segmented, and the optimized post-processing, the component segmentation map after the optimized post-processing is obtained as the component segmentation result of the infrared photovoltaic image of the drone to be segmented.

7. The component segmentation method based on drone infrared photovoltaic images according to claim 6 is characterized in that: The method of obtaining the optimized component segmentation map according to the component segmentation map, the infrared photovoltaic image of the drone to be segmented, and the optimized post-processing comprises the following steps: Obtaining a noise pixel ratio and a noise standard deviation of the component segmentation image, calculating a structural kernel size based on the noise pixel ratio and the noise standard deviation, and obtaining a structural kernel size parameter; performing a dilation-erosion operation on the component segmentation map according to the structure kernel size parameter to obtain a first intermediate component segmentation map; Obtaining a one-dimensional array corresponding to the intensity gradient map and the length of the one-dimensional array, arranging the one-dimensional array in ascending order, and using a linear interpolation method to calculate a position index of the one-dimensional array after the ascending order according to the length of the one-dimensional array to obtain a position index; If the position index is an integer, obtaining the value of the position index corresponding to the one-dimensional array after the ascending order is used as the first edge detection threshold; if the position index is not an integer, taking the integer part of the position index as the first sub-position index and the decimal part of the position index as the second sub-position index, performing edge detection threshold calculation based on the first sub-position index, the second sub-position index, and the one-dimensional array after the ascending order is used to obtain the first edge detection threshold; multiplying the first edge detection threshold by a preset low threshold ratio to obtain the second edge detection threshold; According to the first edge detection threshold and the second edge detection threshold, edge detection is performed on the drone infrared photovoltaic image to be segmented to obtain a second intermediate component segmentation map; the first intermediate component segmentation map and the second intermediate component segmentation map are overlapped to obtain the optimized post-processed component segmentation map.

8. A component segmentation device based on drone infrared photovoltaic imaging, characterized in that: include: A data acquisition module is used to obtain the UAV infrared photovoltaic image to be segmented and a preset photovoltaic image segmentation model, wherein the photovoltaic image segmentation model includes a feature extraction module, a feature reconstruction module, a feature enhancement module, a component segmentation module and a post-processing module; A multi-scale feature extraction module is used to input the infrared photovoltaic image of the drone to be segmented into the feature extraction module to perform multi-scale feature extraction and obtain feature extraction images of several scales; A sub-pixel structure feature reconstruction module, configured to input the feature extraction images at the plurality of scales into the feature reconstruction module to perform sub-pixel structure feature reconstruction to obtain a feature reconstruction image; An edge feature enhancement module is used to input the feature reconstruction map into the feature enhancement module, perform edge feature enhancement according to a multimodal gradient edge attention mechanism, and obtain an edge feature enhancement map; The image component segmentation module is used to input the edge feature enhancement map into the component segmentation module for component segmentation, and obtain a component segmentation map as the component segmentation result of the drone infrared photovoltaic image to be segmented.

9. A computer device, characterized in that: include: A processor, a memory, and a computer program stored in the memory and executable on the processor; when the computer program is executed by the processor, the steps of the component segmentation method based on drone infrared photovoltaic imagery are implemented as described in any one of claims 1 to 7.

10. A storage medium, characterized in that: The storage medium stores a computer program, and when the computer program is executed by the processor, the steps of the component segmentation method based on drone infrared photovoltaic imagery are implemented as described in any one of claims 1 to 7.

Images