Component segmentation method and device based on unmanned aerial vehicle infrared photovoltaic image

Abstract

The present application relates to the technical field of computer vision and image processing, and particularly relates to a component segmentation method and device based on an unmanned aerial vehicle infrared photovoltaic image, equipment and a storage medium, which combines sub-pixel structure feature reconstruction and a multi-modal gradient edge attention mechanism, effectively suppresses background noise of the unmanned aerial vehicle infrared photovoltaic image, reduces the background misjudgment rate, improves the accuracy of identifying sub-pixel level structures, thereby improving the component edge segmentation precision of the unmanned aerial vehicle infrared photovoltaic image, effectively overcoming the problem that it is difficult to accurately segment components of the unmanned aerial vehicle infrared photovoltaic image under complex lighting and shielding scenes, and prone to missing detection and false detection, and meeting the industrial detection requirements.

Description

Technical Field

[0001] This invention relates to the field of computer vision and image processing technology, and in particular to a component segmentation method, apparatus, computer equipment, and storage medium based on UAV infrared photovoltaic images. Background Technology

[0002] With the rapid development of the photovoltaic industry, efficient and accurate monitoring of photovoltaic panels is crucial. Photovoltaic images acquired by drones equipped with infrared devices can be quickly and extensively inspected to identify potential faults and defects.

[0003] Currently, using deep learning models for photovoltaic module segmentation has become the mainstream method, but it still faces problems such as complex background interference, difficulty in detecting small targets, high data annotation requirements, large computational resource consumption, insufficient model generalization ability, the impact of illumination changes and occlusion, insufficient real-time performance, and post-processing complexity. These issues make it difficult to accurately segment photovoltaic modules from UAV infrared images, and can easily lead to missed detections and false detections. Summary of the Invention

[0004] Based on this, the purpose of this invention is to provide a component segmentation method, apparatus, device, and storage medium based on UAV infrared photovoltaic images. By combining sub-pixel structural feature reconstruction and multimodal gradient edge attention mechanism, the background noise of UAV infrared photovoltaic images is effectively suppressed, the background false judgment rate is reduced, and the accuracy of identifying sub-pixel level structures is improved. This improves the component edge segmentation accuracy of UAV infrared photovoltaic images and effectively overcomes the problem of difficulty in accurately segmenting UAV infrared photovoltaic images under complex lighting and occlusion scenarios, which easily leads to missed detections and false detections, thus meeting the needs of industrial inspection.

[0005] In a first aspect, embodiments of this application provide a component segmentation method based on UAV infrared photovoltaic images, comprising the following steps:

[0006] The process involves obtaining an infrared photovoltaic image of a UAV to be segmented and a preset photovoltaic image segmentation model, wherein the photovoltaic image segmentation model includes a feature extraction module, a feature reconstruction module, a feature enhancement module, a component segmentation module, and a post-processing module.

[0007] The UAV infrared photovoltaic image to be segmented is input into the feature extraction module for multi-scale feature extraction to obtain feature extraction maps at several scales.

[0008] The feature extraction maps at the aforementioned scales are input into the feature reconstruction module for sub-pixel structural feature reconstruction to obtain a feature reconstruction map;

[0009] The reconstructed feature map is input into the feature enhancement module, and edge features are enhanced according to the multimodal gradient edge attention mechanism to obtain an enhanced edge feature map.

[0010] The edge feature enhancement map is input into the component segmentation module for component segmentation to obtain a component segmentation map, which serves as the component segmentation result of the UAV infrared photovoltaic image to be segmented.

[0011] Secondly, embodiments of this application provide a component segmentation device based on UAV infrared photovoltaic images, comprising:

[0012] The data acquisition module is used to obtain the UAV infrared photovoltaic image to be segmented and the preset photovoltaic image segmentation model. The photovoltaic image segmentation model includes a feature extraction module, a feature reconstruction module, a feature enhancement module, a component segmentation module, and a post-processing module.

[0013] A multi-scale feature extraction module is used to input the UAV infrared photovoltaic image to be segmented into the feature extraction module for multi-scale feature extraction to obtain feature extraction maps at several scales.

[0014] The sub-pixel structure feature reconstruction module is used to input the feature extraction maps of the several scales into the feature reconstruction module to perform sub-pixel structure feature reconstruction and obtain a feature reconstruction map;

[0015] An edge feature enhancement module is used to input the feature reconstruction map into the feature enhancement module, and perform edge feature enhancement according to the multimodal gradient edge attention mechanism to obtain an edge feature enhancement map;

[0016] The image component segmentation module is used to input the edge feature enhancement map into the component segmentation module for component segmentation to obtain a component segmentation map, which serves as the component segmentation result of the UAV infrared photovoltaic image to be segmented.

[0017] Thirdly, embodiments of this application provide a computer device, including: a processor, a memory, and a computer program stored in the memory and executable on the processor; when the computer program is executed by the processor, it implements the steps of the component segmentation method based on UAV infrared photovoltaic imagery as described in the first aspect.

[0018] Fourthly, embodiments of this application provide a storage medium storing a computer program that, when executed by a processor, implements the steps of the component segmentation method based on UAV infrared photovoltaic images as described in the first aspect.

[0019] This application provides a component segmentation method, apparatus, computer device, and storage medium based on UAV infrared photovoltaic images. By combining sub-pixel structural feature reconstruction and multimodal gradient edge attention mechanism, the background noise of UAV infrared photovoltaic images is effectively suppressed, the background misjudgment rate is reduced, and the accuracy of identifying sub-pixel level structures is improved. This improves the component edge segmentation accuracy of UAV infrared photovoltaic images and effectively overcomes the problem of difficulty in accurately segmenting UAV infrared photovoltaic images under complex lighting and occlusion scenarios, which easily leads to missed detections and false detections, thus meeting the needs of industrial inspection.

[0020] To better understand and implement this invention, the following detailed description is provided in conjunction with the accompanying drawings. Attached Figure Description

[0021] Figure 1 A flowchart illustrating a component segmentation method based on UAV infrared photovoltaic imagery provided in one embodiment of this application;

[0022] Figure 2 This is a flowchart illustrating step S2 of a component segmentation method based on UAV infrared photovoltaic images provided in one embodiment of this application.

[0023] Figure 3 A flowchart illustrating step S2 of a component segmentation method based on UAV infrared photovoltaic imagery provided in another embodiment of this application;

[0024] Figure 4 This is a flowchart illustrating step S3 of a component segmentation method based on UAV infrared photovoltaic images provided in one embodiment of this application.

[0025] Figure 5 This is a flowchart illustrating step S4 of a component segmentation method based on UAV infrared photovoltaic images provided in one embodiment of this application.

[0026] Figure 6 A flowchart illustrating a component segmentation method based on UAV infrared photovoltaic images, provided as another embodiment of this application;

[0027] Figure 7 This is a flowchart illustrating step S6 of a component segmentation method based on UAV infrared photovoltaic images provided in one embodiment of this application.

[0028] Figure 8 A schematic diagram of a component segmentation device based on UAV infrared photovoltaic imagery provided in one embodiment of this application;

[0029] Figure 9 This is a schematic diagram of the structure of a computer device provided in one embodiment of this application. Detailed Implementation

[0030] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims.

[0031] The terminology used in this application is for the purpose of describing particular embodiments only and is not intended to be limiting of the application. The singular forms “a,” “the,” and “the” used in this application and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used herein refers to and includes any or all possible combinations of one or more of the associated listed items.

[0032] It should be understood that although the terms first, second, third, etc., may be used in this application to describe various information, such information should not be limited to these terms. These terms are only used to distinguish information of the same type from one another. For example, without departing from the scope of this application, first information may also be referred to as second information, and similarly, second information may also be referred to as first information. Depending on the context, the word "if" as used herein may be interpreted as "when," "when," or "in response to determination."

[0033] Please see Figure 1 , Figure 1 The flowchart illustrates a component segmentation method based on UAV infrared photovoltaic imagery, as provided in one embodiment of this application. The method includes the following steps:

[0034] S1: Obtain the UAV infrared photovoltaic image to be segmented and the preset photovoltaic image segmentation model.

[0035] The execution entity of the component segmentation method based on UAV infrared photovoltaic imagery is a segmentation device (hereinafter referred to as the segmentation device). In an optional embodiment, the segmentation device may be a computer device, a server, or a server cluster composed of multiple computer devices.

[0036] In this embodiment, the segmentation device can obtain the UAV infrared photovoltaic image to be segmented input by the user, or it can obtain the UAV infrared photovoltaic image to be segmented through a preset database. Specifically, the UAV infrared photovoltaic image is a large-area aerial image of the photovoltaic panel taken by the UAV carrying an infrared device flying over the photovoltaic panel area, which includes all areas that may have defects or abnormalities.

[0037] The segmentation device obtains a preset photovoltaic image segmentation model, wherein the photovoltaic image segmentation model includes a feature extraction module, a feature reconstruction module, a feature enhancement module, a component segmentation module, and a post-processing module.

[0038] S2: Input the UAV infrared photovoltaic image to be segmented into the feature extraction module for multi-scale feature extraction to obtain feature extraction maps at several scales; perform feature fusion based on the feature extraction maps at several scales to obtain a feature reconstruction map.

[0039] In this embodiment, the segmentation device inputs the UAV infrared photovoltaic image to be segmented into the feature extraction module for multi-scale feature extraction to obtain feature extraction maps at several scales; and performs feature fusion based on the feature extraction maps at several scales to obtain a feature reconstruction map.

[0040] The feature extraction module includes an encoding module and a multi-scale pyramid pooling module; the multi-scale pyramid pooling module includes convolutional layers and several pooling layers connected in sequence; please refer to... Figure 2 , Figure 2 The flowchart of S2 in the component segmentation method based on UAV infrared photovoltaic imagery provided in one embodiment of this application includes steps S21 to S22, as follows:

[0041] S21: Input the UAV infrared photovoltaic image into the encoding module for encoding processing to obtain a feature encoding map.

[0042] The encoding module employs the SAM2 (Segment Anything Model 2) encoder. The SAM2 model is a next-generation image and video segmentation model introduced by Meta, designed to achieve real-time, cue-enabled segmentation of objects in any image or video. It is an extension of the original SAM model, achieving significant progress, particularly in video segmentation, capable of handling dynamic video content and generating high-quality segmentation results.

[0043] In this embodiment, the segmentation device inputs the UAV infrared photovoltaic image into the SAM2 model for encoding processing to obtain a feature encoding map.

[0044] S22: Input the feature encoding map into the convolutional layer of the multi-scale pyramid pooling module for convolution processing to obtain a convolutional feature map. Use the convolutional feature map as the input feature map of the first pooling layer for pooling processing to obtain the feature pooling map output by the first pooling layer. Use the feature pooling map output by the first pooling layer as the input feature of the next pooling layer, and repeat the pooling processing until the feature pooling map output by the last pooling layer is obtained. Use the feature pooling map as the feature extraction map.

[0045] Although the SAM2 model has the ability to segment with few samples, its single-scale feature extraction is difficult to adapt to the balance between global semantics and local details in UAV infrared photovoltaic imagery.

[0046] To overcome the above problems, in this embodiment, the segmentation device inputs the feature encoding map into the convolutional layer of the multi-scale pyramid pooling module (SPP) for convolution processing to obtain a convolutional feature map. The convolutional feature map is then used as the input feature map of the first pooling layer for pooling processing to obtain the feature pooling map output by the first pooling layer. The feature pooling map output by the first pooling layer is then used as the input feature map of the next pooling layer, and the pooling processing is repeated until the feature pooling map output by the last pooling layer is obtained. The feature pooling map is then used as the feature extraction map to obtain feature extraction maps at different levels.

[0047] Specifically, the feature extraction maps at several scales include feature extraction maps at the first to fourth scales. The first to fourth scales correspond to 1 / 4, 1 / 8, 1 / 16, and 1 / 32 of the resolution of the UAV infrared photovoltaic image, respectively. The low-resolution feature extraction map can capture the global semantic information of the image, which helps to identify the approximate location and category of the photovoltaic module in the image. The high-resolution feature extraction map retains rich details, such as the texture of the photovoltaic module surface, providing a basis for accurate segmentation.

[0048] Please see Figure 3 , Figure 3 The flowchart of S2 in the component segmentation method based on UAV infrared photovoltaic imagery provided in another embodiment of this application includes steps S23 to S24, as follows:

[0049] S23: Calculate the mean and standard deviation of the feature map pixels for the feature extraction map at the last scale to obtain the mean and standard deviation of the feature map pixels.

[0050] In this embodiment, the segmentation device calculates the mean and standard deviation of the feature map pixels for the last scale feature extraction map, i.e., the feature extraction map with a resolution of 1 / 32 of the UAV infrared photovoltaic image, to obtain the mean and standard deviation of the feature map pixels. The mean of the feature map pixels is:

[0051]

[0052] In the formula, μ is the average pixel value of the feature map, H is , W is , and F4(i,j) is the element at position (i,j) of the feature extraction map at the last scale, i.e., the feature extraction map at the fourth scale.

[0053] The standard deviation of the feature map pixels is:

[0054]

[0055] In the formula, σ is the standard deviation of the feature map pixels.

[0056] S24: Based on the mean pixel value and standard deviation of the feature map, perform illumination correction on the feature extraction map at the last scale to obtain the processed feature extraction map at the last scale.

[0057] In this embodiment, the segmentation device performs illumination correction on the feature extraction map at the last scale based on the mean pixel value, standard deviation of the feature map pixels, and a preset correction algorithm to obtain the processed feature extraction map at the last scale. The correction algorithm is as follows:

[0058]

[0059] In the formula, F 4corr F4 is the feature extraction map at the last scale after processing.

[0060] The Retinex theory is used to decompose the reflection and illumination components of UAV infrared photovoltaic images. Illumination normalization is performed on the low-resolution feature extraction map, i.e., the feature extraction map at the last scale, to suppress interference in overexposed / underexposed areas. The high-resolution feature extraction map retains the details of the reflection component and enhances noise robustness.

[0061] S3: Input the feature extraction maps at the several scales into the feature reconstruction module to perform sub-pixel structure feature reconstruction and obtain the feature reconstruction map.

[0062] In this embodiment, the segmentation device inputs the feature extraction maps of the several scales into the feature reconstruction module to perform sub-pixel structural feature reconstruction and obtain a feature reconstruction map.

[0063] Please see Figure 4 , Figure 4 The flowchart of S3 in the component segmentation method based on UAV infrared photovoltaic imagery provided in one embodiment of this application includes steps S31 to S32, as follows:

[0064] S31: Based on the feature extraction maps at several scales, a step-by-step bilinear interpolation method is used to upsample and fuse the feature extraction maps at the last scale upwards to obtain feature fusion maps at several scales.

[0065] In this embodiment, the segmentation device uses a stepped bilinear interpolation method to upsample and fuse the feature extraction maps at several scales, starting from the last scale and working upwards, to obtain a feature fusion map at several scales, as detailed below:

[0066] F 3fusion =UPsample(F 4corr )+F3)

[0067] F 2fusion =UPsample(F 3fusion )+F2)

[0068] F 1fusion =UPsample(F 2fusion )+F1)

[0069] In the formula, F fusion F 2fusion F 3fusion F1, F2, and F3 are the feature fusion maps of the first, second, and third scales, respectively, and F3 are the feature extraction maps of the first, second, and third scales, respectively. UPsample(·) is the upsampling function.

[0070] The low-resolution feature extraction map is upsampled using bilinear interpolation to make it consistent with the resolution of the high-resolution feature extraction map at the previous scale. Then, bilinear interpolation is used for fusion. Bilinear interpolation is a mature image scaling algorithm that calculates new pixel values ​​by linearly interpolating adjacent pixels. It can maintain the smoothness and continuity of the image well when enlarging the image, so that deep semantic information can be combined with shallow detail information to obtain a feature fusion map of the corresponding scale, thereby enhancing the accuracy of photovoltaic module identification in UAV infrared photovoltaic images.

[0071] S32: Perform subpixel convolution on the feature fusion maps at several scales to obtain subpixel convolution feature maps at several scales; perform weighted fusion based on the subpixel convolution feature maps at several scales and the preset adaptive weight parameters corresponding to the scales to obtain a feature reconstruction map.

[0072] In this embodiment, the segmentation device performs subpixel convolution on the feature fusion map at several scales to reconstruct subpixel-level structural features (e.g., structural features of photovoltaic modules where the size of key defects such as cell gaps and microcracks is smaller than the single-pixel resolution), thereby obtaining subpixel convolution feature maps at several scales and improving the accuracy of photovoltaic module identification.

[0073] The segmentation device performs weighted fusion based on the sub-pixel convolutional feature maps at several scales, preset adaptive weight parameters corresponding to each scale, and a preset weight fusion algorithm to optimize the contribution of features at each scale and obtain a feature reconstruction map. The adaptive weight parameters are normalized weights generated by 1×1 convolution and Softmax, reflecting the importance of features at each scale. The weight fusion algorithm is as follows:

[0074]

[0075] In the formula, F Refactoring For the feature reconstruction map, α i F represents the adaptive weight parameters corresponding to the i-th scale, where n is the number of scales. ifusion Let be the feature fusion map of the i-th scale.

[0076] S4: Input the reconstructed feature map into the feature enhancement module, and enhance the edge features according to the multimodal gradient edge attention mechanism to obtain the enhanced edge feature map.

[0077] In this embodiment, the segmentation device inputs the feature reconstruction map into the feature enhancement module, and performs edge feature enhancement according to the multimodal gradient edge attention mechanism to obtain an edge feature enhancement map.

[0078] Please see Figure 5 , Figure 5 The flowchart of S4 in the component segmentation method based on UAV infrared photovoltaic imagery provided in one embodiment of this application includes steps S41 to S42, as follows:

[0079] S41: Using the Sobel operator, multimodal gradient map calculation is performed on the UAV infrared photovoltaic image to be segmented to obtain intensity gradient map and texture gradient map; the intensity gradient map and texture gradient map are fused to obtain composite edge response map.

[0080] In this embodiment, the segmentation device uses the Sobel operator to perform multimodal gradient map calculation on the feature reconstruction map to obtain intensity gradient map and texture gradient map.

[0081] Specifically, for the intensity gradient map, the segmentation device obtains the intensity gradient map based on the UAV infrared photovoltaic image to be segmented, the Sobel operator, and a preset intensity gradient algorithm, wherein the intensity gradient algorithm is:

[0082]

[0083] In the formula, Here is the intensity gradient map, where I is the UAV infrared photovoltaic image to be segmented, and S... x For Sobel operators, * indicates a convolution operation.

[0084] For the texture gradient map, the segmentation device employs the Local Binary Pattern (LBP) method. This method generates a binary pattern by comparing the grayscale values ​​of the center pixel with its neighboring pixels. Based on the UAV infrared photovoltaic image to be segmented and a preset grayscale algorithm, a grayscale map is obtained. The grayscale algorithm is as follows:

[0085]

[0086] In the formula, T(x,y) is the element at position (x,y) in the grayscale image T, and δ(·) is the value function. That is, when the grayscale difference between the center pixel and its neighboring pixels is greater than 0, the value is 1; when the difference is less than 0, the value is 0.

[0087] I(x,y) is the element at position (x,y) in the UAV infrared photovoltaic image I to be segmented, where k represents the index number of the neighboring pixel. k ,y k ) represents the positions (x, y) surrounding position (x) in the UAV infrared photovoltaic image I to be segmented. k ,y k () elements.

[0088] Based on the grayscale image, the Sobel operator, and a preset texture gradient algorithm, a texture gradient map is obtained, wherein the texture gradient algorithm is:

[0089]

[0090] In the formula, This is a texture gradient map.

[0091] The segmentation device fuses the intensity gradient map and the texture gradient map according to the intensity gradient map and a preset gradient map fusion algorithm to obtain a composite edge response map. The gradient map fusion algorithm is as follows:

[0092]

[0093] In the formula, |ΔF fusion | is a composite edge response map, where α and β are the first and second preset learnable parameters, respectively, initialized to 0.5. Through backpropagation optimization, the contributions of the two gradients to sub-pixel edges are adaptively balanced.

[0094] S42: Calculate attention weights based on the composite edge response map to obtain attention weight parameters; multiply the feature reconstruction map element by element with the attention weight parameters to obtain an edge feature enhancement map.

[0095] In this embodiment, the segmentation device calculates attention weights based on the composite edge response map and a preset attention weight algorithm. It maps the composite edge response map to the attention weight space using a 1x1 convolution function and normalizes the weights to [0,1] using a Sigmoid function. High-weight regions (close to 1) correspond to the photovoltaic panel edges, while low-weight regions (close to 0) suppress background noise, thus obtaining attention weight parameters that highlight the weights of the component edge regions. The attention weight algorithm is as follows:

[0096] A edag =σ(Conv 1×1 (|ΔF fusion |))

[0097] In the formula, A edag Here, σ(·) represents the attention weight parameters, and Conv is the Sigmoid function. 1×1 (·) is a 1x1 convolution function.

[0098] The segmentation device multiplies the reconstructed feature map element-wise with attention weight parameters to enhance the sub-pixel structure and obtain an edge feature enhancement map, wherein the edge feature enhancement map is:

[0099]

[0100] In the formula, F att For edge feature enhancement map, The symbol for element-wise multiplication.

[0101] S5: Input the edge feature enhancement map into the component segmentation module to perform component segmentation and obtain a component segmentation map, which serves as the component segmentation result of the UAV infrared photovoltaic image to be segmented.

[0102] In this embodiment, the segmentation device inputs the edge feature enhancement map into the component segmentation module to perform component segmentation, and obtains a component segmentation map as the component segmentation result of the UAV infrared photovoltaic image to be segmented.

[0103] Specifically, the component segmentation module employs a SAM2 model decoder. The segmentation device inputs the edge feature enhancement map into the decoder. Based on the edge feature enhancement map and a preset mask cue, the mask cue is converted into a feature vector, which is then concatenated with the edge feature enhancement map. The concatenated feature map is then processed sequentially through a multilayer perceptron and a convolutional layer. The fused features are mapped to a probability space to generate a binary mask M. pred , as the component segmentation diagram.

[0104] By combining subpixel structural feature reconstruction and multimodal gradient edge attention mechanism, the background noise of UAV infrared photovoltaic images is effectively suppressed, the background false judgment rate is reduced, and the accuracy of identifying subpixel-level structures is improved. This improves the component edge segmentation accuracy of UAV infrared photovoltaic images and effectively overcomes the problem of difficulty in accurately segmenting UAV infrared photovoltaic images under complex lighting and occlusion scenarios, which easily leads to missed detections and false detections, thus meeting the needs of industrial inspection.

[0105] Please see Figure 6 , Figure 6 A flowchart illustrating a component segmentation method based on UAV infrared photovoltaic imagery, provided in another embodiment of this application, includes step S6, as detailed below:

[0106] S6: Based on the component segmentation map, the UAV infrared photovoltaic image to be segmented, and the post-processing optimization, obtain the post-processed component segmentation map, which serves as the component segmentation result of the UAV infrared photovoltaic image to be segmented.

[0107] In this embodiment, the segmentation device obtains an optimized component segmentation map based on the component segmentation map, the UAV infrared photovoltaic image to be segmented, and the optimized post-processing, which serves as the component segmentation result of the UAV infrared photovoltaic image to be segmented.

[0108] Please see Figure 7 , Figure 7 The flowchart of S6 in the component segmentation method based on UAV infrared photovoltaic imagery provided in another embodiment of this application includes steps S61 to S64, as follows:

[0109] S61: Obtain the noise pixel ratio and noise standard deviation of the component segmentation map; calculate the structural kernel size based on the noise pixel ratio and noise standard deviation to obtain the structural kernel size parameter; perform dilation-erosion operation on the component segmentation map based on the structural kernel size parameter to obtain the first intermediate component segmentation map.

[0110] Dilation operations can expand the boundaries of an object, connecting previously separated small regions; erosion operations can remove noise and tiny bumps. By performing a closing operation that combines dilation and erosion, tiny holes in the segmentation result can be filled, making the segmented region more complete.

[0111] In this embodiment, the segmentation device obtains the noise pixel ratio of the component segmentation map, calculates the structural kernel size based on the noise pixel ratio, and obtains structural kernel size parameters to ensure that small holes are filled while maintaining sub-pixel level gaps. The structural kernel size parameters are:

[0112] K size =[σ′·log(N)noise )]

[0113] In the formula, K size Here, σ′ represents the structural core size parameter, and N represents the noise standard deviation. noise This represents the percentage of noise pixels.

[0114] The segmentation device performs an expansion-erosion operation on the component segmentation map based on the structural core size parameters to obtain a first intermediate component segmentation map, wherein the intermediate component segmentation map is as follows:

[0115] M close =Erode(Dilate(M) pred ))

[0116] In the formula, M close M is a partitioning diagram for intermediate components. pred The component segmentation graph is defined by Erode(·), which is the erosion function, and Dilate(·), which is the dilation function.

[0117] S62: Obtain the one-dimensional array corresponding to the intensity gradient map and the length of the one-dimensional array, sort the one-dimensional array in ascending order, use linear interpolation, calculate the position index of the sorted one-dimensional array according to the length of the one-dimensional array, and obtain the position index.

[0118] In this embodiment, the segmentation device flattens the intensity gradient map to obtain a one-dimensional array corresponding to the intensity gradient map and the length of the one-dimensional array. The one-dimensional array is then sorted in ascending order. Using linear interpolation, a position index is calculated on the ascending one-dimensional array based on the length of the one-dimensional array to obtain the position index. The position index is:

[0119] z = (L-1) × 0.95

[0120] In the formula, z is the position index, and L is the length of the one-dimensional array.

[0121] S63: If the position index is an integer, obtain the value of the corresponding position index of the ascending one-dimensional array as the first edge detection threshold; if the position index is not an integer, take the integer part of the position index as the first sub-position index and the fractional part of the position index as the second sub-position index, calculate the edge detection threshold based on the first sub-position index, the second sub-position index and the ascending one-dimensional array, and obtain the first edge detection threshold; multiply the first edge detection threshold by a preset low threshold ratio to obtain the second edge detection threshold.

[0122] If the position index is an integer, in this embodiment, the segmentation device obtains the value of the corresponding position index of the ascending one-dimensional array as the first edge detection threshold.

[0123] If the location index is not an integer, the segmentation device takes the integer part of the location index as the first sub-location index and the fractional part of the location index as the second sub-location index. Based on the first sub-location index, the second sub-location index, the ascending one-dimensional array, and a preset threshold algorithm, an edge detection threshold is calculated to obtain a first edge detection threshold. The threshold algorithm is as follows:

[0124] G[K]=G[m]+f×(G[m+1]-G[m])

[0125] In the formula, G[K] is the first edge detection threshold, m is the first sub-position index, G[m] is the value of the first sub-position index of the one-dimensional array after ascending order, and f is the second sub-position index.

[0126] S64: Based on the first edge detection threshold and the second edge detection threshold, perform edge detection on the UAV infrared photovoltaic image to be segmented to obtain a second intermediate component segmentation map; overlap the first intermediate component segmentation map and the second intermediate component segmentation map to obtain the optimized component segmentation map.

[0127] The dual-threshold setting effectively avoids false positives and false negatives in edge detection. The low threshold is used to detect weaker edges, while the high threshold is used to detect stronger edges. By combining the two, the true boundaries of the photovoltaic module can be accurately extracted. In this embodiment, the segmentation device performs edge detection on the UAV infrared photovoltaic image to be segmented based on the first edge detection threshold and the second edge detection threshold to obtain a second intermediate module segmentation map.

[0128] The segmentation device overlaps the first intermediate component segmentation map and the second intermediate component segmentation map to obtain the optimized component segmentation map, which can further refine the boundaries and improve the quality of the segmentation results.

[0129] Specifically, the segmentation device uses a logical AND operation to overlap the first intermediate component segmentation map and the second intermediate component segmentation map to obtain the optimized component segmentation map, as described below:

[0130]

[0131] In the formula, M final To optimize the post-processed component partitioning graph, Canny(I input () is the segmentation diagram of the second intermediate component.

[0132] Please refer to Figure 8 , Figure 8 This is a schematic diagram of a component segmentation device based on UAV infrared photovoltaic imagery, provided in one embodiment of this application. This device can be implemented entirely or partially through software, hardware, or a combination of both. The device 8 includes:

[0133] The data acquisition module 81 is used to acquire the UAV infrared photovoltaic image to be segmented and the preset photovoltaic image segmentation model, wherein the photovoltaic image segmentation model includes a feature extraction module, a feature reconstruction module, a feature enhancement module, a component segmentation module and a post-processing module.

[0134] The multi-scale feature extraction module 82 is used to input the UAV infrared photovoltaic image to be segmented into the feature extraction module for multi-scale feature extraction to obtain feature extraction maps at several scales.

[0135] The sub-pixel structure feature reconstruction module 83 is used to input the feature extraction maps of the several scales into the feature reconstruction module to perform sub-pixel structure feature reconstruction and obtain a feature reconstruction map;

[0136] The edge feature enhancement module 84 is used to input the feature reconstruction map into the feature enhancement module, and perform edge feature enhancement according to the multimodal gradient edge attention mechanism to obtain an edge feature enhancement map;

[0137] The image component segmentation module 85 is used to input the edge feature enhancement map into the component segmentation module for component segmentation to obtain a component segmentation map, which serves as the component segmentation result of the UAV infrared photovoltaic image to be segmented.

[0138] In this embodiment, a data acquisition module obtains the UAV infrared photovoltaic image to be segmented and a preset photovoltaic image segmentation model. The photovoltaic image segmentation model includes a feature extraction module, a feature reconstruction module, a feature enhancement module, a component segmentation module, and a post-processing module. The multi-scale feature extraction module inputs the UAV infrared photovoltaic image to be segmented into the feature extraction module for multi-scale feature extraction, obtaining feature extraction maps at several scales. The sub-pixel structure feature reconstruction module inputs the feature extraction maps at several scales into the feature reconstruction module for sub-pixel structure feature reconstruction, obtaining a feature reconstruction map. The edge feature enhancement module inputs the feature reconstruction map into the feature enhancement module, performing edge feature enhancement based on a multimodal gradient edge attention mechanism, obtaining an edge feature enhancement map. The image component segmentation module inputs the edge feature enhancement map into the component segmentation module for component segmentation, obtaining a component segmentation map, which serves as the component segmentation result of the UAV infrared photovoltaic image to be segmented. By combining subpixel structural feature reconstruction and multimodal gradient edge attention mechanism, the background noise of UAV infrared photovoltaic images is effectively suppressed, the background false judgment rate is reduced, and the accuracy of identifying subpixel-level structures is improved. This improves the component edge segmentation accuracy of UAV infrared photovoltaic images and effectively overcomes the problem of difficulty in accurately segmenting UAV infrared photovoltaic images under complex lighting and occlusion scenarios, which easily leads to missed detections and false detections, thus meeting the needs of industrial inspection.

[0139] Please refer to Figure 9 , Figure 9 This is a schematic diagram of the structure of a computer device provided in one embodiment of this application. The computer device 9 includes: a processor 91, a memory 92, and a computer program 93 stored in the memory 92 and executable on the processor 91; the computer device can store multiple instructions, which are adapted to be loaded and executed by the processor 91. Figures 1 to 7 The method steps shown can be found in the following document for detailed execution process. Figures 1 to 7 The specific details shown will not be repeated here.

[0140] The processor 91 may include one or more processing cores. The processor 91 connects to various parts within the server using various interfaces and lines. It executes various functions and processes data of the component segmentation device 8 based on UAV infrared photovoltaic imagery by running or executing instructions, programs, code sets, or instruction sets stored in memory 92, and by calling data from memory 92. Optionally, the processor 91 may be implemented using at least one hardware form of Digital Signal Processing (DSP), Field-Programmable Gate Array (FPGA), or Programmable Logic Array (PLA). The processor 91 may integrate one or more of the following: a Central Processing Unit (CPU), a Graphics Processing Unit (GPU), and a modem. The CPU primarily handles the operating system, user interface, and applications; the GPU is responsible for rendering and drawing the content required to be displayed on the touch screen; and the modem handles wireless communication. It is understood that the modem may also not be integrated into the processor 91 and may be implemented as a separate chip.

[0141] The memory 92 may include random access memory (RAM) or read-only memory. Optionally, the memory 92 may include a non-transitory computer-readable storage medium. The memory 92 can be used to store instructions, programs, code, code sets, or instruction sets. The memory 92 may include a program storage area and a data storage area, wherein the program storage area may store instructions for implementing an operating system, instructions for at least one function (such as touch instructions), instructions for implementing the various method embodiments described above, etc.; the data storage area may store data involved in the various method embodiments described above, etc. Optionally, the memory 92 may also be at least one storage device located remotely from the aforementioned processor 91.

[0142] This application embodiment also provides a storage medium that can store multiple instructions, which are adapted to be loaded and executed by a processor as described above. Figures 1 to 7 The method steps shown can be found in the following document for detailed execution process. Figures 1 to 7 The specific details shown will not be repeated here.

[0143] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is merely an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiments can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit. Furthermore, the specific names of the functional units and modules are only for easy differentiation and are not intended to limit the scope of protection of this application. The specific working process of the units and modules in the above system can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0144] In the above embodiments, the descriptions of each embodiment have different focuses. For parts that are not described in detail or recorded in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0145] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the algorithm. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this invention.

[0146] In the embodiments provided by this invention, it should be understood that the disclosed apparatus / terminal devices and methods can be implemented in other ways. For example, the apparatus / terminal device embodiments described above are merely illustrative. For instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.

[0147] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0148] Furthermore, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0149] If the integrated module / unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments of the present invention can also be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms.

[0150] This invention is not limited to the above-described embodiments. If any modifications or variations to this invention do not depart from the spirit and scope of this invention, and if such modifications and variations fall within the scope of the claims and equivalent technologies of this invention, then this invention also intends to include such modifications and variations.

Claims

1. A component segmentation method based on UAV infrared photovoltaic imagery, characterized in that, Includes the following steps: The process involves obtaining an infrared photovoltaic image of a UAV to be segmented and a preset photovoltaic image segmentation model. The photovoltaic image segmentation model includes a feature extraction module, a feature reconstruction module, a feature enhancement module, and a component segmentation module. The feature extraction module includes an encoding module and a multi-scale pyramid pooling module. The multi-scale pyramid pooling module includes a convolutional layer and several pooling layers connected in sequence. The UAV infrared photovoltaic image to be segmented is input into the encoding module for encoding processing to obtain a feature encoding map; The feature encoding map is input into the convolutional layer of the multi-scale pyramid pooling module for convolution processing to obtain a convolutional feature map. The convolutional feature map is then used as the input feature map of the first pooling layer for pooling processing to obtain the feature pooling map output by the first pooling layer. The feature pooling map output by the first pooling layer is then used as the input feature map of the next pooling layer, and the pooling processing is repeated until the feature pooling map output by the last pooling layer is obtained. The feature pooling map is then used as the feature extraction map to obtain feature extraction maps at several scales. The feature extraction maps at several scales are input into the feature reconstruction module. Based on the feature extraction maps at several scales, a step-wise bilinear interpolation method is used to upsample and fuse layer by layer from the feature extraction map at the last scale upwards to obtain a feature fusion map at several scales. Subpixel convolution is performed on the feature fusion maps at several scales to obtain subpixel convolution feature maps at several scales; weighted fusion is performed based on the subpixel convolution feature maps at several scales and the preset adaptive weight parameters corresponding to the scales to obtain a feature reconstruction map. The UAV infrared photovoltaic image to be segmented and the feature reconstruction map are input into the feature enhancement module. The Sobel operator is used to calculate the multimodal gradient map of the UAV infrared photovoltaic image to be segmented to obtain the intensity gradient map and the texture gradient map. The intensity gradient map and the texture gradient map are fused to obtain the composite edge response map. Attention weights are calculated based on the composite edge response map to obtain attention weight parameters; the feature reconstruction map is then multiplied element-wise with the attention weight parameters to obtain an edge feature enhancement map. The edge feature enhancement map is input into the component segmentation module for component segmentation to obtain a component segmentation map, which serves as the component segmentation result of the UAV infrared photovoltaic image to be segmented.

2. The component segmentation method based on UAV infrared photovoltaic imagery according to claim 1, characterized in that, The step of inputting the UAV infrared photovoltaic image to be segmented into the feature extraction module for multi-scale feature extraction to obtain feature extraction maps at several scales also includes the following steps: The mean pixel value and standard deviation of the feature map are calculated for the feature map extracted at the last scale to obtain the mean pixel value and standard deviation of the feature map. Based on the mean pixel value and standard deviation of the feature map, illumination correction is performed on the feature extraction map at the last scale to obtain the processed feature extraction map at the last scale.

3. The component segmentation method based on UAV infrared photovoltaic imagery according to claim 1, characterized in that, It also includes the following steps: Based on the component segmentation map, the UAV infrared photovoltaic image to be segmented, and the post-processing optimization, an optimized component segmentation map is obtained, which serves as the component segmentation result of the UAV infrared photovoltaic image to be segmented.

4. The component segmentation method based on UAV infrared photovoltaic imagery according to claim 3, characterized in that, The step of obtaining the optimized component segmentation map based on the component segmentation map, the UAV infrared photovoltaic image to be segmented, and the optimized post-processing includes the following steps: Obtain the noise pixel ratio and noise standard deviation of the component segmentation map, and calculate the structure kernel size based on the noise pixel ratio and noise standard deviation to obtain the structure kernel size parameters. Based on the structural core size parameters, an expansion-erosion operation is performed on the component segmentation map to obtain a first intermediate component segmentation map; Obtain the one-dimensional array corresponding to the intensity gradient map and the length of the one-dimensional array. Sort the one-dimensional array in ascending order. Use linear interpolation to calculate the position index of the one-dimensional array after sorting in ascending order based on the length of the one-dimensional array. If the position index is an integer, the value of the corresponding position index of the ascending one-dimensional array is obtained as the first edge detection threshold; if the position index is not an integer, the integer part of the position index is taken as the first sub-position index and the fractional part of the position index is taken as the second sub-position index. The edge detection threshold is calculated based on the first sub-position index, the second sub-position index and the ascending one-dimensional array to obtain the first edge detection threshold; the first edge detection threshold is multiplied by a preset low threshold ratio to obtain the second edge detection threshold. Based on the first edge detection threshold and the second edge detection threshold, edge detection is performed on the UAV infrared photovoltaic image to be segmented to obtain a second intermediate component segmentation map; the first intermediate component segmentation map and the second intermediate component segmentation map are overlapped to obtain the optimized component segmentation map.

5. A component segmentation device based on UAV infrared photovoltaic imagery, characterized in that, include: The data acquisition module is used to obtain the UAV infrared photovoltaic image to be segmented and the preset photovoltaic image segmentation model. The photovoltaic image segmentation model includes a feature extraction module, a feature reconstruction module, a feature enhancement module, a component segmentation module, and a post-processing module. The feature extraction module includes an encoding module and a multi-scale pyramid pooling module. The multi-scale pyramid pooling module includes a convolutional layer and several pooling layers connected in sequence. A multi-scale feature extraction module is used to input the UAV infrared photovoltaic image to be segmented into the encoding module for encoding processing to obtain a feature encoding map; The feature encoding map is input into the convolutional layer of the multi-scale pyramid pooling module for convolution processing to obtain a convolutional feature map. The convolutional feature map is then used as the input feature map of the first pooling layer for pooling processing to obtain the feature pooling map output by the first pooling layer. The feature pooling map output by the first pooling layer is then used as the input feature map of the next pooling layer, and the pooling processing is repeated until the feature pooling map output by the last pooling layer is obtained. The feature pooling map is then used as the feature extraction map to obtain feature extraction maps at several scales. The sub-pixel structure feature reconstruction module is used to input the feature extraction maps of the several scales into the feature reconstruction module, and based on the feature extraction maps of the several scales, to perform upsampling and layer-by-layer fusion from the feature extraction map of the last scale upwards using a step-like bilinear interpolation method to obtain a feature fusion map of the several scales. Subpixel convolution is performed on the feature fusion maps at several scales to obtain subpixel convolution feature maps at several scales; weighted fusion is performed based on the subpixel convolution feature maps at several scales and the preset adaptive weight parameters corresponding to the scales to obtain a feature reconstruction map. The edge feature enhancement module is used to input the feature reconstruction map into the feature enhancement module, use the Sobel operator to perform multimodal gradient map calculation on the UAV infrared photovoltaic image to be segmented, obtain intensity gradient map and texture gradient map, and fuse the intensity gradient map and texture gradient map to obtain composite edge response map; Attention weights are calculated based on the composite edge response map to obtain attention weight parameters; the feature reconstruction map is then multiplied element-wise with the attention weight parameters to obtain an edge feature enhancement map. The image component segmentation module is used to input the edge feature enhancement map into the component segmentation module for component segmentation to obtain a component segmentation map, which serves as the component segmentation result of the UAV infrared photovoltaic image to be segmented.

6. A computer device, characterized in that, include: A processor, a memory, and a computer program stored in the memory and executable on the processor; the computer program, when executed by the processor, implements the steps of the component segmentation method based on UAV infrared photovoltaic imagery as described in any one of claims 1 to 4.

7. A storage medium, characterized in that: The storage medium stores a computer program, which, when executed by a processor, implements the steps of the component segmentation method based on UAV infrared photovoltaic imagery as described in any one of claims 1 to 4.

Images