An adaptive multi-scale photovoltaic panel defect detection method fusing gray level co-occurrence matrix and interactive attention

Abstract

The application discloses a kind of fusion gray level co-occurrence matrix and interactive attention adaptive multi-scale photovoltaic panel defect detection method, it is related to defect detection technical field, including: obtaining photovoltaic panel defect detection dataset and pre-processing, the standard photovoltaic panel defect detection dataset is divided into training set and test set;By the gray level co-occurrence matrix corresponding to the gray level co-occurrence matrix of training set and test set is respectively constructed by the gray level co-occurrence matrix of training set and test set in statistics joint probability distribution;Training set gray level co-occurrence matrix and test set gray level co-occurrence matrix are used respectively to extract the texture feature of training set and test set, training set and training set coarse positioning fusion texture feature map are input into photovoltaic panel defect detection model and are trained, test set and test set coarse positioning fusion texture feature map are input into the detection model trained, feature enhancement and channel adaptive fusion are carried out by interactive attention mechanism, and the defect class and defect boundary positioning of photovoltaic panel are output.The application improves the accuracy of defect detection.

Description

Technical Field

[0001] This invention relates to the field of photovoltaic panel defect detection technology, specifically to an adaptive multi-scale photovoltaic panel defect detection method that integrates gray-level co-occurrence matrix and interactive attention. Background Technology

[0002] With the continuous rise in global energy demand and increasingly severe environmental problems, photovoltaic (PV) power generation, as a clean and renewable energy solution, has received widespread attention. As the core component of a PV power generation system, the surface quality of the PV panel directly determines its photoelectric conversion efficiency and reliability. During production, transportation, and operation and maintenance, various defects inevitably occur on the surface of PV panels, such as microcracks, broken grids, scratches, and dirt. If these defects are not detected in time, they will seriously affect the photoelectric conversion efficiency of the PV panel and may even cause hot spot effects, posing safety hazards. Therefore, developing efficient and accurate methods for detecting defects in PV panels has become a key link in reducing costs, increasing efficiency, and ensuring safety in the PV industry.

[0003] In recent years, with the rapid development of deep learning, several deep learning-based solutions have been proposed in existing technologies. For example, by introducing a multi-head attention mechanism, cross-channel feature extraction capabilities are enhanced, effectively compensating for the limitations of models in channel-dimensional feature extraction. Simultaneously, integrating a bidirectional feature pyramid network strengthens the feature correlation between channels within the connection module, improving the overall integrity of feature representation and reducing information loss during feature transmission, ultimately achieving a significant improvement in detection accuracy. However, for the same type of photovoltaic panel defects, the size proportion of defects in the image can fluctuate significantly due to factors such as changes in shooting angle and shooting height. Some defects also possess inherent characteristics of being small in size. Facing such scale heterogeneity issues, it is difficult to achieve comprehensive and accurate identification of all defects, easily leading to missed detection of small-sized defects or misjudgment of large-scale defect features.

[0004] Another approach proposes a lightweight, high-precision photovoltaic panel defect detection model, employing a nested, cross-stage partial connection structure to enable gradient flow to capture richer information differences. Simultaneously, a cross-stage partial spatial pyramid pooling structure is designed to enhance the network's ability to extract multi-scale defect features. However, during the data acquisition phase, objective factors such as fluctuating lighting conditions and differences in imaging equipment parameters can easily introduce noise interference similar to real defect features into the acquired images. This noise can confuse the model's learning process of defect features, making it difficult for it to focus on the true defect target region. Summary of the Invention

[0005] In order to solve the above-mentioned technical problems, this application proposes the following technical solution: This application provides an adaptive multi-scale photovoltaic panel defect detection method that integrates gray-level co-occurrence matrix and interactive attention, including: A photovoltaic panel defect detection dataset is obtained. After standardization preprocessing, the standard photovoltaic panel defect detection dataset is divided into a training set and a test set. By statistically analyzing the joint gray-level probability distribution of pixel pairs with preset spatial distance and orientation relationships in each image of the training set and the test set, a gray-level co-occurrence matrix for the training set and a gray-level co-occurrence matrix for the test set are constructed. The texture features of photovoltaic panel images in the training set and the test set are extracted using the gray-level co-occurrence matrix of the training set and the gray-level co-occurrence matrix of the test set, respectively, to obtain coarse localization and fusion texture feature maps of the training set and the test set. The training set and the coarse localization fusion texture feature map of the training set are input into the pre-built photovoltaic panel defect detection model for training to obtain the optimal photovoltaic panel defect detection model; The test set and the coarse localization fused texture feature map of the test set are input into the optimal photovoltaic panel defect detection model. Feature enhancement and channel adaptive fusion are performed through interactive attention mechanism to output the defect category and defect boundary localization of the photovoltaic panel.

[0006] In one possible implementation, the gray-level joint probability distribution of pixel pairs with a preset spatial distance and orientation relationship in each image of the training set and the test set is statistically analyzed to construct the gray-level co-occurrence matrix of the training set and the gray-level co-occurrence matrix of the test set. The normalization calculation formulas for the matrix elements are as follows: in, and The center of the sliding window for the training set images Gray-level combination within the gray-level co-occurrence matrix Normalized probability values ​​and the center of the sliding window of the test set image Gray-level combination within the gray-level co-occurrence matrix The normalized probability value, and These represent the number of gray levels in the training set images and the number of gray levels in the test set images, respectively. The baseline pixel grayscale value within the image is The grayscale value of the paired offset pixels is The total frequency of occurrence of pixel pairs.

[0007] In one possible implementation, the texture features of photovoltaic panel images in the training set and the test set are extracted using the gray-level co-occurrence matrix of the training set and the gray-level co-occurrence matrix of the test set, respectively, to obtain a coarse localization and fusion texture feature map of the training set and a coarse localization and fusion texture feature map of the test set, including: Using the gray-level co-occurrence matrix of the training set, the contrast texture feature map and dissimilarity texture feature map of the photovoltaic panel images in the training set are calculated, and the calculation formulas are as follows: in, and These represent the contrast and texture feature maps of photovoltaic panel images in the training set. Pixel values ​​and dissimilarity texture feature maps at the location Pixel value at; The contrast texture feature map and dissimilarity texture feature map of the photovoltaic panel images in the training set are concatenated by channel to obtain the coarse localization fusion texture feature map of the training set. Using the gray-level co-occurrence matrix of the test set, the contrast texture feature map and dissimilarity texture feature map of the photovoltaic panel images in the test set are calculated, and the calculation formulas are as follows: in, and These represent the contrast texture feature maps of photovoltaic panel images in the test set. Pixel values ​​and dissimilarity texture feature maps at the location Pixel value at; The contrast texture feature map and dissimilarity texture feature map of the photovoltaic panel images in the test set are spliced ​​by channel to obtain the coarse localization fusion texture feature map of the test set.

[0008] In one possible implementation, the photovoltaic panel defect detection model includes a backbone network, an adaptive multi-scale feature pyramid network, and a detection output terminal. The backbone network includes a fine localization module. The output terminal of the fine localization module is connected to the input terminal of a first CBS module. The output terminal of the first CBS module is connected to the input terminal of a first C2F module. The output terminal of the first C2F module is connected to the input terminal of a second CBS module. The output terminal of the second CBS module is connected to the input terminal of a second C2F module. The output terminal of the second C2F module is connected to the input terminal of a third CBS module. The output terminal of the third CBS module is connected to the input terminal of a third C2F module. The output terminal of the third C2F module is connected to the input terminal of an SPPF module. The output terminal of the SPPF module is connected to the adaptive multi-scale feature pyramid network. The adaptive multi-scale feature pyramid network includes a first upsampling layer. The input of the first upsampling layer is connected to the output of the SPPF module. The outputs of the first upsampling layer and the second C2F module are connected to the input of a first scale-aware feature enhancement module. The output of the first scale-aware feature enhancement module is connected to the input of a fourth C2F module. The output of the fourth C2F module is connected to the input of a second upsampling layer. The outputs of the second upsampling layer and the first C2F module are connected to the input of a second scale-aware feature enhancement module. The output of the second scale-aware feature enhancement module is connected to the input of a fifth C2F module. The output of the fifth C2F module is connected to the first detection head of both the input of the fourth CBS module and the detection output. The output of the fourth CBS module is connected to the input of the first downsampling layer. The outputs of the first downsampling layer and the fourth C2F module are both connected to the input of the third scale-aware feature enhancement module. The output of the third scale-aware feature enhancement module is connected to the input of the sixth C2F module. The output of the sixth C2F module is connected to the input of the fifth CBS module and the second detection head of the detection output, respectively. The output of the fifth CBS module is connected to the input of the second downsampling layer. The outputs of the second downsampling layer and the SPPF module are both connected to the input of the fourth scale-aware feature enhancement module. The output of the fourth scale-aware feature enhancement module is connected to the input of the seventh C2F module. The output of the seventh C2F module is connected to the third detection head of the detection output.

[0009] In one possible implementation, the fine localization module includes a feature extraction module, which includes a first branch and a second branch. The first branch and the second branch have the same structure, each containing a first convolutional layer, a second convolutional layer, and a C2F module connected in sequence. The outputs of the first branch and the second branch are respectively connected to the inputs of the first defect texture interaction attention module and the second defect texture interaction attention module. The outputs of the first defect texture interaction attention module and the second defect texture interaction attention module are connected to the texture feature fusion module.

[0010] In one possible implementation, the loss function used during training of the photovoltaic panel defect detection model is: in, For the total loss, For bounding box regression loss, For classifying losses, For the distribution focus loss, To balance hyperparameters, This represents the actual coordinate offset value. and For adjacent discrete coordinate scales, This represents the probability that the photovoltaic panel defect detection model outputs a value that falls on a discrete coordinate system. These are the width and height of the predicted bounding box, respectively. These are the width and height of the actual bounding box, respectively. The aspect ratio loss factor To balance the proportional parameters, For intersection, union, and comparison, For the prediction box, For the true frame, This is the area of ​​the region where the predicted bounding box intersects with the ground truth bounding box. The area of ​​the entire region where the predicted bounding box and the ground truth bounding box are merged. The diagonal length of the smallest bounding rectangle that can simultaneously cover both the predicted and ground truth bounding boxes. This indicates the calculation of the Euclidean distance between two center points. These represent the geometric center points of the predicted bounding box and the ground truth bounding box, respectively. The true label representing the defect category, This indicates the predicted defect category.

[0011] In one possible implementation, the test set and the coarse-localized fused texture feature map of the test set are input into the optimal photovoltaic panel defect detection model. Feature enhancement and channel adaptive fusion are performed through an interactive attention mechanism to output the defect category and defect boundary localization of the photovoltaic panel, including: The test set and the coarse localization fused texture feature map of the test set are input into the fine localization module of the optimal photovoltaic panel defect detection model, and the fine localization feature map of the test set is output through feature extraction and defect texture interaction. The fine localization feature map is input into an adaptive multi-scale feature pyramid network, and after cross-scale bidirectional fusion and scale-aware weighted enhancement, a multi-scale enhanced defect feature map is output. The multi-scale enhanced defect feature maps are input into the corresponding detection heads at the detection output end, and the defect categories and defect boundary locations of the photovoltaic panels are output.

[0012] In one possible implementation, the test set and the coarse localization fused texture feature map of the test set are input into the fine localization module of the optimal photovoltaic panel defect detection model. A fine localization feature map of the test set is output through feature extraction and defect texture interaction, including: The test set is input into the first branch of the fine localization module, and the original semantic feature map is output through continuous convolutional downsampling and C2F module processing. Simultaneously, the coarse localization fused texture feature map of the test set is input into the second branch of the fine localization module, and after being processed by the same convolution downsampling and C2F module, the fused texture feature map is output. The original semantic feature map is subjected to convolutional linear projection to obtain the corresponding first query matrix and first key-value matrix. The fused texture feature map is subjected to convolutional linear projection to obtain the first value matrix. The calculation formula is as follows: in, and Let represent the first query matrix and the first key-value matrix obtained by linearly projecting the original semantic feature map through a convolution with a kernel size of 1, respectively. This represents the first value matrix obtained by linearly projecting the fused texture feature map through a convolution with a kernel size of 1. This indicates a convolutional layer with a kernel size of 1. These represent the original semantic feature map and the fused texture feature map, respectively. The attention weights of the original semantic feature extraction branch are calculated based on the first query matrix, the first key matrix, and the first value matrix. The attention weights of the original semantic feature extraction branch are then residually connected to the original semantic feature map to output an enhanced original semantic feature map. Similarly, the fused texture feature map is subjected to convolutional linear projection to obtain the corresponding second query matrix and second key-value matrix, and the original semantic feature map is subjected to convolutional linear projection to obtain the second value matrix. The calculation formula is as follows: in, and Let represent the second query matrix and the second key-value matrix obtained by linearly projecting the fused texture feature map through a convolution with a kernel size of 1, respectively. This represents the second-value matrix obtained by linearly projecting the original semantic feature map through a convolution with a kernel size of 1; The attention weights of the fused texture feature extraction branch are calculated based on the second query matrix, the second key matrix, and the second value matrix. The attention-weighted features are then residually connected to the fused texture feature map to output an enhanced fused texture feature map. The enhanced original semantic feature map and the enhanced fused texture feature map are input into the texture feature fusion module for channel-weighted fusion, and the test set fine localization feature map is output.

[0013] In one possible implementation, the attention weights of the original semantic feature extraction branch are calculated based on the first query matrix, the first key-value matrix, and the first value matrix. The attention weights of the original semantic feature extraction branch are then residually connected to the original semantic feature map, and the calculation formulas for the enhanced original semantic feature map are output as follows: in, To enhance the original semantic feature map, To extract attention weights for branches of the original semantic features, For the number of channels, for transpose, This is the activation function.

[0014] In one possible implementation, the attention weights of the fused texture feature extraction branch are calculated based on the second query matrix, the second key-value matrix, and the second value matrix. The attention weights of the fused texture feature extraction branch are then residually concatenated with the fused texture feature map. The calculation formulas for the enhanced fused texture feature map are as follows: in, To enhance the fusion of texture feature maps, Attention weights are extracted for branches to fuse texture features. for The transpose of .

[0015] Compared with the prior art, the beneficial effects of this application are as follows: This application proposes a novel photovoltaic panel defect detection method based on multi-scale feature fusion and spatial attention guidance, which can overcome the problems of insufficient weak defect feature representation ability and insufficient utilization of multi-scale information, and achieve comprehensive and accurate identification of defects.

[0016] This application's coarse and fine localization modules for the target region enable the extraction of defect features from coarse to fine, reducing redundant features while improving the ability to perceive defect features. Simultaneously, the adaptive multi-scale feature pyramid network is improved, and a cross-scale fusion path integrated scale-aware feature enhancement module is constructed based on bidirectional efficient information flow to achieve efficient fusion of features at different scales while enhancing feature information beneficial to detection. Attached Figure Description

[0017] Figure 1A flowchart illustrating an adaptive multi-scale photovoltaic panel defect detection method that integrates gray-level co-occurrence matrix and interactive attention, provided for an embodiment of this application; Figure 2 The contrast texture feature map and dissimilarity texture feature map provided in the embodiments of this application are as follows: (a) and (d) are photovoltaic panel defect images in the training set and test set, respectively; (b) and (c) are the contrast texture feature map and dissimilarity texture feature map corresponding to (a), respectively; and (e) and (f) are the contrast texture feature map and dissimilarity texture feature map corresponding to (d), respectively. Figure 3 This is a schematic diagram of the overall structure of the photovoltaic panel defect detection model provided in the embodiments of this application; Figure 4 This is a schematic diagram of the fine positioning module provided in an embodiment of this application; Figure 5 A schematic diagram of the structure of the first defect texture interactive attention module provided in the embodiments of this application; Figure 6 A schematic diagram of the structure of the second defect texture interactive attention module provided in the embodiments of this application; Figure 7 This is a schematic diagram of the structure of the texture feature fusion module provided in the embodiments of this application; Figure 8 The scale-aware feature enhancement module provided in the embodiments of this application. Detailed Implementation

[0018] The present solution will now be described in conjunction with the accompanying drawings and specific embodiments.

[0019] Figure 1 A schematic diagram illustrating an adaptive multi-scale photovoltaic panel defect detection method integrating gray-level co-occurrence matrix and interactive attention, provided in an embodiment of this application, is shown below. Figure 1 An adaptive multi-scale photovoltaic panel defect detection method integrating gray-level co-occurrence matrix and interactive attention in this embodiment includes: S101, Obtain the photovoltaic panel defect detection dataset. After standardizing and preprocessing the photovoltaic panel defect detection dataset, divide the standard photovoltaic panel defect detection dataset into a training set and a test set.

[0020] In this embodiment, a dataset containing common photovoltaic panel defect detection data such as broken grids, scratches, and hot spots is obtained. The dataset undergoes preprocessing to remove missing and blurred data, and is then divided into training and testing sets proportionally. Subsequently, the training set is augmented using rotation, scaling, translation, and cropping to expand the dataset. Each image in the augmented dataset is then set to 640×640 pixels and normalized.

[0021] S102, by statistically analyzing the gray-level joint probability distribution of pixel pairs with preset spatial distance and orientation relationships in each image of the training set and the test set respectively, a gray-level co-occurrence matrix for the training set and a gray-level co-occurrence matrix for the test set are constructed.

[0022] In this embodiment, a gray-level co-occurrence matrix is ​​constructed by statistically analyzing the joint probability distribution of gray-level pixel pairs with preset spatial distance and orientation relationships in each image of the training set. The normalization calculation formula for the matrix elements is as follows: in, Center the sliding window for the training set images Gray-level combination within the gray-level co-occurrence matrix The normalized probability value, The gray levels of the training set images. The baseline pixel grayscale value within the image is The grayscale value of the paired offset pixels is The total frequency of occurrence of pixel pairs.

[0023] By statistically analyzing the joint probability distribution of gray levels of pixel pairs with preset spatial distance and directional relationships in the test set, a gray-level co-occurrence matrix for the test set is constructed. The normalization formula for the matrix elements is as follows: in, Center the sliding window for the test set images Gray-level combination within the gray-level co-occurrence matrix The normalized probability value, The grayscale level of the test set images.

[0024] S103, using the gray-level co-occurrence matrix of the training set and the gray-level co-occurrence matrix of the test set, respectively, the texture features of the photovoltaic panel images in the training set and the test set are extracted to obtain the coarse localization fusion texture feature map of the training set and the coarse localization fusion texture feature map of the test set.

[0025] See Figure 2 (a) and (d) are photovoltaic panel defect images from the training and test sets, respectively; (b) and (c) are the contrast texture feature map and dissimilarity texture feature map corresponding to (a), respectively; (e) and (f) are the contrast texture feature map and dissimilarity texture feature map corresponding to (d), respectively. In this embodiment, the contrast texture feature map and dissimilarity texture feature map of the photovoltaic panel images in the training set are calculated using the gray-level co-occurrence matrix of the training set. The calculation formulas are as follows: in, and These represent the contrast and texture feature maps of photovoltaic panel images in the training set. Pixel values ​​and dissimilarity texture feature maps at the location The pixel value at that location.

[0026] The contrast texture feature map and dissimilarity texture feature map of the photovoltaic panel images in the training set are concatenated by channel to obtain the coarse localization fusion texture feature map of the training set.

[0027] Using the gray-level co-occurrence matrix of the test set, the contrast texture feature map and dissimilarity texture feature map of the photovoltaic panel images in the test set are calculated. The calculation formulas are as follows: in, and These represent the contrast texture feature maps of photovoltaic panel images in the test set. Pixel values ​​and dissimilarity texture feature maps at the location The pixel value at that location.

[0028] The contrast texture feature map and dissimilarity texture feature map of the photovoltaic panel images in the test set are stitched together by channel to obtain the coarse localization fusion texture feature map of the test set.

[0029] S104. Input the training set and the coarse localization fusion texture feature map of the training set into the pre-built photovoltaic panel defect detection model for training, and obtain the optimal photovoltaic panel defect detection model.

[0030] See Figure 3 In this embodiment, the photovoltaic panel defect detection model includes a backbone network, an adaptive multi-scale feature pyramid network, and a detection output terminal. The backbone network includes a fine localization module. The output terminal of the fine localization module is connected to the input terminal of the first CBS module. The output terminal of the first CBS module is connected to the input terminal of the first C2F module. The output terminal of the first C2F module is connected to the input terminal of the second CBS module. The output terminal of the second CBS module is connected to the input terminal of the second C2F module. The output terminal of the second C2F module is connected to the input terminal of the third CBS module. The output terminal of the third CBS module is connected to the input terminal of the third C2F module. The output terminal of the third C2F module is connected to the input terminal of the SPPF module. The output terminal of the SPPF module is connected to the adaptive multi-scale feature pyramid network.

[0031] The adaptive multi-scale feature pyramid network includes a first upsampling layer. The input of the first upsampling layer is connected to the output of the SPPF module. The output of the first upsampling layer and the output of the second C2F module are connected to the input of the first scale-aware feature enhancement module. The output of the first scale-aware feature enhancement module is connected to the input of the fourth C2F module. The output of the fourth C2F module is connected to the input of the second upsampling layer. The outputs of the second upsampling layer and the first C2F module are connected to the input of the second scale-aware feature enhancement module. The output of the second scale-aware feature enhancement module is connected to the input of the fifth C2F module. The output of the fifth C2F module is connected to the first detection head of the fourth CBS module and the detection output, respectively. The output of the BS module is connected to the input of the first downsampling layer. The outputs of the first downsampling layer and the fourth C2F module are both connected to the input of the third scale-aware feature enhancement module. The output of the third scale-aware feature enhancement module is connected to the input of the sixth C2F module. The output of the sixth C2F module is connected to the input of the fifth CBS module and the second detection head of the detection output, respectively. The output of the fifth CBS module is connected to the input of the second downsampling layer. The outputs of the second downsampling layer and the SPPF module are both connected to the input of the fourth scale-aware feature enhancement module. The output of the fourth scale-aware feature enhancement module is connected to the input of the seventh C2F module. The output of the seventh C2F module is connected to the third detection head of the detection output.

[0032] The loss function used during training of the photovoltaic panel defect detection model is: in, For the total loss, For bounding box regression loss, For classifying losses, For the distribution focus loss, To balance hyperparameters, This represents the actual coordinate offset value. and For adjacent discrete coordinate scales, This represents the probability that the photovoltaic panel defect detection model outputs a value that falls on a discrete coordinate system. These are the width and height of the predicted bounding box, respectively. These are the width and height of the actual bounding box, respectively. The aspect ratio loss factor To balance the proportional parameters, For intersection, union, and comparison, For the prediction box, For the true frame, This is the area of ​​the region where the predicted bounding box intersects with the ground truth bounding box. The area of ​​the entire region where the predicted bounding box and the ground truth bounding box are merged. The diagonal length of the smallest bounding rectangle that can simultaneously cover both the predicted and ground truth bounding boxes. This indicates the calculation of the Euclidean distance between two center points. These represent the geometric center points of the predicted bounding box and the ground truth bounding box, respectively. The true label representing the defect category, This indicates the predicted defect category.

[0033] See Figure 4 In this embodiment, the fine localization module includes a feature extraction module, which comprises two branches. Each branch consists of two consecutively stacked convolutional layers with a kernel size of 3×3 and a stride of 2, and a C2F module to achieve feature extraction and downsampling. The outputs of the first and second branches are connected to the inputs of the first and second defect texture interaction attention modules, respectively. The outputs of the first and second defect texture interaction attention modules are connected to the texture feature fusion module.

[0034] S105: Input the test set and the coarse localization fused texture feature map of the test set into the optimal photovoltaic panel defect detection model. Through interactive attention mechanism, feature enhancement and channel adaptive fusion are performed to output the defect category and defect boundary localization of the photovoltaic panel.

[0035] In this embodiment, the test set and the coarse localization fusion texture feature map of the test set are input into the fine localization module of the optimal photovoltaic panel defect detection model, and the fine localization feature map of the test set is output through feature extraction and defect texture interaction.

[0036] See Figure 5 In this embodiment, the test set is input into the first branch of the fine localization module. Through continuous convolutional downsampling and C2F module processing, the original semantic feature map is output. Simultaneously, the coarse localization fused texture feature map of the test set is input into the second branch of the fine localization module. After the same convolutional downsampling and C2F module processing, the fused texture feature map is output. Then, the original semantic feature map and the fused texture feature map are input into two defect texture interaction attention modules. The specific calculation interaction process of the defect texture interaction attention module is as follows: the original semantic feature map is linearly projected through a 1×1 convolution to obtain the corresponding first query matrix and first key-value matrix; the fused texture feature map is linearly projected through a convolution to obtain the first value matrix. The calculation formula is: in, and Let represent the first query matrix and the first key-value matrix obtained by linearly projecting the original semantic feature map through a convolution with a kernel size of 1, respectively. This represents the first value matrix obtained by linearly projecting the fused texture feature map through a convolution with a kernel size of 1. This indicates a convolutional layer with a kernel size of 1. These represent the original semantic feature map and the fused texture feature map, respectively.

[0037] The attention weights for the original semantic feature extraction branch are calculated based on the first query matrix, the first key-value matrix, and the first value matrix. The calculation formula is as follows: To enhance the details of defects while fully preserving basic semantic information such as the global structure and spatial relationships of the image, the attention weights of the original semantic feature extraction branch are residually connected to the original semantic feature map to output an enhanced original semantic feature map. The calculation formula is as follows: in, To enhance the original semantic feature map, To extract attention weights for branches of the original semantic features, For the number of channels, for transpose, This is the activation function.

[0038] See Figure 6 Similarly, the fused texture feature map is convolved and linearly projected to obtain the corresponding second query matrix and second key-value matrix, and the original semantic feature map is convolved and linearly projected to obtain the second value matrix. The calculation formula is as follows: in, and Let represent the second query matrix and the second key-value matrix obtained by linearly projecting the fused texture feature map through a convolution with a kernel size of 1, respectively. This represents the second-value matrix obtained by linearly projecting the original semantic feature map through a convolution with a kernel size of 1.

[0039] The attention weights of the fused texture feature extraction branch are calculated based on the second query matrix, the second key-value matrix, and the second value matrix. The attention weights of the fused texture feature extraction branch are then residually connected to the fused texture feature map to output an enhanced fused texture feature map. The calculation formula is as follows: in, To enhance the fusion of texture feature maps, Attention weights are extracted for branches to fuse texture features. for The transpose of .

[0040] The enhanced original semantic feature map and the enhanced fused texture feature map are input into the texture feature fusion module for channel-weighted fusion. See [link to module]. Figure 7 First, the enhanced original semantic feature map and the enhanced fused texture feature map are concatenated along the channel dimension, and then a convolutional layer and a 1×1 dimensionality reduction convolutional layer are used to compress the dimension, resulting in a fused feature map. Next, a global average pooling layer compresses the two-dimensional spatial features into a one-dimensional global feature vector, comprehensively capturing the overall semantic information of each channel. Finally, the weight vector obtained by sequentially passing through a convolutional layer, a ReLU activation function layer, another convolutional layer, and a Softmax activation function layer is weighted and processed channel-by-channel with the fused feature map to reduce redundant features while improving the ability to perceive defective features, outputting a fine-grained localization feature map for the test set.

[0041] In this embodiment, the fine localization feature map is input into the adaptive multi-scale feature pyramid network. After cross-scale bidirectional fusion and scale-aware weighted enhancement, the multi-scale enhanced defect feature map is output. The multi-scale enhanced defect feature map is output through feature extraction and weighted enhancement interaction.

[0042] See Figure 3 The fine-grained localization feature map is sequentially input into the first CBS module and the first C2F module to obtain the shallow feature map. Then, the shallow feature map is sequentially input into the second CBS module and the second C2F module to obtain the mid-level feature map. Finally, the mid-level feature map is sequentially input into the third CBS module, the third C2F module, and the SPPF module to obtain the deep feature map.

[0043] Shallow, mid-level, and deep feature maps are input into an adaptive multi-scale feature pyramid network. First, the deep feature map is upsampled. Then, the upsampled deep and mid-level feature maps are combined... Figure 1 The same input is fed into the first-scale perceptual feature enhancement module. See also... Figure 8The scale-aware feature enhancement module does not simply concatenate the multi-scale feature maps output by the autonomous backbone network along the channel dimension and then perform feature fusion using convolution. Instead, it concatenates two feature maps of different scales along the channel dimension. For example, the first scale-aware feature enhancement module concatenates the upsampled deep and mid-level feature maps along the channel dimension to obtain a fused feature map. Subsequently, a global spatial information aggregation operation is performed on the fused feature map. Specifically, it uses a global average pooling layer to compress the two-dimensional spatial features into a one-dimensional global feature vector, comprehensively capturing the overall semantic information of each channel. Then, a convolutional layer with a kernel size of 1×1 is used to complete the dimensionality compression and extract key channel correlation information. After a nonlinear transformation using a ReLU activation function layer, a convolutional layer with a kernel size of 1×1 is used to restore the original channel dimension. Then, the weight vector obtained through the Softmax activation function layer is used to perform channel-wise weighted operations with the fused feature map to achieve efficient fusion of features at different scales while enhancing feature information beneficial to detection. Finally, a C2F module is used to obtain the first multi-scale enhanced defect feature map.

[0044] Similarly, after upsampling, the first multi-scale enhanced defect feature map and the shallow feature map are sequentially input into the second-scale sensing feature enhancement module and the C2F module to obtain the second multi-scale enhanced defect feature map. The second multi-scale enhanced defect feature map and the first multi-scale enhanced defect feature map, after being processed by the CBS module and downsampled, are sequentially input into the third-scale sensing feature enhancement module and the C2F module to obtain the third multi-scale enhanced defect feature map. The third multi-scale enhanced defect feature map and the deep feature map, after being processed by the CBS module and downsampled, are sequentially input into the fourth-scale sensing feature enhancement module and the C2F module to obtain the fourth multi-scale enhanced defect feature map. Finally, the second, third, and fourth multi-scale enhanced defect feature maps are input into the first, second, and third detection heads, respectively, and the defect categories and defect boundaries of the output photovoltaic panel are located through two parallel CBS modules and convolutional layers.

[0045] In this embodiment, "multiple" refers to two or more. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent the existence of A alone, the simultaneous existence of A and B, or the existence of B alone. A and B can be singular or plural. The character " / " generally indicates that the preceding and following related objects have an "or" relationship.

[0046] The above description is merely a specific embodiment of this application. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the protection scope of this application.

Claims

1. An adaptive multi-scale photovoltaic panel defect detection method integrating gray-level co-occurrence matrix and interactive attention, characterized in that, include: A photovoltaic panel defect detection dataset is obtained. After standardization preprocessing, the standard photovoltaic panel defect detection dataset is divided into a training set and a test set. By statistically analyzing the joint gray-level probability distribution of pixel pairs with preset spatial distance and orientation relationships in each image of the training set and the test set, a gray-level co-occurrence matrix for the training set and a gray-level co-occurrence matrix for the test set are constructed. The texture features of photovoltaic panel images in the training set and the test set are extracted using the gray-level co-occurrence matrix of the training set and the gray-level co-occurrence matrix of the test set, respectively, to obtain coarse localization and fusion texture feature maps of the training set and the test set. The training set and the coarse localization fusion texture feature map of the training set are input into the pre-built photovoltaic panel defect detection model for training to obtain the optimal photovoltaic panel defect detection model; The photovoltaic panel defect detection model includes a backbone network, an adaptive multi-scale feature pyramid network, and a detection output terminal. The backbone network includes a fine localization module; the fine localization module includes a feature extraction module, which includes a first branch and a second branch. The first branch and the second branch have the same structure, each containing a first convolutional layer, a second convolutional layer, and a C2F module connected in sequence. The output terminals of the first branch and the second branch are respectively connected to the input terminals of the first defect texture interaction attention module and the second defect texture interaction attention module. The output terminals of the first defect texture interaction attention module and the second defect texture interaction attention module are connected to a texture feature fusion module. The test set and the coarse-localized fused texture feature map of the test set are input into the fine-localization module of the optimal photovoltaic panel defect detection model. A fine-localization feature map of the test set is output through feature extraction and defect texture interaction. This includes: inputting the test set into the first branch of the fine-localization module, processing it through continuous convolutional downsampling and C2F module, and outputting the original semantic feature map; simultaneously, inputting the coarse-localized fused texture feature map of the test set into the second branch of the fine-localization module, processing it through the same convolutional downsampling and C2F module, and outputting the fused texture feature map; obtaining the corresponding first query matrix and first key-value matrix by convolutional linear projection of the original semantic feature map, obtaining the first value matrix by convolutional linear projection of the fused texture feature map, calculating the attention weights of the original semantic feature extraction branch based on the first query matrix, first key-value matrix, and first value matrix, and performing a residual connection between the attention weights of the original semantic feature extraction branch and the original semantic feature map to output an enhanced original semantic feature map. The fused texture feature map is subjected to convolutional linear projection to obtain the corresponding second query matrix and second key-value matrix. The original semantic feature map is subjected to convolutional linear projection to obtain the second value matrix. The attention weights of the fused texture feature extraction branch are calculated based on the second query matrix, the second key-value matrix, and the second value matrix. The attention weights of the fused texture feature extraction branch are then residually connected to the fused texture feature map to output an enhanced fused texture feature map. The enhanced original semantic feature map and the enhanced fused texture feature map are input into the texture feature fusion module for channel-weighted fusion to output a fine-grained localization feature map for the test set. The fine localization feature map is input into an adaptive multi-scale feature pyramid network. After cross-scale bidirectional fusion and scale-aware weighted enhancement, a multi-scale enhanced defect feature map is output. The multi-scale enhanced defect feature map is then input into the corresponding detection head at the detection output end to output the defect category and defect boundary location of the photovoltaic panel.

2. The adaptive multi-scale photovoltaic panel defect detection method based on the fusion of gray-level co-occurrence matrix and interactive attention as described in claim 1, characterized in that, The output of the fine localization module is connected to the input of the first CBS module, the output of the first CBS module is connected to the input of the first C2F module, the output of the first C2F module is connected to the input of the second CBS module, the output of the second CBS module is connected to the input of the second C2F module, the output of the second C2F module is connected to the input of the third CBS module, the output of the third CBS module is connected to the input of the third C2F module, the output of the third C2F module is connected to the input of the SPPF module, and the output of the SPPF module is connected to the adaptive multi-scale feature pyramid network. The adaptive multi-scale feature pyramid network includes a first upsampling layer. The input of the first upsampling layer is connected to the output of the SPPF module. The outputs of the first upsampling layer and the second C2F module are connected to the input of a first scale-aware feature enhancement module. The output of the first scale-aware feature enhancement module is connected to the input of a fourth C2F module. The output of the fourth C2F module is connected to the input of a second upsampling layer. The outputs of the second upsampling layer and the first C2F module are connected to the input of a second scale-aware feature enhancement module. The output of the second scale-aware feature enhancement module is connected to the input of a fifth C2F module. The output of the fifth C2F module is connected to the first detection head of both the input of the fourth CBS module and the detection output. The output of the fourth CBS module is connected to the input of the first downsampling layer. The outputs of the first downsampling layer and the fourth C2F module are both connected to the input of the third scale-aware feature enhancement module. The output of the third scale-aware feature enhancement module is connected to the input of the sixth C2F module. The output of the sixth C2F module is connected to the input of the fifth CBS module and the second detection head of the detection output, respectively. The output of the fifth CBS module is connected to the input of the second downsampling layer. The outputs of the second downsampling layer and the SPPF module are both connected to the input of the fourth scale-aware feature enhancement module. The output of the fourth scale-aware feature enhancement module is connected to the input of the seventh C2F module. The output of the seventh C2F module is connected to the third detection head of the detection output.

3. The adaptive multi-scale photovoltaic panel defect detection method based on the fusion of gray-level co-occurrence matrix and interactive attention as described in claim 1, characterized in that, The original semantic feature map is subjected to convolutional linear projection to obtain the corresponding first query matrix and first key-value matrix, and the fused texture feature map is subjected to convolutional linear projection to obtain the first value matrix. The calculation formula is as follows: in, and Let represent the first query matrix and the first key-value matrix obtained by linearly projecting the original semantic feature map through a convolution with a kernel size of 1, respectively. This represents the first value matrix obtained by linearly projecting the fused texture feature map through a convolution with a kernel size of 1. This indicates a convolutional layer with a kernel size of 1. These represent the original semantic feature map and the fused texture feature map, respectively. The second query matrix and the second key-value matrix are obtained by convolutional linear projection of the fused texture feature map, and the second value matrix is ​​obtained by convolutional linear projection of the original semantic feature map. The calculation formula is as follows: in, and Let represent the second query matrix and the second key-value matrix obtained by linearly projecting the fused texture feature map through a convolution with a kernel size of 1, respectively. This represents the second-value matrix obtained by linearly projecting the original semantic feature map through a convolution with a kernel size of 1.

4. The adaptive multi-scale photovoltaic panel defect detection method based on the fusion of gray-level co-occurrence matrix and interactive attention as described in claim 3, characterized in that, The attention weights of the original semantic feature extraction branch are calculated based on the first query matrix, the first key-value matrix, and the first value matrix. The attention weights of the original semantic feature extraction branch are then residually connected to the original semantic feature map. The calculation formulas for enhancing the original semantic feature map are as follows: in, To enhance the original semantic feature map, To extract attention weights for branches of the original semantic features, For the number of channels, for transpose, This is the activation function.

5. The adaptive multi-scale photovoltaic panel defect detection method based on the fusion of gray-level co-occurrence matrix and interactive attention as described in claim 3, characterized in that, The attention weights of the fused texture feature extraction branch are calculated based on the second query matrix, the second key-value matrix, and the second value matrix. The attention weights of the fused texture feature extraction branch are then residually concatenated with the fused texture feature map. The calculation formulas for the enhanced fused texture feature map are as follows: in, To enhance the fusion of texture feature maps, Attention weights are extracted for branches to fuse texture features. for The transpose of .

Images