Photovoltaic panel detection method and system based on lightweight RT-DETR model

Abstract

This invention belongs to the field of target detection technology and provides a photovoltaic panel detection method and system based on a lightweight RT-DETR model. The method includes: acquiring a photovoltaic panel surface image; obtaining a dirt detection result using the photovoltaic panel surface image and a pre-set photovoltaic panel surface dirt detection model; and performing photovoltaic panel dirt detection using the RT-DETR model. Based on the StarNet backbone network, the method reduces redundant computation through a simplified network design, improves the hybrid encoder using a multi-scale multi-head attention mechanism and an internal feature interaction module, and achieves a lightweight improvement to the RT-DETR model. Compared with traditional target detection algorithms, this method can acquire more high-level feature information while reducing computation when processing photovoltaic panel images with irregular dirt distribution and small dirt targets, resulting in higher accuracy and faster speed. It also demonstrates high accuracy in detecting small targets such as photovoltaic panel dirt.

Description

Technical Field

[0001] This invention belongs to the field of target detection technology, and in particular relates to a photovoltaic panel detection method and system based on a lightweight RT-DETR model. Background Technology

[0002] After prolonged use, photovoltaic panels accumulate dirt on their surface, affecting their operation and power generation efficiency. With the development of image processing and detection technology, it is now possible to quickly detect the dirt on the surface of photovoltaic panels.

[0003] Current technologies that use image processing and network models for photovoltaic panel detection suffer from the following characteristics: irregular distribution and small size of dirt in the acquired photovoltaic panel images. This results in a high computational load for current network models, which cannot obtain more advanced image feature information. Consequently, the detection accuracy of photovoltaic panels is low and the speed is slow, which is not conducive to the rapid determination of targeted cleaning measures and affects power generation. Summary of the Invention

[0004] To address the aforementioned problems, this invention proposes a photovoltaic panel detection method and system based on a lightweight RT-DETR model. Based on the StarNet backbone network, this invention reduces redundant computation through a simplified network design, improves the hybrid encoder by employing a multi-scale, multi-head attention mechanism with an internal feature interaction module, and achieves a lightweight improvement to the RT-DETR model. Compared to traditional target detection algorithms, this method can acquire more advanced feature information while reducing computation when processing photovoltaic panel images characterized by irregular dirt distribution and small dirt targets. It also boasts higher accuracy and faster speed, exhibiting high accuracy in detecting small targets such as dirt on photovoltaic panels.

[0005] To achieve the above objectives, the present invention is implemented through the following technical solution:

[0006] In a first aspect, the present invention provides a photovoltaic panel detection method based on a lightweight RT-DETR model, comprising:

[0007] Acquire images of the photovoltaic panel surface;

[0008] The dirt detection results are obtained by using images of the photovoltaic panel surface and a pre-set dirt detection model for the photovoltaic panel surface.

[0009] The photovoltaic panel surface dirt detection model is the RT-DETR model, which reduces redundant computation based on the StarNet backbone network. The StarNet backbone network adopts a four-stage hierarchical architecture and achieves feature extraction through progressive abstraction. After acquiring the photovoltaic panel surface image, downsampling is performed through convolutional layers at each stage. While compressing the feature map size in the spatial dimension, the modular structure enhances the feature expression capability. The hybrid encoder is improved by using a multi-scale multi-head attention mechanism with an internal feature interaction module.

[0010] Furthermore, the network employs a constant channel expansion factor to ensure that feature information is fully integrated in the channel dimension at each stage, while the network width of each new stage follows the doubling principle; multi-scale feature maps are generated through stepwise downsampling: shallow features capture the edge and texture information of the photovoltaic panel; deep features extract global semantic information.

[0011] Furthermore, the feature interaction module within the multi-scale multi-head attention mechanism includes: performing intra-scale interaction using a single-scale encoder, applying self-attention operations to higher-level features with richer semantic concepts to capture the connections between conceptual entities; dynamically adjusting the attention weights of reflective and shadow regions to suppress environmental interference; and feeding low-level and high-level features into the multi-scale feature extraction module, which performs a set of three-way dilated convolutions, residual connections, normalization, and adaptive pooling operations before finally outputting the features.

[0012] Furthermore, the dilated convolution group processes features in parallel using different dilation rates. The 3×3 convolution captures local texture, the equivalent 7×7 convolution perceives medium-scale structures, and the equivalent 11×11 convolution covers a wide range of context, including the arrangement pattern of entire rows of panels. The residual connection adds the original features to the multi-scale features, preserving fine-grained information while enhancing semantic expression. The adaptive pooling dynamically adjusts the pooling kernel size according to the size of the input feature map.

[0013] Furthermore, the improved hybrid encoder includes: capturing global contextual information through a self-attention mechanism, considering information from all regions in the image; the specific calculation method is as follows:

[0014]

[0015] Where d is the dimension of the attention head, Q, K and V are the query, key and value, respectively; W is the weight matrix.

[0016] Furthermore, multi-head self-attention projects features into multiple independent subspaces, capturing semantic information in different dimensions: some heads focus on panel edge alignment and array arrangement patterns; others analyze surface conditions and environmental disturbances.

[0017] Secondly, the present invention also provides a photovoltaic panel inspection system based on a lightweight RT-DETR model, comprising:

[0018] The data acquisition module is configured to acquire images of the photovoltaic panel surface.

[0019] The detection module is configured to obtain dirt detection results using images of the photovoltaic panel surface and a preset dirt detection model for the photovoltaic panel surface;

[0020] The photovoltaic panel surface dirt detection model is the RT-DETR model, which reduces redundant computation based on the StarNet backbone network. The StarNet backbone network adopts a four-stage hierarchical architecture and achieves feature extraction through progressive abstraction. After acquiring the photovoltaic panel surface image, downsampling is performed through convolutional layers at each stage. While compressing the feature map size in the spatial dimension, the modular structure enhances the feature expression capability. The hybrid encoder is improved by using a multi-scale multi-head attention mechanism with an internal feature interaction module.

[0021] Thirdly, the present invention also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the photovoltaic panel detection method based on the lightweight RT-DETR model described in the first aspect.

[0022] Fourthly, the present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and capable of running on the processor, wherein the processor executes the program to implement the steps of the photovoltaic panel detection method based on the lightweight RT-DETR model described in the first aspect.

[0023] Fifthly, the present invention also provides a computer program product, the computer program product comprising a computer program, which, when executed by a processor, implements the steps of the photovoltaic panel detection method based on the lightweight RT-DETR model described in the first aspect.

[0024] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0025] This invention employs the RT-DETR model for photovoltaic panel dirt detection. Based on the StarNet backbone network, it reduces redundant computation through a simplified network design and improves the hybrid encoder by using a multi-scale, multi-head attention mechanism with an internal feature interaction module. This lightweight improvement to the RT-DETR model yields better results than traditional target detection algorithms. When processing photovoltaic panel images characterized by irregular dirt distribution and small dirt targets, it can acquire more high-level feature information while reducing computation, resulting in higher accuracy and faster speed. It also demonstrates high accuracy in detecting small targets such as dirt on photovoltaic panels. Attached Figure Description

[0026] The accompanying drawings, which form part of this embodiment, are used to provide a further understanding of this embodiment. The illustrative embodiments and their descriptions are used to explain this embodiment and do not constitute an improper limitation of this embodiment.

[0027] Figure 1 This is a flowchart of the method in Embodiment 1 of the present invention;

[0028] Figure 2 This is a structural diagram of the algorithm model in Embodiment 1 of the present invention;

[0029] Figure 3 This is a diagram of the four-stage StarNet network structure in Embodiment 1 of the present invention;

[0030] Figure 4 This is a structural diagram of the feature interaction module within the multi-scale multi-head attention mechanism of Embodiment 1 of the present invention;

[0031] Figure 5 This is a structural diagram of the multi-scale feature extraction module in Embodiment 1 of the present invention. Detailed Implementation

[0032] The present invention will be further described below with reference to the accompanying drawings and embodiments.

[0033] It should be noted that the following detailed descriptions are exemplary and intended to provide further explanation of this application. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application pertains.

[0034] Example 1:

[0035] In recent years, target detection algorithms have been widely used. However, the speed and accuracy of target detection remain at a relatively rudimentary stage. With the development of the social economy and the construction of more and more photovoltaic power plants, the development of photovoltaic panel surface detection algorithms will become a very challenging problem. Furthermore, with the improvement of large-scale models and the advancement of related electronic technologies, the market for photovoltaic panel inspection will continue to grow.

[0036] As described in the background section, the irregular distribution and small size of dirt make current network models computationally intensive and unable to obtain more advanced image feature information. This results in low accuracy and slow speed in detecting dirt on photovoltaic panels, hindering the rapid and targeted determination of cleaning measures and impacting power generation.

[0037] To address at least one of the aforementioned problems, this embodiment provides a photovoltaic panel detection method based on a lightweight RT-DETR model, such as... Figure 1As shown, firstly, an image of the photovoltaic panel surface is acquired; then, using the photovoltaic panel surface image and a pre-defined photovoltaic panel surface dirt detection model, dirt detection results are obtained. Specifically, the RT-DETR model is used for photovoltaic panel dirt detection. Based on the StarNet backbone network, a simplified network design reduces redundant computation. A multi-scale, multi-head attention mechanism with an internal feature interaction module improves the hybrid encoder, resulting in a lightweight improvement to the RT-DETR model. Compared to traditional object detection algorithms, when processing photovoltaic panel images characterized by irregular dirt distribution and small dirt targets, it can acquire more high-level feature information while reducing computation, achieving higher accuracy and faster speed. It also demonstrates high accuracy in detecting small targets such as photovoltaic panel dirt. Figure 2 As shown, this embodiment addresses the problems of low recognition accuracy and slow speed in existing algorithms in the prior art. The method includes:

[0038] S1. Construct photovoltaic panel surface dirt data based on photovoltaic panel surface dirt images; images of photovoltaic panel surface dirt can be acquired using equipment such as cameras;

[0039] S2. Based on the StarNet backbone network, a simplified network design is used to reduce redundant computation. The AIFI-MSMHSA (Multi-Scale Multi-Head Attention Mechanism Intra-Feature Interaction) module is used to improve the efficient hybrid encoder, thereby achieving a lightweight improvement to the RT-DETR model. The photovoltaic panel surface dirt detection model is obtained by training the RT-DETR model.

[0040] S3. Based on the photovoltaic panel surface dirt detection model, detect the collected photovoltaic panel images to determine the target dirt in the photovoltaic panel images; such as the location, size or other indicators used to evaluate the dirt.

[0041] In some embodiments, the RT-DETR lightweight improvement model includes:

[0042] like Figure 3 As shown, the StarNet backbone network adopts a four-stage hierarchical architecture design, achieving efficient feature extraction through progressive abstraction. After the original image is input into the backbone network, downsampling is performed through convolutional layers at each stage. This compresses the feature map size in the spatial dimension while enhancing feature representation capabilities through a modular structure, as detailed below:

[0043]

[0044] Here, ω is defined as a single entity combining the weight matrix and the bias term, with input vector x.

[0045] in,

[0046]

[0047] In the feature processing module design, depthwise separable convolutions are introduced at the end of each basic block. This design maintains spatial feature extraction capabilities while significantly reducing parameter computation. After passing through the feature processing module, low-level and high-level features of the original image are obtained. The network uses a constant channel expansion factor of 4 to ensure full fusion of feature information in the channel dimension at each stage. Simultaneously, the network width (number of channels) of each new stage strictly follows a doubling principle, forming a typical channel growth pattern of 32-64-128-256. Multi-scale feature maps are generated through progressive downsampling: shallow features (32-64 channels) capture detailed information such as the edges and textures of photovoltaic panels, suitable for locating small-sized or localized targets; deep features (128-256 channels) extract global semantic information, identifying occlusion relationships in large-area panel arrays or complex backgrounds. This design significantly improves the model's adaptability to diverse photovoltaic panel sizes and densely arranged scenarios.

[0048] In some embodiments, the flexibility of this architecture is reflected in the ability to derive different scale variants by adjusting the number of stacked layers of modules in each stage and the initial input channel cardinality. The expansion factor of 4 ensures that fine-grained information can still be preserved through dimensionality increase operations in the low-channel stages, preventing the loss of small target features. The network introduces residual connection structures between modules, which effectively alleviates the gradient vanishing problem. At the same time, grouped convolution techniques are used to improve parallel computing efficiency.

[0049] In some embodiments, such as Figure 4 As shown, the AIFI-MSMHSA (Artificial Intelligence of Features-Multi-Scale Multi-Head Attention Mechanism) module includes: AIFI performs intra-scale interactions by using a single-scale Transformer encoder, further reducing computational costs. Applying self-attention operations to high-level features with richer semantic concepts captures the connections between conceptual entities, which helps subsequent modules locate and identify objects. In high-level features, the semantic concepts of photovoltaic panels are highly abstracted; the self-attention mechanism establishes global associations between different semantic concepts, for example, by dynamically adjusting the attention weights of "reflective areas" and "shaded areas," suppressing environmental interference. Figure 5 As shown, the low-level and high-level features are then fed into the multi-scale feature extraction module. This module performs a set of three dilated convolutions, followed by residual connections, normalization, and adaptive pooling, finally outputting the features. Compared to ordinary convolutions where each element can only perceive a local region of the input feature map, dilated convolutions insert "holes" (spaced pixels) between kernel elements, allowing each element to cover a larger area, thus efficiently expanding the receptive field and capturing a wide range of context with a small number of parameters. The equivalent convolution kernel calculation process is as follows:

[0050] K' = K + (K-1)*(d-1);

[0051] Where K' is the equivalent kernel size, K is the actual kernel size, and d is the dilation rate.

[0052] At the same time, it incorporates a flexible multi-scale design, allowing for adjustments to the expansion rate combination to suit different task requirements. The specific output is as follows:

[0053]

[0054] Among them, S out With S in These represent the input and output dimensions, p is the number of pixels to fill, d is the dilation rate, K is the kernel size, and s is the kernel stride.

[0055] The three dilated convolutional groups process features in parallel using different dilation rates: Dilation rate 1: a standard 3×3 convolution captures local texture; Dilation rate 2: an equivalent 7×7 convolution perceives mid-scale structures; Dilation rate 3: an equivalent 11×11 convolution covers a wide range of context, including the arrangement of entire rows of panels. Residual connections add the original features to multi-scale features, preserving fine-grained information while enhancing semantic expression. Adaptive pooling dynamically adjusts the pooling kernel size according to the input feature map size, ensuring that: aerial images output fixed-dimensional feature vectors; and close-up images retain key detail features, avoiding information loss.

[0056] In some embodiments, the efficient hybrid encoder includes: a large Transformer model capturing global contextual information through a self-attention mechanism, considering information from all regions of the image, rather than being limited to local regions. The specific calculation method is as follows:

[0057]

[0058] Where d is the dimension of the attention head, and Q, K, and V are the query, key, and value, respectively, as follows:

[0059] Q = QW Q ;

[0060] K = KW K ;

[0061] V = VW V ;

[0062] Where W is the weight matrix.

[0063] Multi-scale features are used to extract and analyze global information more effectively through a multi-head self-attention mechanism. By projecting features into parallel independent subspaces, attention distributions across different semantic dimensions are calculated separately, and finally, these are concatenated and fused to form a global feature map containing rich contextual relationships. The specific calculation method is as follows:

[0064] MultiHead=concat(head1,head2,...,head n W o ;

[0065] Multi-head self-attention projects features into 8-16 independent subspaces, capturing semantic information of different dimensions: some heads focus on physical structure, such as panel edge alignment and array arrangement patterns; others analyze surface conditions and environmental interference. The concatenation and fusion of multi-head outputs forms decision features containing multi-dimensional criteria, significantly improving the accuracy of defect classification. Learnable positional encoding injects absolute / relative positional information into each pixel: absolute encoding marks pixel coordinates, ensuring the model understands the geometric position of the panel in the array; relative encoding, through a distance decay function in the attention matrix, strengthens the feature continuity between adjacent panels, avoiding erroneous associations of distant panels, ultimately resulting in an accurate recognition image. This design can accurately distinguish densely packed individual panels in aerial images, preventing overlapping detection boxes or missed detections.

[0066] In some embodiments, the RT-DETR multi-layer decoder architecture decomposes the detection task into multiple layers, each responsible for different tasks, such as object localization, object classification, and keypoint detection. This hierarchical processing allows the model to adjust the allocation of computational resources according to different real-time scenarios, thereby achieving flexible speed adjustment and adapting to various real-time scenarios without retraining.

[0067] In some embodiments, detecting the location and number of photovoltaic panels can be further expanded to enable the detection and classification of defects in photovoltaic panels. For example, models can be trained to identify, classify, and locate defects such as cracks, stains, and breaks on photovoltaic panels, thereby improving the operation and maintenance efficiency and quality of photovoltaic power plants.

[0068] In this embodiment, a photovoltaic panel surface dirt dataset is constructed based on images of dirt on the photovoltaic panel surface. Based on the StarNet backbone network, a simplified network design reduces redundant computation. An improved high-efficiency hybrid encoder is achieved using a multi-scale multi-head attention mechanism with internal feature interaction (AIFI-MSMHSA) module, resulting in a lightweight improvement to the RT-DETR model. The photovoltaic panel surface dirt detection model is used to detect and identify target dirt in the acquired photovoltaic panel images. Compared to traditional object detection algorithms, this invention can acquire more high-level feature information, reduce computation, and achieve higher accuracy and faster speed in model detection, while maintaining high accuracy for small object detection. Thanks to DETR's multi-layer decoder architecture, RT-DETR supports flexible speed adjustment and can adapt to various real-time scenarios without retraining.

[0069] Example 2:

[0070] This embodiment provides a photovoltaic panel inspection system based on a lightweight RT-DETR model, including:

[0071] The data acquisition module is configured to acquire images of the photovoltaic panel surface.

[0072] The detection module is configured to obtain dirt detection results using images of the photovoltaic panel surface and a preset dirt detection model for the photovoltaic panel surface;

[0073] The photovoltaic panel surface dirt detection model is the RT-DETR model, which reduces redundant computation based on the StarNet backbone network. The StarNet backbone network adopts a four-stage hierarchical architecture and achieves feature extraction through progressive abstraction. After acquiring the photovoltaic panel surface image, downsampling is performed through convolutional layers at each stage. While compressing the feature map size in the spatial dimension, the modular structure enhances the feature expression capability. The hybrid encoder is improved by using a multi-scale multi-head attention mechanism with an internal feature interaction module.

[0074] The working method of the system is the same as that of the photovoltaic panel detection method based on the lightweight RT-DETR model in Example 1, and will not be repeated here.

[0075] Example 3:

[0076] This embodiment provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of the photovoltaic panel detection method based on the lightweight RT-DETR model described in Embodiment 1.

[0077] Example 4:

[0078] This embodiment provides an electronic device, including a memory, a processor, and a computer program stored in the memory and capable of running on the processor. When the processor executes the program, it implements the steps of the photovoltaic panel detection method based on the lightweight RT-DETR model described in Embodiment 1.

[0079] Example 5:

[0080] This embodiment provides a computer program product, which includes a computer program. When the computer program is executed by a processor, it implements the steps of the photovoltaic panel detection method based on the lightweight RT-DETR model described in Embodiment 1.

[0081] The above description is merely a preferred embodiment of this practice and is not intended to limit the scope of this practice. Various modifications and variations can be made to this practice by those skilled in the art. Any modifications, equivalent substitutions, or improvements made within the spirit and principles of this practice should be included within the protection scope of this practice.

Claims

1. A photovoltaic panel detection method based on a lightweight RT-DETR model, characterized in that, include: Acquire images of the photovoltaic panel surface; The dirt detection results are obtained by using images of the photovoltaic panel surface and a pre-set dirt detection model for the photovoltaic panel surface. The photovoltaic panel surface dirt detection model is the RT-DETR model, which reduces redundant computation based on the StarNet backbone network. The StarNet backbone network adopts a four-stage hierarchical architecture and achieves feature extraction through progressive abstraction. After acquiring the photovoltaic panel surface image, downsampling is performed through convolutional layers at each stage. While compressing the feature map size in the spatial dimension, the modular structure enhances the feature expression capability. A multi-scale multi-head attention mechanism with an internal feature interaction module is used to improve the hybrid encoder. The multi-scale multi-head attention mechanism's feature interaction module includes: using a single-scale encoder to perform intra-scale interaction, applying self-attention operations to higher-level features with richer semantic concepts to capture the connections between conceptual entities; dynamically adjusting the attention weights of reflective and shadow regions to suppress environmental interference; and feeding low-level and high-level features into the multi-scale feature extraction module, which performs a set of three-way dilated convolutions, residual connections, normalization, and adaptive pooling operations before finally outputting the features. The dilated convolution group processes features in parallel using different dilation rates. The 3×3 convolution captures local texture, the equivalent 7×7 convolution perceives medium-scale structures, and the equivalent 11×11 convolution covers a wide range of context, including the arrangement of entire rows of panels. The residual connection adds the original features to the multi-scale features, preserving fine-grained information while enhancing semantic expression. The adaptive pooling dynamically adjusts the pooling kernel size according to the size of the input feature map.

2. The photovoltaic panel detection method based on the lightweight RT-DETR model as described in claim 1, characterized in that, The network employs a constant channel expansion factor to ensure that feature information is fully integrated in the channel dimension at each stage, while the network width of each new stage follows the doubling principle; multi-scale feature maps are generated through stepwise downsampling: shallow features capture the edge and texture information of photovoltaic panels; deep features extract global semantic information.

3. The photovoltaic panel detection method based on the lightweight RT-DETR model as described in claim 1, characterized in that, The improved hybrid encoder includes: capturing global contextual information through a self-attention mechanism, considering information from all regions in the image; the specific calculation method is as follows: ； in, d For the dimension of attention head, Q , K and V These are query, key, and value, respectively. This is the weight matrix.

4. The photovoltaic panel detection method based on the lightweight RT-DETR model as described in claim 1, characterized in that, Multi-head self-attention projects features into multiple independent subspaces, capturing semantic information in different dimensions: some heads focus on panel edge alignment and array arrangement patterns; others analyze surface conditions and environmental disturbances.

5. A photovoltaic panel inspection system based on a lightweight RT-DETR model, characterized in that, include: The data acquisition module is configured to acquire images of the photovoltaic panel surface. The detection module is configured to obtain dirt detection results using images of the photovoltaic panel surface and a preset dirt detection model for the photovoltaic panel surface; The photovoltaic panel surface dirt detection model is the RT-DETR model, which reduces redundant computation based on the StarNet backbone network. The StarNet backbone network adopts a four-stage hierarchical architecture and achieves feature extraction through progressive abstraction. After acquiring the photovoltaic panel surface image, downsampling is performed through convolutional layers at each stage. While compressing the feature map size in the spatial dimension, the modular structure enhances the feature expression capability. A multi-scale multi-head attention mechanism with an internal feature interaction module is used to improve the hybrid encoder. The multi-scale multi-head attention mechanism's feature interaction module includes: using a single-scale encoder to perform intra-scale interaction, applying self-attention operations to higher-level features with richer semantic concepts to capture the connections between conceptual entities; dynamically adjusting the attention weights of reflective and shadow regions to suppress environmental interference; and feeding low-level and high-level features into the multi-scale feature extraction module, which performs a set of three-way dilated convolutions, residual connections, normalization, and adaptive pooling operations before finally outputting the features. The dilated convolution group processes features in parallel using different dilation rates. The 3×3 convolution captures local texture, the equivalent 7×7 convolution perceives medium-scale structures, and the equivalent 11×11 convolution covers a wide range of context, including the arrangement of entire rows of panels. The residual connection adds the original features to the multi-scale features, preserving fine-grained information while enhancing semantic expression. The adaptive pooling dynamically adjusts the pooling kernel size according to the size of the input feature map.

6. A computer-readable storage medium having a computer program stored thereon, characterized in that, When executed by the processor, the program implements the steps of the photovoltaic panel detection method based on the lightweight RT-DETR model as described in any one of claims 1-4.

7. An electronic device comprising a memory, a processor, and a computer program stored in the memory and capable of running on the processor, characterized in that, When the processor executes the program, it implements the steps of the photovoltaic panel detection method based on the lightweight RT-DETR model as described in any one of claims 1-4.

8. A computer program product, characterized in that, The computer program product includes a computer program that, when executed by a processor, implements the steps of the photovoltaic panel detection method based on the lightweight RT-DETR model as described in any one of claims 1-4.

Images