A method and system for classifying surface defects of a photovoltaic panel

Abstract

The application provides a photovoltaic panel surface defect classification method and system, the method comprises the following steps: constructing a composite image by combining a color image of a photovoltaic panel and its corresponding infrared image as input data, inputting a trained photovoltaic panel defect classification model, and obtaining a fault classification of the photovoltaic panel; the application has the advantages that: the application uses the infrared image of the photovoltaic panel as additional input, combines it with the color image to form new input data of 6 channels, and fuses the two images at the input end, thereby reducing the network volume and increasing the detection speed; the application adopts a model based on a Vision Transformer, can model global information of an image, and has performance superior to that of an ordinary convolutional neural network; and the application further adds a deformable convolution on this basis, enhances the transformation modeling capability of a convolutional layer, further improves the network performance, and increases the detection precision.

Description

Technical Field

[0001] This invention belongs to the field of computer vision and pattern recognition, and specifically relates to a method and system for classifying surface defects of photovoltaic panels. Background Technology

[0002] With the exponential growth of global photovoltaic (PV) power generation capacity, reliable operation and maintenance management and fault diagnosis of PV arrays have become paramount. Because PV arrays are exposed to the outdoors year-round and subject to varying environmental conditions, PV panels are prone to various faults such as breakage, shading, dissolution, and dust accumulation, significantly impacting their lifespan and power generation. Therefore, classifying surface defects on PV panels is crucial. Quick and accurate classification of defects in faulty PV panels allows staff to take timely repair measures, minimizing losses. However, manually classifying images of faulty PV panels consumes significant manpower and time. PV panel images captured by drones often contain redundant background information, and the differences between different defect categories in the images are very small, posing a significant challenge to PV panel defect classification. These difficulties have not yet been adequately addressed in previous research. Summary of the Invention

[0003] The purpose of this invention is to overcome the shortcomings of low efficiency and high cost of manual defect detection of photovoltaic arrays.

[0004] To achieve the above objectives, this invention proposes a method for classifying surface defects in photovoltaic panels, the method comprising:

[0005] A composite image is constructed by combining the color image of the photovoltaic panel with its corresponding infrared image and used as input data. This composite image is then input into a trained photovoltaic panel defect classification model to obtain the fault classification of the photovoltaic panel.

[0006] The processing steps of the photovoltaic panel defect classification model include:

[0007] Step 1: The photovoltaic panel defect classification model receives composite images, uses infrared images as guidance, and uses a multi-head attention module and deformable deep convolutional layers as the core to focus the network's attention on the areas that are beneficial to the classification of photovoltaic panel defects, while removing background and useless information.

[0008] Step 2: Use a deformable convolutional classification head to classify the faults of the photovoltaic panel.

[0009] As an improvement to the above method, the training process of the photovoltaic panel defect classification model includes:

[0010] The deviation between the classification results generated by the photovoltaic panel defect classification model and the calculated true labels is reduced by using the backpropagation algorithm and stochastic gradient descent method to train the model. After multiple iterations of training, the final photovoltaic panel defect classification model is obtained.

[0011] As an improvement to the above method, the step of constructing a composite image from the infrared image and the color image of the photovoltaic panel specifically includes:

[0012] The RGB color image and corresponding infrared image of the photovoltaic panel are constructed into 6-channel input data. Where H and W are the resolutions of the input image.

[0013] As an improvement to the above method, the photovoltaic panel defect classification model is a Vision Transformer deformable deep convolutional neural network model, which includes a Transformer encoding layer and a deformable convolutional layer.

[0014] As an improvement to the above method, step 1 specifically includes:

[0015] Step 1-1: Normalize the input data to a uniform size;

[0016] Step 1-2: Divide the data processed in Step 1-1 into photovoltaic panel image blocks using a convolutional layer with a kernel size of P×P and a stride of P. The number of image patches obtained is N = H × W / P 2 Each image patch is mapped to a one-dimensional vector using a linear mapping to obtain... in Let n represent the nth image patch, where n ∈ (1, N); E represents a linear mapping. Where D represents the image patch that needs to be mapped to D dimensions;

[0017] Steps 1-3: Insert a one-dimensional vector x specifically for photovoltaic panel classification into the multiple one-dimensional vectors obtained in step 1-2. class The vector is optimized through training; the vector x class with vector Concatenating them together yields a one-dimensional vector x:

[0018] Steps 1-4: Combine the one-dimensional vector x with the one-dimensional vector E used for photovoltaic panel location encoding. pos The vectors are added together, giving each vector its own positional information; the positional encoding vectors are trainable parameters; the output tensor is... in C represents the number of input image channels;

[0019] Steps 1-5 involve inputting tensor z0 into L Transformer encoding blocks for feature extraction of the photovoltaic panel; a random depth operation is performed when the tensor is input into each encoding block; the formula for random depth is as follows, and the survival probability of each encoding block decreases linearly:

[0020]

[0021] Where, p l p represents the survival probability of layer l during training; L represents the total number of encoding blocks; p L This indicates the survival probability of the last residual block;

[0022] Steps 1-6: When the tensor z0 from steps 1-5 is input into each encoding block, a layer normalization operation is first performed to map values ​​with different ranges of variation to the same fixed range. Then, the normalized tensor is input into the multi-head self-attention module to model the correlation between each image block in the photovoltaic panel. The attention extraction formula is as follows:

[0023]

[0024] Where Q, K, and V represent the query, index, and content matrices, respectively, obtained from the input feature vectors through linear transformation; T represents the matrix transpose; d k Indicates the dimension of the index matrix;

[0025] Output tensor z' l =MSA(LN(z) l-1 ))+z l-1 , where l=1…L; MSA represents multi-head self-attention module, and LN() represents layer normalization;

[0026] Steps 1-7, convert tensor z' l Layer normalization is performed to map values ​​of different ranges to the same fixed range again; deformable convolutional layers are introduced to replace the MLP layers of the original Vision Transformer network for image feature extraction.

[0027] The formula for deformable convolution is: Where y(p0) represents a point in the output feature map; p0 represents the coordinates of the center point of the convolution kernel corresponding to point y(p0) in the original feature map; p n This represents the coordinates of the convolution kernel relative to p0; Δp n Indicates pixel offset; w(p) n ) represents p n The convolution kernel weights corresponding to the points; x(p0+p n +Δp n () represents the offset pixel value;

[0028] Output tensor z l =DC(LN(z') l ))+z' l , where l = 1…L, and DC represents deformable convolution operation.

[0029] As an improvement to the above method, P is preferably 16.

[0030] As an improvement to the above method, L is preferably 12.

[0031] As an improvement to the above method, step 2 specifically includes:

[0032] The tensor z output in step 1 l Perform layer normalization; extract the classification vector x. class Corresponding output results The result is then fed into a deformable convolutional classification layer to obtain the final classification result for photovoltaic panel faults. Where DC head represents the deformable convolution class header.

[0033] The present invention also provides a photovoltaic panel surface defect classification system, implemented based on the above method, the system comprising:

[0034] The data preprocessing module is used to construct a composite image from the color image of the photovoltaic panel and its corresponding infrared image as input data.

[0035] The classification module processes the input data using a trained photovoltaic panel defect classification model to obtain the fault classification of the photovoltaic panel; and

[0036] The training module is used to train the photovoltaic panel defect classification model.

[0037] Compared with the prior art, the advantages of the present invention are:

[0038] 1. This invention uses the infrared image of the photovoltaic panel as an additional input, which is combined with the color image to form a new 6-channel input data. The two images are fused at the input end, reducing the network size and increasing the detection speed.

[0039] 2. This invention employs a model based on the Vision Transformer, which incorporates a self-attention mechanism, enabling it to model global information of images and outperform ordinary convolutional neural networks. Furthermore, this invention adds deformable convolution to enhance the transformation modeling capability of the convolutional layers, further improving network performance and increasing detection accuracy.

[0040] 3. This invention can classify faults on the surface of photovoltaic panels using only images of photovoltaic panels collected by drones, which greatly reduces the human resources required for traditional manual fault classification and improves operation and maintenance efficiency. Attached Figure Description

[0041] Figure 1 The diagram shows a flowchart of a method for classifying surface defects in photovoltaic panels.

[0042] Figure 2 The diagram shown illustrates the construction of six-channel input data.

[0043] Figure 3 The diagram shown is a structural diagram of a photovoltaic panel defect classification model.

[0044] Figure 4 The diagram shows the Transformer encoder and deformable convolution structure. Detailed Implementation

[0045] The technical solution of the present invention will now be described in detail with reference to the accompanying drawings.

[0046] The purpose of this invention is to address the difficulties encountered in the classification of surface defects in photovoltaic panels in existing technologies. On the one hand, it introduces infrared images of photovoltaic panels to provide temperature information. On the other hand, it introduces a self-attention mechanism to model the relationship between each part of the photovoltaic panel, focusing attention on the areas useful for defect classification. Furthermore, it introduces deformable depthwise convolution to replace ordinary linear layers, thereby better sampling the image features of the photovoltaic panel and ultimately extracting features that are beneficial to the classification of photovoltaic panel defects, thus improving the accuracy of surface defect classification of photovoltaic panels.

[0047] This invention provides a method for classifying surface defects in photovoltaic panels. First, a 6-channel input data is constructed using infrared and color images. This input is fed into a Vision Transformer deformable deep convolutional neural network model. Then, using the infrared image as guidance, and with a multi-head attention module and deformable deep convolutional layers as the core, the network's attention is focused on the region that is beneficial for classifying photovoltaic panel defects, removing background and useless information. Finally, a deformable convolutional classification head is used to correctly classify the defects in the photovoltaic panel.

[0048] like Figure 1 As shown, the method for classifying surface defects in photovoltaic panels includes the following steps:

[0049] Step S0: Construct 6-channel input data from the RGB photovoltaic panel images and corresponding infrared images in the dataset. Where H and W are the resolutions of the input image; for example Figure 2 As shown;

[0050] Step S1: Normalize the photovoltaic panel data in S0 to a uniform size (e.g., 224×224×6 pixels), and then feed this input into the Vision Transformer deformable deep convolutional neural network. This network contains several Transformer coding layers and deformable convolutional layers, such as... Figure 3 As shown;

[0051] Step S2: The photovoltaic data input in S1 is segmented into photovoltaic panel image blocks through a convolutional layer with a kernel of P×P and a stride of P. The number of image patches obtained is N = H × W / P 2 Each image patch is mapped to a one-dimensional vector using a linear mapping to obtain... in Let n represent the nth image patch, n∈(1,N), and E represent a linear mapping. Where D represents the image patch that needs to be mapped to D dimensions; P is preferably 16;

[0052] Step S3: Insert a one-dimensional vector x specifically for photovoltaic panel classification into the multiple one-dimensional vectors obtained in S2. class This vector can be optimized through training; here, it is concatenated with the vector in S2.

[0053] Step S4: Combine the multiple one-dimensional vectors obtained in S3 with the one-dimensional vector E specifically used for photovoltaic panel location encoding. pos Adding them together gives each vector its own positional information. The positional encoding vectors are also trainable parameters, and the output tensor is... in Where C is the number of input image channels; E, E pos The initialization is random.

[0054] Step S5: The tensors from S4 are input into L (L is preferably 12) Transformer encoding blocks for feature extraction from the photovoltaic panel. When the tensors are input into each encoding block, a random depth operation is performed, and the tensor has a chance to skip that block. This reduces the dependence of the encoding block on a specific feature of the photovoltaic panel, alleviates the overfitting problem of deep networks, and enhances the network's generalization performance. The formula for random depth is as follows: the survival probability of each encoding block decreases linearly:

[0055]

[0056] Where, p l p represents the survival probability of the l-th layer during training, where L represents the total number of encoding blocks. L It refers to the survival probability of the last residual block.

[0057] Step S6: When the tensor from S5 is input into each encoding block, a layer normalization operation is first performed to map values ​​with different ranges of variation to the same fixed range. Then, the normalized tensor is input into the multi-head self-attention module to model the correlation between each image block in the photovoltaic panel. The attention extraction formula is as follows:

[0058]

[0059] Where Q, K, and V are the query, index, and content matrices, respectively, obtained by linear transformation of the input feature vectors; d k Represents the dimension of the index matrix; outputs the tensor z' l =MSA(LN(z) l-1 ))+z l-1 Where l = 1…L; MSA represents a multi-head self-attention module; LN() represents layer normalization; such as Figure 4 As shown;

[0060] Step S7 involves performing layer normalization on the tensor output from S6, mapping values ​​from different ranges of variation back to the same fixed range. Subsequently, deformable convolutional layers are introduced to replace the original VisionTransformer network's MLP (Multi-Layer Perceptron) layers for image feature extraction.

[0061] The formula for deformable convolution is: Where y(p0) represents a point in the output feature map; p0 represents the coordinates of the center point of the convolution kernel corresponding to this point in the original feature map; p n It is the coordinate of the convolution kernel relative to p0; Δp n Indicates pixel offset; w(p) n ) represents p n The convolution kernel weights corresponding to the points; x(p0+p n +Δp n () represents the offset pixel value; output tensor z l =DC(LN(z′) l ))+z′ l Where l = 1…L, and DC represents deformable convolution operation; such as Figure 4 As shown;

[0062] Step S8 involves performing layer normalization on the tensor output from S7, mapping values ​​from different ranges of variation back to the same fixed range. Then, the classification vector x is extracted. class Corresponding output results We input this into a deformable convolutional classification layer to obtain our final classification result for photovoltaic panel faults. Where DC head represents the deformable convolution class header.

[0063] Step S9: Calculate the deviation between the classification result of S8 and the true label, and use the backpropagation algorithm and stochastic gradient descent method to reduce the overall prediction error to train the model. After multiple iterations of training, the final photovoltaic panel defect classification model is obtained. It usually needs to iterate 300 times on the entire dataset. Training stops when the loss no longer decreases.

[0064] Step S10: Input the photovoltaic panel sample image to be tested into the photovoltaic panel defect classification model obtained in S9, and the output is the defect classification result of the sample to be tested.

[0065] Taking a large photovoltaic panel database as an example, containing 14,708 photovoltaic panel images and corresponding infrared images, the specific steps of the photovoltaic panel surface defect classification method are as follows:

[0066] Step S0: Construct 6-channel input data from 14,708 RGB photovoltaic panel images and their corresponding infrared images in the dataset. Where H and W are the resolutions of the input image, this results in 14708 input data points;

[0067] Step S1: Normalize the photovoltaic panel data in S0 to a uniform size (e.g., 224×224×6 pixels), and then feed the input into the Vision Transformer deformable deep convolutional neural network, which contains several Transformer coding layers and deformable convolutional layers.

[0068] Step S2: The photovoltaic data input in S1 is divided into photovoltaic panel image blocks by a convolutional layer with a kernel size of 16×16 and a stride of 16. The number of image patches obtained is N = H × W / 16 2 Each image patch is mapped to a one-dimensional vector using a linear mapping to obtain... in Let n represent the nth image patch, where n ∈ (1, N), and E represent a linear mapping. Where D represents the image patch that needs to be mapped to D dimensions;

[0069] Step S3: Insert a one-dimensional vector x specifically for photovoltaic panel classification into the multiple one-dimensional vectors obtained in S2. class This vector can be optimized through training; here, it is concatenated with the vector in S2.

[0070] Step S4: Combine the multiple one-dimensional vectors obtained in S3 with the one-dimensional vector E specifically used for photovoltaic panel location encoding.pos Adding them together gives each vector its own positional information. The positional encoding vectors are also trainable parameters, and the output tensor is... in

[0071] Step S5: The tensors from S4 are input into L (L=12) Transformer encoding blocks for feature extraction from the photovoltaic panel. When the tensors are input into each encoding block, a random depth operation is performed, and the tensor has a chance to skip that block. This reduces the dependence of the encoding block on a specific feature of the photovoltaic panel, alleviates the overfitting problem of deep networks, and enhances the network's generalization performance. The formula for random depth is as follows: the survival probability of each encoding block decreases linearly:

[0072]

[0073] Where, p l p represents the survival probability of the l-th layer during training, where L represents the total number of blocks. l It refers to the survival probability of the last residual block.

[0074] Step S6: When the tensor from S5 is input into each encoding block, a layer normalization operation is first performed to map values ​​with different ranges of variation to the same fixed range. Then, the normalized tensor is input into the multi-head self-attention module to model the correlation between each image block in the photovoltaic panel. The attention extraction formula is as follows:

[0075]

[0076] Where Q, K, and V are the query, index, and content matrices, respectively, obtained by linear transformation of the input feature vectors; T represents the matrix transpose; d k Indicates the dimension of the index matrix;

[0077] Output tensor z' l =MSA(LN(z) l-1 ))+z l-1 , where l=1…L; MSA represents multi-head self-attention module, and LN() represents layer normalization;

[0078] Step S7 involves performing layer normalization on the tensor output from S6, mapping values ​​from different ranges of variation back to the same fixed range. Subsequently, a deformable convolutional layer is introduced to replace the original Vision Transformer network's MLP layer for image feature extraction.

[0079] The formula for deformable convolution is: Where y(p0) represents a point in the output feature map, and p0 represents the coordinates of the center point of the convolution kernel at that point in the original feature map. n It is the coordinate of the convolution kernel relative to p0, Δp n w(p) represents the pixel offset. n ) represents p n The convolution kernel weights corresponding to the points, x(p0+p n +Δp n The 'z' represents the offset pixel value. The output tensor z... l =DC(LN(z') l ))+z' l , where l = 1…L, and DC represents deformable convolution operation;

[0080] Step S8 involves performing layer normalization on the tensor output from S7, mapping values ​​from different ranges of variation back to the same fixed range. Then, the classification vector x is extracted. class Corresponding output results We input this into a deformable convolutional classification layer to obtain our final classification result for photovoltaic panel faults. DC head represents the deformable convolution class header.

[0081] Step S9: Calculate the deviation between the classification result of S8 and the true label, and use the backpropagation algorithm and stochastic gradient descent method to reduce the overall prediction error to train the model. After multiple iterations of training, the final photovoltaic panel defect classification model is obtained. It usually needs to iterate 300 times on the entire dataset. Training stops when the loss no longer decreases.

[0082] Step S10: Input the photovoltaic panel sample image to be tested into the photovoltaic panel defect classification model obtained in S9, and the output is the defect classification result of the sample to be tested.

[0083] The photovoltaic panel surface defect classification method provided by this invention first introduces infrared images of the photovoltaic panel as additional input, and synthesizes them with color images into new 6-channel input data; then, it introduces a self-attention mechanism to focus the network attention on information more useful for photovoltaic panel fault classification; finally, it introduces deformable depthwise convolution to better sample information from the photovoltaic panel, enhance the robustness of the network, eliminate useless information from non-photovoltaic panels, and achieve the effect of removing background noise.

[0084] This invention also provides a method for classifying surface defects in photovoltaic panels, based on the above method. The system includes:

[0085] The data preprocessing module is used to construct a composite image from the color image of the photovoltaic panel and its corresponding infrared image as input data.

[0086] The classification module processes the input data using a trained photovoltaic panel defect classification model to obtain the fault classification of the photovoltaic panel; and

[0087] The training module is used to train the photovoltaic panel defect classification model.

[0088] The present invention may also provide a computer device, comprising: at least one processor, a memory, at least one network interface, and a user interface. The various components of the device are coupled together via a bus system. It is understood that the bus system is used to enable communication between these components. In addition to a data bus, the bus system also includes a power bus, a control bus, and a status signal bus.

[0089] The user interface can include a display, keyboard, or clicking device. Examples include a mouse, trackball, touchpad, or touchscreen.

[0090] It is understood that the memory in the embodiments disclosed in this application may be volatile memory or non-volatile memory, or may include both volatile and non-volatile memory. The non-volatile memory may be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. The volatile memory may be random access memory (RAM), which is used as an external cache. By way of example, but not limitation, many forms of RAM are available, such as Static Random Access Memory (SRAM), Dynamic Random Access Memory (DRAM), Synchronous DRAM (SDRAM), Double Data Rate Synchronous DRAM (DDRSDRAM), Enhanced Synchronous DRAM (ESDRAM), Synchlink DRAM (SLDRAM), and Direct Rambus RAM (DRRAM). The memories described herein are intended to include, but are not limited to, these and any other suitable types of memory.

[0091] In some implementations, the memory stores elements such as executable modules or data structures, or subsets thereof, or extended sets thereof: operating systems and applications.

[0092] The operating system includes various system programs, such as the framework layer, core library layer, and driver layer, used to implement various basic business functions and handle hardware-based tasks. The application programs include various applications, such as media players and browsers, used to implement various application functions. Programs implementing the methods of the embodiments of this disclosure can be included in the application programs.

[0093] In the above embodiments, the processor can also invoke programs or instructions stored in memory, specifically programs or instructions stored in an application program, for the following purposes:

[0094] Follow the steps described above.

[0095] The above methods can be applied to or implemented by a processor. The processor may be an integrated circuit chip with signal processing capabilities. During implementation, each step of the above methods can be completed by integrated logic circuits in the processor's hardware or by instructions in software form. The processor can be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. It can implement or execute the disclosed methods, steps, and logic block diagrams. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the disclosed methods can be directly implemented by a hardware decoding processor, or implemented by a combination of hardware and software modules in the decoding processor. The software modules can reside in random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, or other mature storage media in the art. This storage medium is located in memory; the processor reads information from the memory and, in conjunction with its hardware, completes the steps of the above methods.

[0096] It is understood that the embodiments described in this invention can be implemented in hardware, software, firmware, middleware, microcode, or a combination thereof. For hardware implementation, the processing unit can be implemented in one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field-programmable gate arrays (FPGAs), general-purpose processors, controllers, microcontrollers, microprocessors, other electronic units for performing the functions described in this application, or combinations thereof.

[0097] For software implementation, the technology of this invention can be achieved by executing the functional modules (e.g., procedures, functions, etc.) of this invention. The software code can be stored in memory and executed by a processor. The memory can be implemented in the processor or externally.

[0098] The present invention may also provide a non-volatile storage medium for storing a computer program. When the computer program is executed by a processor, it can implement the steps in the above method embodiments.

[0099] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to the embodiments, those skilled in the art should understand that modifications or equivalent substitutions to the technical solutions of the present invention do not depart from the spirit and scope of the technical solutions of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.

Claims

1. A method for classifying surface defects in a photovoltaic panel, the method comprising: A composite image is constructed by combining the color image of the photovoltaic panel with its corresponding infrared image and used as input data. This composite image is then input into a trained photovoltaic panel defect classification model to obtain the fault classification of the photovoltaic panel. The processing steps of the photovoltaic panel defect classification model include: Step 1: The photovoltaic panel defect classification model receives composite images, uses infrared images as guidance, and uses a multi-head attention module and deformable deep convolutional layers as the core to focus the network's attention on the areas that are beneficial to the classification of photovoltaic panel defects, while removing background and useless information. Step 2: Classify the faults of the photovoltaic panel using a deformable convolution classification head; The process of constructing a composite image from infrared and color images of a photovoltaic panel specifically includes: The RGB color image and corresponding infrared image of the photovoltaic panel are constructed into 6-channel input data. ,in, H and W This is the resolution of the input image; The photovoltaic panel defect classification model is a Vision Transformer deformable deep convolutional neural network model, including a Transformer encoding layer and a deformable convolutional layer; Step 1 specifically includes: Step 1-1: Normalize the input data to a uniform size; Step 1-2: Pass the data processed in Step 1-1 through a convolution kernel. P × P Step size is P The convolutional layer is divided into individual photovoltaic panel image blocks. The number of image patches obtained N = Each image patch is mapped to a one-dimensional vector using a linear mapping to obtain... ;in Indicates the first n Image blocks, ; E Represents a linear mapping. ,in D This indicates that the image patch needs to be mapped to D dimension; Steps 1-3: Insert a one-dimensional vector specifically for photovoltaic panel classification into the multiple one-dimensional vectors obtained in step 1-2. This vector is optimized through training; the vector with vector Concatenating them together yields a one-dimensional vector. : ; Steps 1-4, convert the one-dimensional vector Compared with the one-dimensional vector used for photovoltaic panel location encoding The vectors are added together, giving each vector its own positional information; the positional encoding vectors are trainable parameters; the output tensor is... ,in ; C Indicates the number of channels in the input image; Steps 1-5, convert the tensor Input L Feature extraction of photovoltaic panels is performed within Transformer encoding blocks; a random depth operation is performed when tensors are input into each encoding block; the formula for random depth is as follows, and the survival probability of each encoding block decreases linearly: ； in, Indicates the first The survival probability of a layer during training; L Indicates the total number of coded blocks; This indicates the survival probability of the last residual block; Steps 1-6, when the tensors in steps 1-5 When inputting into each encoded block, a layer normalization operation is first performed to map values ​​with different ranges of variation to the same fixed range. Then, the normalized tensor is input into the multi-head self-attention module to model the correlation between each image block in the photovoltaic panel. The attention extraction formula is as follows: ； in, Q , K , V These represent the query, index, and content matrices, respectively, which are obtained from the input feature vectors through linear transformation. T Indicates matrix transpose; Indicates the dimension of the index matrix; Output Tensor ,in ; MSA This indicates a multi-head self-attention module. LN () indicates layer normalization; Steps 1-7, convert the tensor Perform layer normalization to map values ​​from different ranges of variation back to the same fixed range; introduce deformable convolutional layers to replace the original Vision Transformer network. MLP The layer extracts image features; The formula for deformable convolution is: ;in, This represents a single point in the output feature map; Point The coordinates of the center point of the corresponding convolution kernel when it operates in the original feature map; Indicates the convolution kernel relative to The coordinates; Indicates pixel offset; express The kernel weights corresponding to the points; This represents the pixel value after offset; Output Tensor ,in , DC This indicates a deformable convolution operation.

2. The method for classifying surface defects of photovoltaic panels according to claim 1, characterized in that, The training process of the photovoltaic panel defect classification model includes: The deviation between the classification results generated by the photovoltaic panel defect classification model and the calculated true labels is reduced by using the backpropagation algorithm and stochastic gradient descent method to train the model. After multiple iterations of training, the final photovoltaic panel defect classification model is obtained.

3. The method for classifying surface defects of photovoltaic panels according to claim 1, characterized in that, The P The preferred value is 16.

4. The method for classifying surface defects of photovoltaic panels according to claim 1, characterized in that, The L The preferred value is 12.

5. The method for classifying surface defects of photovoltaic panels according to claim 1, characterized in that, Step 2 specifically includes: The tensor output in step 1 Perform layer normalization; extract the classification vectors. Corresponding output results The result is then fed into a deformable convolutional classification layer to obtain the final classification result for photovoltaic panel faults. ,in This indicates the class header for deformable convolution.

6. A photovoltaic panel surface defect classification system, implemented based on any one of the methods described in claims 1-5, characterized in that, The system includes: The data preprocessing module is used to construct a composite image from the color image of the photovoltaic panel and its corresponding infrared image as input data. The classification module processes the input data using a trained photovoltaic panel defect classification model to obtain the fault classification of the photovoltaic panel; and The training module is used to train the photovoltaic panel defect classification model.

Images