A Method and System for Image Defect Recognition of Photovoltaic Modules Based on Imbalanced Samples

Abstract

This invention discloses a method and system for photovoltaic module image defect recognition based on imbalanced samples. The method includes: inputting a first dataset into an improved DCGAN model, the DCGAN model outputting various defect image samples, and analyzing each defect image sample according to a preset image quality evaluation strategy to obtain at least one target defect image sample that meets preset conditions; packaging the at least one target defect image sample and at least one photovoltaic module image to obtain a second dataset, and inputting the second dataset into a preset YOLOv8 model for iterative training to obtain a photovoltaic module image defect recognition model; inputting the acquired real-time photovoltaic module image into the photovoltaic module image defect recognition model, and outputting the defect recognition result corresponding to the real-time photovoltaic module image.

Description

Technical Field

[0001] This invention belongs to the field of target detection technology, and in particular relates to a method and system for identifying defects in photovoltaic module images based on unbalanced samples. Background Technology

[0002] Photovoltaic modules are the core component of the entire solar power generation system, and their quality directly affects power generation performance and safety. After several years of operation, photovoltaic power generation systems may experience aging of the modules, leading to defects such as hot spots, cracks, and broken grids in some cells. Natural factors such as air pollution, obstruction by foreign objects (e.g., bird droppings, small pebbles), and plant shading can also affect the power output of the photovoltaic system and even cause major safety accidents. Therefore, regular inspection of photovoltaic panels for defects is crucial.

[0003] Current research has achieved good detection results on experimental samples with a balanced distribution of specific defects. However, in actual photovoltaic power plants, visible light defects in photovoltaic modules typically exhibit uneven sample distribution. Surface defect detection algorithms still suffer from high computational complexity and long processing times. Imbalanced samples make it difficult to capture and identify small-sample defects. To address the problem of imbalanced sample defect diagnosis, data can be balanced through oversampling or undersampling, or data augmentation methods can be used to increase the number of samples. These methods include image rotation and cropping, noise addition, oversampling, and data generation models to generate new training data, thereby improving the generalization ability of the detection model and avoiding overfitting. Traditional GAN ​​models such as BigGAN and PGGAN can amplify high-quality samples, but they themselves require a certain scale of samples for training, which contradicts the small-sample characteristics of workpiece defects. How to accurately control the details of the generated images and solve the training difficulties of GAN models, as well as the gradient vanishing and model collapse, is a key issue for the widespread application of GANs in the field of data augmentation. Traditional DCGAN can mitigate the problem of imbalanced samples to some extent, but it still has the following problems in generating sample images: 1) The generated images have low resolution, and increasing the number of network layers can easily lead to gradient explosion; 2) It is impossible to accurately control the details of the generated images, while the proportion of defects in photovoltaic module images is small, requiring the generation of detailed images; 3) Data augmentation based on traditional DCGAN can often only generate images with a single style, repetitive and similar, and low signal-to-noise ratio, which is inefficient. Summary of the Invention

[0004] This invention provides a method and system for identifying defects in photovoltaic modules based on unbalanced samples, which solves the technical problem of difficulty in identifying defects in small-sample photovoltaic modules under unbalanced sample conditions.

[0005] In a first aspect, the present invention provides a method for identifying defects in photovoltaic module images based on unbalanced samples, comprising:

[0006] At least one photovoltaic module image is acquired, the at least one photovoltaic module image is preprocessed to obtain target images of photovoltaic modules of the same size, and the target images of photovoltaic modules are packaged into a first dataset;

[0007] The first dataset is input into the improved DCGAN model, which outputs various defect image samples. The DCGAN model then analyzes these defect image samples according to a preset image quality evaluation strategy to obtain at least one target defect image sample that meets the preset conditions.

[0008] The at least one target defect image sample and the at least one photovoltaic module image are packaged to obtain a second dataset, and the second dataset is input into a preset YOLOv8 model for iterative training to obtain a photovoltaic module image defect recognition model;

[0009] The acquired real-time photovoltaic module image is input into the photovoltaic module image defect recognition model, and the photovoltaic module image defect recognition model outputs the defect recognition result corresponding to the real-time photovoltaic module image.

[0010] Secondly, the present invention provides a photovoltaic module image defect recognition system based on unbalanced samples, comprising:

[0011] The preprocessing module is configured to acquire at least one photovoltaic module image, preprocess the at least one photovoltaic module image to obtain target images of photovoltaic modules of the same size, and package the target images of photovoltaic modules into a first dataset;

[0012] The analysis module is configured to input the first dataset into the improved DCGAN model, the DCGAN model outputs various defect image samples, and analyzes the various defect image samples according to a preset image quality evaluation strategy to obtain at least one target defect image sample that meets the preset conditions.

[0013] The training module is configured to package the at least one target defect image sample and the at least one photovoltaic module image to obtain a second dataset, and input the second dataset into a preset YOLOv8 model for iterative training to obtain a photovoltaic module image defect recognition model;

[0014] The output module is configured to input the acquired real-time photovoltaic module image into the photovoltaic module image defect recognition model, and the photovoltaic module image defect recognition model outputs the defect recognition result corresponding to the real-time photovoltaic module image.

[0015] Thirdly, an electronic device is provided, comprising: at least one processor, and a memory communicatively connected to the at least one processor, wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform the steps of the photovoltaic module image defect identification method based on unbalanced samples according to any embodiment of the present invention.

[0016] Fourthly, the present invention also provides a computer-readable storage medium having a computer program stored thereon, wherein when the program instructions are executed by a processor, the processor performs the steps of the photovoltaic module image defect identification method based on unbalanced samples according to any embodiment of the present invention.

[0017] This application presents a photovoltaic module image defect recognition method and system based on imbalanced samples. By improving the DCGAN framework, high-quality defect samples are generated, which specifically alleviates the sample scarcity of local categories (such as dust) in the photovoltaic panel dataset. During the generation process, the improved Wasserstein distance (which assigns dynamic weights to small samples and enhances the contribution of loss) is used to force the generator to focus on the fine features such as edge fragmentation and gray-level skewness of small samples (dust), while retaining the global morphological features of large samples (bird droppings), thus providing a balanced and rich training data foundation for subsequent defect detection. Attached Figure Description

[0018] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the following description of the embodiments will be briefly introduced. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0019] Figure 1 A flowchart illustrating a photovoltaic module image defect recognition method based on unbalanced samples, provided in an embodiment of the present invention;

[0020] Figure 2 A schematic diagram of a DCGAN network according to a specific embodiment of the present invention is provided;

[0021] Figure 3 An improved DCGAN network structure diagram of a specific embodiment of the present invention is provided;

[0022] Figure 4 A residual unit structure diagram of a specific embodiment of the present invention is provided;

[0023] Figure 5 A residual block structure diagram of a specific embodiment of the present invention is provided;

[0024] Figure 6 This is a structural block diagram of a photovoltaic module image defect recognition system based on unbalanced samples, provided in an embodiment of the present invention.

[0025] Figure 7 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention. Detailed Implementation

[0026] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0027] Please see Figure 1 The diagram shows a flowchart of a photovoltaic module image defect recognition method based on unbalanced samples according to this application.

[0028] like Figure 1 As shown, the photovoltaic module image defect recognition method based on unbalanced samples specifically includes the following steps:

[0029] Step S101: Obtain at least one photovoltaic module image, preprocess the at least one photovoltaic module image to obtain target images of each photovoltaic module of the same size, and package the target images of each photovoltaic module into a first dataset.

[0030] In this step, the acquired photovoltaic module images are preprocessed, including cropping, rotating, and flipping, to obtain images of the same size, which are then packaged into a dataset.

[0031] Step S102: Input the first dataset into the improved DCGAN model. The DCGAN model outputs various defect image samples. Analyze the various defect image samples according to a preset image quality evaluation strategy to obtain at least one target defect image sample that meets the preset conditions.

[0032] In this step, the obtained dataset is trained on the improved DCGAN model. The improved DCGAN model is based on the traditional DCGAN. First, fully connected layers are added at two locations: after the DCGAN generator receives random noise and after the discriminator flattens out. Residual blocks are used to replace transposed convolutional and convolutional layers, thus generating higher resolution images without excessively increasing the number of network layers. Second, as... Figure 3As shown, a self-attention mechanism is introduced at four positions in the modified residual structure: between G2 and G3, between G3 and G4, between D2 and D3, and between D3 and D4. This improves the model's ability to capture details of surface defects, helps the model better focus on important regions in the image, and makes it easier for the network to learn complex image features. Finally, the loss function is changed to weighted Wasserstein distance. Traditional Wasserstein distance quantifies the "minimum transport cost" required to transform one probability distribution into another, providing a smoother and more continuous way to optimize the generator and make the generated images more diverse. However, directly using traditional Wasserstein distance will cause small sample detail features to be ignored. Therefore, weighted Wasserstein distance is used to assign dynamic weights to different class sample pairs, enhancing the loss contribution of small samples. The traditional DCGAN model structure diagram is shown below. Figure 2 As shown, the improved DCGAN model structure diagram is as follows: Figure 3 As shown.

[0033] Network structure improvement:

[0034] Traditional DCGAN networks generate 64 images. 64. The size of the generated image is insufficient to meet the requirements of the input image for object detection. Therefore, in order to generate 512... For images with a resolution of 512, two improvements were made to the network structure: first, convolutional layers were added to increase the network depth; second, fully connected layers were added to enlarge the input size, thereby increasing the output size of all subsequent layers. To avoid performance degradation due to increased network layers, fully connected layers were added, and their outputs were then reshaped into tensors with higher dimensions. Adding fully connected layers and reshaping the tensors increases the network's non-linear expressive power, making it easier for the network to learn complex image features. Furthermore, by increasing the size of the input tensor, higher-resolution images can be generated without excessively increasing the number of network layers, thus maintaining an efficient learning process.

[0035] In addition, using residual units in a network can help information propagate faster, helping to solve the problems of vanishing and exploding gradients, while also improving the network's nonlinear expressive power. Residual units pass parameters by introducing skip connections, thereby enabling information transfer between network layers. The structure of a residual unit is as follows: Figure 4 As shown, input After a series of convolutions and activation functions, we obtain... Then, directly connect via jump link With output Add them together to get the final input. This operation allows the model to learn the difference between the input and output, i.e., the residual, more effectively. The activation function used is ReLU, which is computationally simpler and more direct, helping to reduce overfitting and avoid gradient vanishing. This design allows deep networks to propagate gradients more effectively during training, improving training stability and enabling deeper network structures.

[0036] Adjusting the network depth and adding residual units improves the resolution of the generated images while maintaining model performance. Residual blocks added to the generator and discriminator are as follows: Figure 5 As shown.

[0037] In the generator, the input random noise first passes through a fully connected layer, and then the output of the fully connected layer enters the AdaIN layer to inject defect category labels and adjust the distribution of high-dimensional features. The expression is as follows: In the formula, This is the output of the AdaIN layer. For the output of the fully connected layer, For category labels, The mean of the features, The standard deviation of the features is used to reshape the output of the AdaIN layer to 16. The 16-feature map, after four layers of upsampling operations, gradually increases in size, ultimately generating a resolution of 512. A 512 resolution image. The input to the discriminator is 512. The 512 image is processed through four layers of downsampling convolution to gradually reduce the resolution to 16. 16. Finally, through flattening operations and fully connected layers, a final score is output to determine the authenticity of the image. After this improvement, the original six upsampling layers required to generate 512... The network structure for 512-resolution images now requires only four layers, effectively reducing the network depth, which helps to reduce model complexity and may improve training efficiency.

[0038] The generator and discriminator include a dual attention mechanism, specifically:

[0039] The channel branch first undergoes global average pooling, expressed as:

[0040] ,

[0041] In the formula, The mean of c channels for each sample. For the feature space size, Let represent the feature value of the b-th sample, the c-th channel, the i-th row, and the j-th column.

[0042] The output is processed through a fully connected layer to generate channel weights, expressed as follows:

[0043] ,

[0044] In the formula, For the channel corresponding weight, It is the Sigmoid activation function. It is the ReLU activation function. , The weights and biases for the first-level fully connected layer. , The weights and biases for the second-level fully connected layer;

[0045] The spatial branch line undergoes deformable convolution. By learning the kernel offset, the kernel deformation covers the defect region. The expression is:

[0046] ,

[0047] In the formula, This represents the current feature map position. The kernel size is [size]. Set a fixed offset for the convolution kernel. For learnable offsets, For convolution kernel weights, The output feature at position p;

[0048] The output undergoes multi-scale fusion, adaptive Top-K filtering, and spatial weighting graph calculation. Key positions are dynamically retained based on the input. The expression is:

[0049] ,

[0050]

[0051] In the formula, For multi-scale fusion output, This is the original spatial weight map. It is the Sigmoid activation function. The base retention ratio is set to 0.3. The dust sample compensation coefficient is set to 0.3. The proportion of dust in the input image. For selection Return the indices of the K positions with the highest median values. ;

[0052] Self-attention is calculated for the Top-K positions using sparse attention computation, expressed as follows:

[0053] ,

[0054] In the formula, These are the query matrix, key matrix, and value matrix, respectively. For feature dimension, , and From , and The subset selected from the middle The symbol for the transpose matrix is ​​. For normalization function, This is an expression of the self-attention mechanism;

[0055] The expression for fusing dual-path features through channel weighting is:

[0056] ,

[0057] In the formula, As a feature of fusion, This is a sparse attention feature. Channel weight;

[0058] The ability to detect defect locations is enhanced by relative position encoding, as expressed in the following expression:

[0059] ,

[0060] In the formula, For the final output features, It is a learnable relative position encoding function for the relative coordinates of the feature map location and the reference location.

[0061] Loss function improvement

[0062] The expression for the total loss function of the improved DCGAN model is as follows:

[0063] ,

[0064] In the formula, To counteract the loss of weight, For class loss weights, For texture loss weights, For position loss weights, To combat the losses, For category loss, For texture loss, For position loss;

[0065] The expression for calculating adversarial loss is:

[0066] ,

[0067] ,

[0068] ,

[0069] In the formula, The distribution of real photovoltaic defects, For the distribution of random noise, For the distribution of defect category labels, For generator, For discriminator, For expectation calculation;

[0070] The expression for calculating the class loss is:

[0071] ,

[0072] In the formula, One-hot encoding for category labels, The output of the discriminator's category classification head;

[0073] The expression for calculating texture loss is:

[0074] ,

[0075] In the formula, The probability distribution of texture features. The dimension of the texture feature;

[0076] The expression for calculating the position loss is:

[0077] ,

[0078] In the formula, Let x be the relative position coordinates of the defect region in image x. For relative position encoding functions, It is an L1 norm.

[0079] The improved DCGAN network generates images with a resolution reduced from 64. Size increased from 64 to 512 The size of 512 leads to training instability, causing the generator to output highly similar images, which greatly reduces output diversity. The traditional Wasserstein distance has been shown to significantly improve training stability and output quality, but directly calculating the distribution difference ignores the detailed features of small samples (dust), causing the generated samples to be biased towards the large sample pattern. To solve these problems, in the improved Wasserstein distance calculation, dynamic weights are assigned to different class sample pairs, focusing on enhancing the loss contribution of small samples. The traditional Wasserstein distance formula is shown in Equation (8):

[0080] ,

[0081] The improved Wasserstein distance formula is shown in equation (9):

[0082] ,

[0083] Where, For the improved Wasserstein distance, The number of real photovoltaic defect images. The number of photovoltaic defect images generated by the generator. The matching weights between real samples and generated samples. These are real photovoltaic defect images. Images of photovoltaic defects generated by the generator. These are the weighting coefficients. To minimize handling costs among all possible transportation plans, To satisfy the joint distribution set of marginal distributions, For transportation planning.

[0084] Where y belongs to a small sample, then =1.5, otherwise =1, by increasing the cost weight of dust sample pairs, the generator is forced to pay more attention to small sample details; =0.8, adjusting class weights The overall strength of the sample size is controlled to manage the magnitude of the penalty for small samples and avoid overfitting.

[0085] It should be noted that the high-quality defect samples generated based on the improved DCGAN framework effectively alleviate the problem of local category sample scarcity in photovoltaic panel datasets. However, the feasibility of deploying the generated images in actual industrial scenarios still needs to be systematically verified through a multi-dimensional evaluation system to ensure that it meets the reliability requirements of photovoltaic operation and maintenance systems. To evaluate whether the defects in the generated images conform to the physical and morphological characteristics of real defects, a photovoltaic defect generated image quality assessment method based on multi-modal feature fusion is adopted. This method addresses the problems of sample imbalance and insufficient physical characteristic matching of existing image quality assessment indicators (such as SSIM and PSNR) in photovoltaic defect detection scenarios. It achieves high-precision screening of defect generated images by constructing a multi-scale morphological template library, introducing an electroluminescence (EL) physical characteristic verification mechanism, and a dynamic weight allocation strategy. Specifically, it includes:

[0086] The complexity of defect edges in each defect image sample is quantified using box counting to obtain the fractal dimension, expressed as:

[0087] ,

[0088] In the formula, The number of boxes with side length s required to cover the edge of the defect;

[0089] The KL divergence was used to verify the gray-level distribution matching degree in each defect image sample. The expression for calculating the KL divergence is as follows:

[0090] ,

[0091] In the formula, To generate the defect grayscale distribution, For the true EL distribution, This represents the KL divergence between the defect grayscale distribution and the true EL distribution.

[0092] We strengthen edge structure similarity constraints for bird dropping defects and score the texture diversity of dust defects based on a weighted linear fusion scoring function to obtain a comprehensive score result. The expression is:

[0093] ,

[0094] In the formula, The global maximum value of the fractal dimension D. The global minimum of the fractal dimension D. The global maximum value of the KL divergence. The global minimum of the KL divergence. , , All of these are hyperparameters, and their values ​​were manually tuned. It is a dynamic weighting function, with values ​​manually tuned. For fractal dimensions, Let KL divergence be a metric.

[0095] Using the comprehensive scoring results And set a threshold T, if If the result is greater than or equal to T, it is retained and added to the training set; otherwise, it is discarded and the iteration is restarted. This method effectively solves the problems of semantic feature loss and deviation from physical laws in traditional evaluation indicators in photovoltaic scenarios, and significantly improves the generalization ability of the defect detection model.

[0096] Step S103: Pack the at least one target defect image sample and the at least one photovoltaic module image into a second dataset, and input the second dataset into a preset YOLOv8 model for iterative training to obtain a photovoltaic module image defect recognition model.

[0097] Step S104: The acquired real-time photovoltaic module image is input into the photovoltaic module image defect recognition model, and the photovoltaic module image defect recognition model outputs the defect recognition result corresponding to the real-time photovoltaic module image.

[0098] In summary, the method in this application generates high-quality defect samples by improving the DCGAN framework, specifically alleviating the sample scarcity of local categories (such as dust) in photovoltaic panel datasets. During the generation process, an improved Wasserstein distance is used (to assign dynamic weights to small samples and enhance their contribution to the loss), forcing the generator to focus on fine features such as edge fragmentation and gray-level skewness of small samples (dust), while retaining the global morphological features of large samples (bird droppings), providing a balanced and rich training data foundation for subsequent defect detection. Addressing the shortcomings of traditional image quality metrics (SSIM, PSNR) in photovoltaic scenarios, which suffer from "missing semantic features and deviation from physical laws," this application innovatively proposes a multimodal feature fusion-based generated image quality assessment method, achieving three major technological breakthroughs: accurate morphological matching, compliance verification of physical characteristics, and category-aware weight adjustment.

[0099] Please see Figure 6 The diagram shows a structural block diagram of a photovoltaic module image defect recognition system based on unbalanced samples according to this application.

[0100] like Figure 6 As shown, the photovoltaic module image defect recognition system 200 includes a preprocessing module 210, an analysis module 220, a training module 230, and an output module 240.

[0101] The preprocessing module 210 is configured to acquire at least one photovoltaic module image, preprocess the at least one photovoltaic module image to obtain target images of photovoltaic modules of the same size, and package the target images of photovoltaic modules into a first dataset; the analysis module 220 is configured to input the first dataset into an improved DCGAN model, the DCGAN model outputs various defect image samples, and analyzes the various defect image samples according to a preset image quality evaluation strategy to obtain at least one target defect image sample that meets preset conditions; the training module 230 is configured to package the at least one target defect image sample and the at least one photovoltaic module image to obtain a second dataset, and input the second dataset into a preset YOLOv8 model for iterative training to obtain a photovoltaic module image defect recognition model; the output module 240 is configured to input the acquired real-time photovoltaic module image into the photovoltaic module image defect recognition model, the photovoltaic module image defect recognition model outputs a defect recognition result corresponding to the real-time photovoltaic module image.

[0102] It should be understood that Figure 6The modules and references described in the document Figure 1 The steps described in the text correspond to those in the method described above. Therefore, the operations, features, and corresponding technical effects described above also apply to the method described in the text. Figure 6 The various modules in the document will not be described in detail here.

[0103] In other embodiments, the present invention also provides a computer-readable storage medium having a computer program stored thereon, wherein when the program instructions are executed by a processor, the processor performs the photovoltaic module image defect recognition method based on unbalanced samples in any of the above method embodiments.

[0104] In one embodiment, the computer-readable storage medium of the present invention stores computer-executable instructions, which are configured as follows:

[0105] At least one photovoltaic module image is acquired, the at least one photovoltaic module image is preprocessed to obtain target images of photovoltaic modules of the same size, and the target images of photovoltaic modules are packaged into a first dataset;

[0106] The first dataset is input into the improved DCGAN model, which outputs various defect image samples. The DCGAN model then analyzes these defect image samples according to a preset image quality evaluation strategy to obtain at least one target defect image sample that meets the preset conditions.

[0107] The at least one target defect image sample and the at least one photovoltaic module image are packaged to obtain a second dataset, and the second dataset is input into a preset YOLOv8 model for iterative training to obtain a photovoltaic module image defect recognition model;

[0108] The acquired real-time photovoltaic module image is input into the photovoltaic module image defect recognition model, and the photovoltaic module image defect recognition model outputs the defect recognition result corresponding to the real-time photovoltaic module image.

[0109] Computer-readable storage media may include a stored program area and a stored data area, wherein the stored program area may store an operating system and an application program required for at least one function; the stored data area may store data created based on the use of the unbalanced sample-based photovoltaic module image defect recognition system, etc. Furthermore, the computer-readable storage medium may include high-speed random access memory, and may also include memory, such as at least one disk storage device, flash memory device, or other non-volatile solid-state storage device. In some embodiments, the computer-readable storage medium may optionally include memory remotely configured relative to a processor, which can be connected to the unbalanced sample-based photovoltaic module image defect recognition system via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0110] Figure 7 This is a schematic diagram of the structure of the electronic device provided in the embodiment of the present invention, such as... Figure 7 As shown, the device includes a processor 310 and a memory 320. The electronic device may also include an input device 330 and an output device 340. The processor 310, memory 320, input device 330, and output device 340 can be connected via a bus or other means. Figure 7 Taking a bus connection as an example, the memory 320 is the computer-readable storage medium described above. The processor 310 executes various server functions and data processing by running non-volatile software programs, instructions, and modules stored in the memory 320, thereby implementing the photovoltaic module image defect recognition method based on unbalanced samples described in the above embodiment. The input device 330 can receive input digital or character information and generate key signal inputs related to user settings and function control of the photovoltaic module image defect recognition system based on unbalanced samples. The output device 340 may include a display screen or other display device.

[0111] The aforementioned electronic device can execute the method provided in the embodiments of the present invention, and has the corresponding functional modules and beneficial effects for executing the method. Technical details not described in detail in this embodiment can be found in the method provided in the embodiments of the present invention.

[0112] In one implementation, the above-described electronic device is applied to a photovoltaic module image defect recognition system based on unbalanced samples, for a client, and includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to:

[0113] At least one photovoltaic module image is acquired, the at least one photovoltaic module image is preprocessed to obtain target images of photovoltaic modules of the same size, and the target images of photovoltaic modules are packaged into a first dataset;

[0114] The first dataset is input into the improved DCGAN model, which outputs various defect image samples. The DCGAN model then analyzes these defect image samples according to a preset image quality evaluation strategy to obtain at least one target defect image sample that meets the preset conditions.

[0115] The at least one target defect image sample and the at least one photovoltaic module image are packaged to obtain a second dataset, and the second dataset is input into a preset YOLOv8 model for iterative training to obtain a photovoltaic module image defect recognition model;

[0116] The acquired real-time photovoltaic module image is input into the photovoltaic module image defect recognition model, and the photovoltaic module image defect recognition model outputs the defect recognition result corresponding to the real-time photovoltaic module image.

[0117] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., including several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods of various embodiments or some parts of embodiments.

[0118] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for identifying defects in photovoltaic module images based on unbalanced samples, characterized in that, include: At least one photovoltaic module image is acquired, the at least one photovoltaic module image is preprocessed to obtain target images of photovoltaic modules of the same size, and the target images of photovoltaic modules are packaged into a first dataset; The first dataset is input into the improved DCGAN model, which outputs various defect image samples. These defect image samples are then analyzed according to a preset image quality evaluation strategy to obtain at least one target defect image sample that meets preset conditions. The improved DCGAN model includes a generator and a discriminator. The generator specifically includes the following steps: Random noise is used as the initial input to the generator. The random noise first enters the fully connected layer, which maps the noise vector to a feature space. The output of the fully connected layer is fed into the AdaIN layer, where defect category labels are injected to adjust the distribution of high-dimensional features. The expression is as follows: ， In the formula, This is the output of the AdaIN layer. For the output of the fully connected layer, For category labels, The mean of the features, The standard deviation of the feature; The output of the AdaIN layer undergoes a reshape operation, which reshapes the feature vector into a tensor shape suitable for subsequent convolution operations. Enter the batch normalization layer to perform normalization processing on the data; The output of the batch normalization layer passes sequentially through the ReLU activation function layer, the first residual block, the second residual block, the first self-attention layer, the third residual block, the second self-attention layer, and the fourth residual block; The output of the fourth residual block enters the transposed convolutional layer to upsample the feature map, and the output of the transposed convolutional layer passes through the Tanh activation function layer to map the pixel values ​​to the range of [-1, 1] to generate the final image data; The discriminator specifically includes the following steps: Image data enters the discriminator, passes through the convolutional layer, and features are extracted from the image data; The output of the convolutional layer passes sequentially through the Leaky ReLU activation function layer, the fifth residual block, the sixth residual block, the first self-attention layer, the seventh residual block, the second self-attention layer, and the eighth residual block; The output of the eighth residual block passes through the Flatten layer, which flattens the multidimensional feature map into a one-dimensional vector. The flattened one-dimensional vector enters the fully connected layer and outputs a scalar value, representing the discriminator's confidence that the input image is a real image; The at least one target defect image sample and the at least one photovoltaic module image are packaged to obtain a second dataset, and the second dataset is input into a preset YOLOv8 model for iterative training to obtain a photovoltaic module image defect recognition model; The acquired real-time photovoltaic module image is input into the photovoltaic module image defect recognition model, and the photovoltaic module image defect recognition model outputs the defect recognition result corresponding to the real-time photovoltaic module image.

2. The photovoltaic module image defect recognition method based on unbalanced samples according to claim 1, characterized in that, The generator and discriminator include a dual attention mechanism, specifically: The channel branch first undergoes global average pooling, expressed as: ， In the formula, The mean of c channels for each sample. For the feature space size, Let represent the feature value of the b-th sample, the c-th channel, the i-th row, and the j-th column; The output is processed through a fully connected layer to generate channel weights, expressed as follows: ， In the formula, For the channel corresponding weight, It is the Sigmoid activation function. It is the ReLU activation function. , The weights and biases for the first-level fully connected layer. , The weights and biases for the second-level fully connected layer; The spatial branch line undergoes deformable convolution. By learning the kernel offset, the kernel deformation covers the defect region. The expression is: ， In the formula, This represents the current feature map position. The kernel size is [size]. Set a fixed offset for the convolution kernel. For learnable offsets, For convolution kernel weights, The output feature at position p; The output undergoes multi-scale fusion, adaptive Top-K filtering, and spatial weighting graph calculation. Key positions are dynamically retained based on the input. The expression is: ， ， In the formula, For multi-scale fusion output, This is the original spatial weight map. It is the Sigmoid activation function. The base retention ratio is set to 0.

3. The dust sample compensation coefficient is set to 0.

3. The proportion of dust in the input image. For selection Return the indices of the K positions with the highest median values. ; Self-attention is calculated for the Top-K positions using sparse attention computation, expressed as follows: ， In the formula, These are the query matrix, key matrix, and value matrix, respectively. For feature dimension, , and From , and The subset selected from the middle The symbol for the transpose matrix is ​​. For normalization function, This is an expression of the self-attention mechanism; The expression for fusing dual-path features through channel weighting is: ， In the formula, As a feature of fusion, This is a sparse attention feature. Channel weight; The ability to detect defect locations is enhanced by relative position encoding, as expressed in the following expression: ， In the formula, For the final output features, It is a learnable relative position encoding function for the relative coordinates of the feature map location and the reference location.

3. The photovoltaic module image defect recognition method based on unbalanced samples according to claim 1, characterized in that, The expression for the total loss function of the improved DCGAN model is as follows: ， In the formula, To counteract the loss of weight, For class loss weights, For texture loss weights, For position loss weights, To combat the losses, For category loss, For texture loss, For position loss; The expression for calculating adversarial loss is: ， ， ， In the formula, The distribution of real photovoltaic defects, For the distribution of random noise, For the distribution of defect category labels, For generator, For discriminator, For expectation calculation; The expression for calculating the class loss is: ， In the formula, One-hot encoding for category labels, The output of the discriminator's category classification head; The expression for calculating texture loss is: ， In the formula, The probability distribution of texture features. The dimension of the texture feature; The expression for calculating the position loss is: ， In the formula, Let x be the relative position coordinates of the defect region in image x. For relative position encoding functions, It is an L1 norm.

4. The photovoltaic module image defect recognition method based on unbalanced samples according to claim 1, characterized in that, The expression for the improved Wasserstein distance in the generator is as follows: ， In the formula, For the improved Wasserstein distance, The number of real photovoltaic defect images. The number of photovoltaic defect images generated by the generator. The matching weights between real samples and generated samples. These are real photovoltaic defect images. Images of photovoltaic defects generated by the generator. These are the weighting coefficients. To minimize handling costs among all possible transportation plans, To satisfy the joint distribution set of marginal distributions, For transportation planning.

5. The photovoltaic module image defect recognition method based on unbalanced samples according to claim 1, characterized in that, The step of analyzing each defective image sample according to a preset image quality evaluation strategy to obtain at least one target defective image sample that meets preset conditions includes: The complexity of defect edges in each defect image sample is quantified using box counting to obtain the fractal dimension, expressed as: ， In the formula, The number of boxes with side length s required to cover the edge of the defect; The KL divergence was used to verify the gray-level distribution matching degree in each defect image sample. The expression for calculating the KL divergence is as follows: ， In the formula, To generate the defect grayscale distribution, For the true EL distribution, The KL divergence between the defect grayscale distribution and the true EL distribution; We strengthen edge structure similarity constraints for bird dropping defects and score the texture diversity of dust defects based on a weighted linear fusion scoring function to obtain a comprehensive score result. The expression is: ， In the formula, The global maximum value of the fractal dimension D. The global minimum of the fractal dimension D. The global maximum value of the KL divergence. The global minimum of the KL divergence. , , All of these are hyperparameters, and their values ​​were manually tuned. It is a dynamic weighting function, with values ​​manually tuned. For fractal dimensions, Let KL divergence be a metric. Using the comprehensive scoring results And set a threshold T, if If the value is greater than or equal to T, then the value is retained and added to the training set; otherwise, it is discarded and the iteration is restarted.

6. A photovoltaic module image defect recognition system based on unbalanced samples, characterized in that, include: The preprocessing module is configured to acquire at least one photovoltaic module image, preprocess the at least one photovoltaic module image to obtain target images of photovoltaic modules of the same size, and package the target images of photovoltaic modules into a first dataset; The analysis module is configured to input the first dataset into an improved DCGAN model, the DCGAN model outputs various defect image samples, and analyzes the various defect image samples according to a preset image quality evaluation strategy to obtain at least one target defect image sample that meets preset conditions. The improved DCGAN model includes a generator and a discriminator, and the generator specifically includes the following steps: Random noise is used as the initial input to the generator. The random noise first enters the fully connected layer, which maps the noise vector to a feature space. The output of the fully connected layer is fed into the AdaIN layer, where defect category labels are injected to adjust the distribution of high-dimensional features. The expression is as follows: ， In the formula, This is the output of the AdaIN layer. For the output of the fully connected layer, For category labels, The mean of the features, The standard deviation of the feature; The output of the AdaIN layer undergoes a reshape operation, which reshapes the feature vector into a tensor shape suitable for subsequent convolution operations. Enter the batch normalization layer to perform normalization processing on the data; The output of the batch normalization layer passes sequentially through the ReLU activation function layer, the first residual block, the second residual block, the first self-attention layer, the third residual block, the second self-attention layer, and the fourth residual block; The output of the fourth residual block enters the transposed convolutional layer to upsample the feature map, and the output of the transposed convolutional layer passes through the Tanh activation function layer to map the pixel values ​​to the range of [-1, 1] to generate the final image data; The discriminator specifically includes the following steps: Image data enters the discriminator, passes through the convolutional layer, and features are extracted from the image data; The output of the convolutional layer passes sequentially through the Leaky ReLU activation function layer, the fifth residual block, the sixth residual block, the first self-attention layer, the seventh residual block, the second self-attention layer, and the eighth residual block; The output of the eighth residual block passes through the Flatten layer, which flattens the multidimensional feature map into a one-dimensional vector. The flattened one-dimensional vector enters the fully connected layer and outputs a scalar value, representing the discriminator's confidence that the input image is a real image; The training module is configured to package the at least one target defect image sample and the at least one photovoltaic module image to obtain a second dataset, and input the second dataset into a preset YOLOv8 model for iterative training to obtain a photovoltaic module image defect recognition model; The output module is configured to input the acquired real-time photovoltaic module image into the photovoltaic module image defect recognition model, and the photovoltaic module image defect recognition model outputs the defect recognition result corresponding to the real-time photovoltaic module image.

7. An electronic device, characterized in that, include: At least one processor, and a memory communicatively connected to the at least one processor, wherein the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 5.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the method described in any one of claims 1 to 5.

Images