Film tape missing detection method, device and equipment of photovoltaic module and storage medium

Abstract

The invention provides a film tape missing detection method, device and equipment of a photovoltaic module and a storage medium. According to the method, sample pictures of the photovoltaic module are obtained firstly, the sample pictures comprise a plurality of defect sample pictures and a plurality of qualified sample pictures, the defect sample pictures comprise first segmentation pictures corresponding to the positions, where film tape missing exists, of the photovoltaic module, and the qualified sample pictures comprise second segmentation pictures corresponding to the positions, where film tape missing does not exist, of the photovoltaic module. And inputting the sample picture into the initial detection network, and training the initial detection network to obtain a target detection network. And finally, inputting the product image of the to-be-detected product into the target detection network, and detecting whether the to-be-detected product has the film belt missing or not to obtain a target detection result. According to the invention, the efficiency and the accuracy of detecting the film tape deficiency of the photovoltaic module can be improved.

Description

Technical Field

[0001] The present invention relates to the technical field of product detection, and in particular to a method, device, computer equipment and readable storage medium for detecting film tape missing of a photovoltaic module. Background Art

[0002] In the industrial fields of photovoltaic module production, battery production, thin film manufacturing and electronic component packaging, film tape missing detection is mainly achieved through manual inspection. Manual inspection relies on the operator's vision and experience to identify the missing area through manual visual inspection of the film tape image. In order to improve the efficiency and accuracy of inspection, rule-based image processing technology has been increasingly used for automatic inspection in recent years.

[0003] However, manual inspection is difficult to maintain consistency and reliability because it relies on human visual judgment, and as the speed of the production line increases, the inspection efficiency is difficult to meet actual needs. Although rule-based image processing methods can achieve a certain degree of automation, they are essentially based on preset feature extraction rules and are difficult to cope with complex and changeable actual working conditions. For example, when there are scratches, stains or other interferences on the surface of the film tape, these methods are prone to false detection or missed detection. Summary of the invention

[0004] The present invention provides a method, an apparatus, a computer device and a readable storage medium for detecting film tape missing of a photovoltaic module, which can improve the efficiency and accuracy of detecting film tape missing of a photovoltaic module.

[0005] In a first aspect, the present invention provides a method for detecting film tape missing of a photovoltaic module, the method comprising:

[0006] Acquire sample images of photovoltaic modules; the sample images include a plurality of defective sample images and a plurality of qualified sample images, the defective sample images include a first cut image corresponding to a position where a film strip of the photovoltaic module is missing, and the qualified sample images include a second cut image corresponding to a position where a film strip of the photovoltaic module is not missing;

[0007] Inputting the sample image into an initial detection network, and training the initial detection network to obtain a target detection network;

[0008] The product image of the product to be detected is input into the target detection network, and the product to be detected is detected to see whether there is a missing film strip, so as to obtain a target detection result.

[0009] Optionally, the initial detection network includes a feature processing layer, and the feature processing layer includes a feature extraction layer and a feature splicing layer;

[0010] The training of the initial detection network to obtain a target detection network includes:

[0011] Inputting the sample image into the feature extraction layer, performing multi-scale feature extraction on the sample image through the feature extraction layer, and outputting multiple features of the sample image;

[0012] Input multiple features of the sample image into the feature splicing layer to obtain a first splicing feature output by the feature splicing layer.

[0013] Optionally, the feature extraction layer includes a plurality of convolution kernels of different sizes;

[0014] Performing multi-scale feature extraction on the sample image and outputting multiple features of the sample image includes:

[0015] The sample image is input into a plurality of the convolution kernels, each of the convolution kernels outputs a feature of a corresponding scale, and features of a plurality of scales of the sample image are obtained.

[0016] Optionally, the initial detection network further includes an embedded coding layer, and the embedded coding layer includes an embedding layer, a coding layer and a de-embedding layer;

[0017] The training of the initial detection network to obtain a target detection network includes:

[0018] Inputting the first splicing feature into the embedding layer, mapping the first splicing feature into a low-dimensional vector through the embedding layer, and obtaining an embedded first splicing feature;

[0019] Inputting the embedded first splicing feature into the encoding layer, performing multi-layer extraction on the embedded first splicing feature, and outputting a plurality of encoding feature vectors with richer feature information;

[0020] The encoded feature vector is input into the de-embedding layer, the low-dimensional encoded feature vector is mapped and concatenated into a high-dimensional vector, and a second concatenated feature is output.

[0021] Optionally, the initial detection network further includes a first fully connected layer, a second fully connected layer and an activation function layer connected in sequence, each of the first fully connected layers includes a plurality of first fully connected layers, each of the second fully connected layers includes a plurality of second fully connected layers, and the number of the first fully connected layers is equal to the number of the second fully connected layers and both are odd numbers;

[0022] The training of the initial detection network to obtain a target detection network includes:

[0023] The second splicing feature is input into the first fully connected layer, and after being processed by the first fully connected layer, the second fully connected layer and the activation function layer, a probability distribution of whether the sample image has defects or does not have defects is output.

[0024] Optionally, the initial detection network further includes a voting layer;

[0025] The training of the initial detection network to obtain a target detection network includes:

[0026] Input each of the probability distributions into the voting layer to analyze the probability distributions and obtain a voting result of whether the product to be tested has a missing membrane strip;

[0027] According to the voting result, a detection result of the initial detection network performing film tape missing detection on the photovoltaic module is determined.

[0028] Optionally, the step of inputting the product image of the product to be detected into the target detection network, detecting the missing film band of the product to be detected, and obtaining the target detection result includes:

[0029] Outputting a first detection result of whether the product to be detected has a missing film band through the target detection network, and outputting a second detection result when the product to be detected has a missing film band;

[0030] The second detection result includes the defect mark and position of the missing film tape of the product to be detected.

[0031] In a second aspect, the present invention further provides a device for detecting film tape missing of a photovoltaic module, comprising:

[0032] A data acquisition module, used to acquire sample images of photovoltaic modules; the sample images include a plurality of defective sample images and a plurality of qualified sample images, the defective sample images include a first cut image corresponding to a position where a film strip of the photovoltaic module is missing, and the qualified sample images include a second cut image corresponding to a position where a film strip of the photovoltaic module is not missing;

[0033] A model training module, inputting the sample image into an initial detection network, and training the initial detection network to obtain a target detection network;

[0034] The defect detection module is used to input the product image of the product to be detected into the target detection network, detect whether there is any film strip missing on the product to be detected, and obtain the target detection result.

[0035] In a third aspect, the present invention also provides a computer device, comprising a processor and a memory; the memory is used to store a computer program; the processor is used to execute the computer program stored in the memory to implement the steps of the method for detecting missing film strips of photovoltaic modules as described above.

[0036] In a fourth aspect, the present invention further provides a computer-readable storage medium storing a computer program, wherein the computer program is loaded by a processor to perform the steps of the method for detecting missing film strips of photovoltaic modules as described above.

[0037] In the present invention, the defective sample images and qualified sample images of photovoltaic modules are used to train the initial detection network to obtain the target detection network. The target detection network is used to detect the missing film strips of photovoltaic modules, and the multi-level features of the film strip images are automatically learned and extracted efficiently, and the key areas in the film strip images are focused on, which effectively enhances the perception of the subtle missing film strip areas, reduces missed detection and false detection, and thus can improve the accuracy and efficiency of the missing film strip detection of photovoltaic modules. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative work.

[0039] Figure 1 is a schematic flow chart of a method for detecting missing film strips of a photovoltaic module provided in an embodiment of the present invention;

[0040] Figure 2 is a schematic diagram of providing defect detection through an initial detection network in an embodiment of the present invention;

[0041] Figure 3 is a schematic diagram of defect labeling by a target detection network provided in an embodiment of the present invention;

[0042] Figure 4 It is a schematic diagram of a functional module for detecting film strip missing of a photovoltaic module provided in an embodiment of the present invention;

[0043] Figure 5 It is a schematic diagram of the structure of a computer device provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0044] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present invention.

[0045] In the description of the present invention, it should be understood that the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of the features. In the description of the present invention, the meaning of "plurality" is two or more, unless otherwise clearly and specifically defined.

[0046] In the present invention, the word "exemplary" is used to mean "used as an example, illustration or description". Any embodiment described as "exemplary" in the present invention is not necessarily to be construed as being more preferred or advantageous than other embodiments. At the same time, it is to be understood that in the specific implementation of the present invention, when the above embodiments of the present invention are applied to specific products or technologies, user permission or consent is required, and the collection, use and processing of relevant data shall comply with relevant laws, regulations and standards of relevant countries and regions.

[0047] In industrial fields such as photovoltaic module production, battery production, thin film manufacturing and electronic component packaging, film tape missing detection is an important link in quality control. There are two main types of existing film tape missing detection methods: manual detection and rule-based image processing technology. Manual detection relies on the operator's vision and experience, and identifies the missing area by manually visually inspecting the film tape image, but this method is inefficient, subjective, and easily affected by operator fatigue and distraction. In order to improve detection efficiency and accuracy, rule-based image processing technology has been increasingly used for automatic detection in recent years. Such methods usually extract the edge features of the film tape through image preprocessing, edge detection, morphological analysis and other means to identify the missing area. These methods can achieve certain detection effects under the conditions of uniform illumination and simple background, and have been applied in some industrial production lines.

[0048] However, existing detection methods have obvious shortcomings when faced with complex backgrounds, uneven lighting, and slight film strip missing. Manual detection is difficult to maintain consistency and reliability because it relies on human visual judgment, and as the speed of the production line increases, the detection efficiency is difficult to meet actual needs. Although rule-based image processing methods can achieve a certain degree of automation, they are essentially based on preset feature extraction rules and are difficult to cope with complex and changeable actual working conditions. For example, when there are scratches, stains or other interferences on the surface of the film strip, these methods are prone to false detection or missed detection. In addition, rule-based image processing methods show poor adaptability when dealing with missing areas of different sizes and shapes, and it is difficult to accurately model different types of film strip missing. Especially for subtle missing areas, these methods are difficult to effectively detect under conditions of high noise and complex backgrounds, resulting in low overall detection accuracy and stability.

[0049] Therefore, the present invention can provide a more intelligent and accurate detection method, which can accurately identify various missing conditions of the membrane band under complex backgrounds and improve the reliability and robustness of the detection.

[0050] The present invention provides a method, device, computer equipment and readable storage medium for detecting film tape missing of a photovoltaic module, which are described in detail below.

[0051] like Figure 1 As shown, Figure 1 FIG. 1 is a flow chart of a method for detecting missing film strips of a photovoltaic module according to an embodiment of the present invention. The method for detecting missing film strips of a photovoltaic module is applied to Figure 1 The collector in the photovoltaic module is used to perform the film tape missing detection method of the photovoltaic module through the film tape missing detection of the photovoltaic module to improve the accuracy, efficiency and stability of parameter setting of the equipment. The film tape missing detection method of the photovoltaic module may include the following steps 101 to 103.

[0052] Step 101: Obtain sample images of photovoltaic modules.

[0053] The sample images may include multiple defective sample images and multiple qualified sample images. The defective sample images may be sample images obtained by taking images of bad (Not Good, NG) products during the production process, and refer to sample images with defects. The defective sample images indicate defects that occur during product manufacturing or production. The defective sample images may have corresponding defect labels to indicate the defects corresponding to the defective sample images.

[0054] Among them, the qualified sample images can be sample images obtained by taking images of qualified (OK) products, which refer to sample images without obvious defects and samples that meet product specifications and quality standards. The qualified sample images are used to verify the accuracy and reliability of the detection network to ensure that the detection network can correctly identify and distinguish real defects. The processor obtains multiple defect sample images and multiple qualified sample images of the target product to train the model for defect detection.

[0055] In one example, both defect sample images and qualified sample images of an embodiment of the present invention can be selected by sampling with control of potential influencing factors, such as sampling error, sample size, and sample distribution, so that the obtained samples are more representative.

[0056] The defective sample image may include a first cut image corresponding to a location where a film strip is missing on the photovoltaic module, and the qualified sample image may include a second cut image corresponding to a location where a film strip is not missing on the photovoltaic module.

[0057] In some embodiments, the segmentation graph and sample image can be obtained by the following steps:

[0058] First, obtain the original image of the photovoltaic module;

[0059] Secondly, the original image is segmented to obtain multiple segmented images, the segmented image with missing film strips is used as the first segmented image, and the first segmented image is annotated to obtain a defect sample image;

[0060] Next, the cut image without missing membrane strips is used as the second cut image to obtain a qualified sample image.

[0061] In the specific implementation, during the production or testing process, a high-definition camera device can be used to obtain the original image of the photovoltaic module. The original image contains the overall information of the photovoltaic module, including its surface condition and any possible defects, providing a basis for subsequent segmentation and analysis.

[0062] Next, the acquired original image can be segmented. Specifically, a complete original image can be segmented into multiple small images, i.e. segmented images. Thus, the image can be divided into smaller units, which helps to perform more detailed analysis and processing on each unit.

[0063] It can be understood that the segmentation process can divide the original large image into multiple small areas, making the subsequent defect detection more detailed and accurate, avoiding the detection error caused by the overall image being too large. Each small area of ​​the image can be processed separately, thereby improving the accuracy and efficiency of detection.

[0064] In the segmented pictures, the defective segmented pictures are identified through image processing algorithms or manual inspection, and are used as the first segmented pictures, that is, the defective pictures. Then, each first segmented picture can be annotated to mark the type, location and range of the defects to generate defect sample pictures to provide reference data for model training or subsequent defect identification. By annotating defective pictures, the model can better learn how to identify different types of defects and improve the accuracy of detection. Finally, all defective segmented pictures, that is, the first segmented pictures, can be annotated in detail. The annotation information can include specific details such as the type, location, shape, etc. of the defect, and finally generate a defect sample picture. Correspondingly, for the second segmented picture without defects, no annotation can be made to distinguish it from the first segmented picture as a qualified sample picture.

[0065] Through the above methods, a high-quality defect sample library can be created as an important reference data for subsequent model training and defect detection. By accurately labeling defect samples, the model can better learn and identify defect features, thereby accurately identifying similar defects in actual detection.

[0066] Step 102: Input the sample image into the initial detection network, and train the initial detection network to obtain the target detection network.

[0067] The initial detection network of the present invention can be an attention convolutional neural network, that is, a basic network for predicting membrane band missing. The processor can input the acquired first segmentation graph and second segmentation graph into the initial detection network for training, and continuously iterate and update the parameters of the attention convolutional neural network. When the iteration termination condition is met, the detection network obtained is the target detection network.

[0068] The attention convolutional neural network of the present invention combines the convolutional neural network with the attention mechanism to better extract and process features, and provide a more accurate and efficient detection solution for problems such as missing film strips and photovoltaic module defects in complex scenarios.

[0069] In some embodiments, the initial detection network may include a feature processing layer, the feature processing layer includes a feature extraction layer and a feature concatenation layer, and the initial detection network may also be trained in the following manner:

[0070] First, the sample image is input into the feature extraction layer, and multi-scale feature extraction is performed on the sample image through the feature extraction layer, and multiple features of the sample image are output;

[0071] Next, multiple features of the sample image are input into the feature splicing layer to obtain a first splicing feature output by the feature splicing layer.

[0072] The feature extraction layer may include multiple convolution kernels of different sizes, and the sample image may be input into multiple convolution kernels, each convolution kernel outputs a feature of a corresponding scale, and features of multiple scales of the sample image are obtained. Figure 2 As shown in the figure, the size of the convolution kernel can vary from 1×1 to 64×64, for example, a total of 64 convolution kernels are included, and convolution kernels of different sizes can extract features of different scales in the image. For example, smaller convolution kernels (such as 1×1, 3×3, 5×5) can capture detailed features in the image, such as edges, textures, etc.; larger convolution kernels (such as 32×32, 64×64) can obtain image information in a larger range, such as overall shape and layout.

[0073] In some embodiments, each convolution kernel can output a feature map of a corresponding scale when performing a convolution operation on the image. For example, each convolution layer outputs a feature map of dimension dim×h×w=128×128×16. These feature maps represent the information of the sample image at different scales. For example, the feature map extracted by a 1×1 convolution kernel can retain the most basic information of each pixel, while a 64×64 convolution kernel can extract global information in a larger range. Through these multi-scale feature extractions, the model can understand the content of the image at different scales and provide multi-level information support for subsequent splicing and detection.

[0074] In some embodiments, the feature concatenation layer can combine feature maps extracted by multiple different convolution kernels according to certain rules, such as concatenating each feature map in the channel dimension. Through this concatenation operation, the model can simultaneously retain multiple scale information of the image, so that subsequent detection can combine detail features (small-scale information) and global features (large-scale information) to make more accurate judgments.

[0075] In some embodiments, feature maps of all scales may be spliced ​​together, and the obtained first spliced ​​features contain comprehensive information of the sample image at different scales, so that the model can make full use of these multi-scale features for more comprehensive analysis and judgment.

[0076] It can be understood that since the first splicing feature contains feature information from different convolution kernels, the model can simultaneously consider local fine features (such as small defects and fine cracks) and global structural features (such as overall shape abnormalities and large missing pieces) during detection. The fusion of multi-scale information makes the model more adaptable and robust when dealing with complex scenes and diverse defects.

[0077] The present invention extracts multi-scale features through the feature extraction layer, and then fuses the features of different scales through the feature splicing layer to form a first splicing feature, and finally performs detection based on the first splicing feature. This multi-scale feature fusion method can better adapt to complex scenes and diverse defect features, and improve the accuracy and robustness of detection.

[0078] In some embodiments, the initial detection network may further include an embedding coding layer, the embedding coding layer including an embedding layer, a coding layer and a de-embedding layer, and the initial detection network may also be trained in the following manner:

[0079] First, the first splicing feature is input into the embedding layer, and the first splicing feature is mapped into a low-dimensional vector through the embedding layer to obtain the embedded first splicing feature;

[0080] Next, the embedded first splicing feature is input into the encoding layer, and multi-layer extraction is performed on the embedded first splicing feature to output multiple encoding feature vectors with richer feature information;

[0081] Then, the encoded feature vector is input into the de-embedding layer, the low-dimensional encoded feature vector is mapped and concatenated into a high-dimensional vector, and the second concatenated feature is output.

[0082] In some embodiments, the embedding layer can map high-dimensional features to low-dimensional vector representations through an embedding matrix. Specifically, the input high-dimensional features can be mapped to a compact, low-dimensional vector space through matrix multiplication. The purpose of the embedding layer is to compress the dimension of the features while ensuring the ability to express information, thereby reducing the amount of calculation and storage overhead. After mapping by the embedding layer, the low-dimensional first concatenated features can be obtained. These low-dimensional feature vectors contain the main information of the original high-dimensional features, but their representation is more compact, which is helpful for the processing of subsequent coding layers.

[0083] In some embodiments, the encoding layer may be a stack of multiple encoders, such as Figure 2 The middle encoding layer can be a "3×encoder layer" consisting of three layers of encoders, which performs multi-level feature extraction on the input embedded features. Each encoder layer is usually composed of multiple sub-layers, such as Multi-Head Self-Attention and Feed-Forward Neural Network. Through the combination of these sub-layers, the encoding layer can capture deeper associations between features and enrich feature representation. After multi-layer encoding extraction, multiple encoded feature vectors with richer feature information are output. These feature vectors deeply refine and enhance the original high-dimensional features in low-dimensional space, and can better represent the key feature information of sample images.

[0084] In some embodiments, the de-embedding layer can map the low-dimensional feature vector back to the high-dimensional space through a de-embedding matrix (usually the transpose or similar operation of the embedding matrix) to obtain features of dimensions b×c×h×w. The high-dimensional representation of the features can be restored from the previous low-dimensional features so that these features can contain more detailed information. At the same time, the de-embedding layer can splice multiple encoded feature vectors together to form a more comprehensive high-dimensional feature representation. The output of the de-embedding layer is the second spliced ​​feature, which contains rich feature information after multi-layer encoding. This feature representation not only retains the multi-scale information of the original input, but also undergoes deep extraction and high-dimensional restoration through multi-layer encoding, and has stronger expressive power.

[0085] It can be understood that the second splicing feature has integrated multi-scale and deep feature information. It can be input into the final classification or detection model, such as a convolutional layer, a fully connected layer, or other detection modules. Through these detection modules, the model can better utilize the information in the second splicing feature to analyze the sample image and determine whether there are defects or other anomalies.

[0086] The present invention can effectively reduce the computational complexity while retaining the main information of the features through bidirectional mapping of the embedding layer and the de-embedding layer. The introduction of multi-layer encoders further enriches the feature expression, enabling the model to better detect and identify in complex scenes, and is particularly suitable for tasks that require accurate extraction and expression of multi-scale and multi-level features, such as micro-defect detection in images, target recognition in complex backgrounds, etc. Through multi-layer feature processing of embedding, encoding, and de-embedding, the model's ability to extract and express features of sample images is greatly enhanced, and the accuracy and robustness of detection are improved.

[0087] In some embodiments, the initial detection network may also include a first fully connected layer, a second fully connected layer and an activation function layer connected in sequence, each of the first fully connected layers includes multiple first fully connected layers, each of the second fully connected layers includes multiple second fully connected layers, the number of the first full connectors and the number of the second full connectors are equal and are both odd numbers, the second splicing feature can be input into multiple fully connected layers, and each fully connected layer outputs a probability distribution of whether a sample image has defects or not.

[0088] Among them, the fully connected layer (FC) is used to map the input features to the final classification or regression results. In this scheme, the attention convolutional neural network can include a first fully connected layer, a second fully connected layer and an activation function layer connected in sequence, and the number of first fully connected layers in the first fully connected layer and the number of second fully connected layers in the second fully connected layer are both odd numbers. The role of the fully connected layer and the fully connected layer is to map the high-dimensional features that have been extracted and processed to a low-dimensional space that is easier to classify or detect, and finally output the detection results through multiple levels of transformation.

[0089] In some embodiments, multiple fully connected layers can process the second splicing features in sequence. Since the number of full connectors contained in the two fully connected layers is an odd number (for example, 3, 5, 7, etc.), during the processing, the input features will be converted into different feature vectors. This layer-by-layer conversion can gradually refine the features and compress the amount of information, and finally form a compact feature representation for classification. The function of each fully connected layer is to linearly transform the input features through the weight matrix and the bias term, and then apply a nonlinear activation function (such as softmax, ReLU, sigmoid, etc.) to perform nonlinear processing on the features. This can enhance the nonlinear expression ability of the features and improve the classification performance of the model.

[0090] Each input feature, after being processed by the first full connector, the second full connector, and the corresponding activation function layer, will output a probability distribution of the corresponding category. For example, the output result may be a vector whose value represents the probability of the sample image belonging to the two categories of "defective" and "non-defective".

[0091] like Figure 2 As shown in the figure, the processing process for a certain feature is: FC(128×1024)→FC(1024×2)+softmax. Among them, FC(128×1024) indicates that the input dimension of the full connector is 128 and the output dimension is 1024. It means that the 128-dimensional input feature vector is converted into a 1024-dimensional feature representation. FC(1024×2) means that the full connector maps the 1024-dimensional feature representation to a 2-dimensional output space. This 2-dimensional output vector usually corresponds to the probability distribution of two categories (defects and no defects). Finally, the softmax function can convert the output 2-dimensional vector into a probability distribution, that is, each value represents the probability of the corresponding category. The softmax function will standardize the output value between 0 and 1, and the sum of all output values ​​is 1. For example, the output result can be: the probability of the sample image of the photovoltaic module being defective is 80%, and the probability of no defects is 20%.

[0092] It can be understood that the use of multiple full connectors can refine and process feature information layer by layer, so that the model can understand the input features more comprehensively and enhance the model's discrimination ability. By integrating the probability distribution of multiple full connectors, the judgment error of a single layer can be effectively avoided, and the stability and accuracy of the overall detection results can be improved. The introduction of multiple full connectors enables the model to capture more complex nonlinear relationships between features, which helps to improve the model's ability to recognize complex defect patterns.

[0093] In summary, the present invention processes the second splicing feature layer by layer through multiple full connectors to generate the defect probability distribution of the sample image, and obtains more accurate detection results during the training process of the initial detection network by integrating multiple probability distribution information. This multi-level and comprehensive detection strategy can effectively improve the detection performance and robustness of the model.

[0094] In some embodiments, the initial detection network may further include a voting layer, and the initial detection network may further be trained in the following manner:

[0095] First, each probability distribution is input into the voting layer to analyze the probability distribution and obtain the voting result of whether the product to be tested has a missing membrane band;

[0096] Next, the detection result of the film tape missing detection of the photovoltaic module by the initial detection network is determined according to the voting result.

[0097] The main function of the voting layer is to make a comprehensive judgment on the probability distribution of the output of the previous layer after being processed by multiple full connectors and activation function layers. The voting mechanism is used to determine the final detection result of the initial detection network for the missing film strip detection of photovoltaic modules. This is a strategy commonly used in ensemble learning or multi-model output integration, which can effectively improve the stability and accuracy of model decision-making.

[0098] In the attention convolutional neural network of the present invention, after being processed by multiple fully connected layers, each fully connected layer will output a probability distribution for whether the input sample image has defects. For example, a fully connected layer may output probability values ​​of two categories, "defects exist" and "defects do not exist" (such as [0.3, 0.7], indicating that the probability of the sample image having defects is 30%, and the probability of not having defects is 70%). All probability distributions can be used as input to the voting layer. The output of each fully connected layer represents its independent judgment results, and these judgment results are further comprehensively analyzed in the voting layer.

[0099] In some embodiments, the category corresponding to the maximum probability in each probability distribution can be regarded as the voting result of the layer. Then the voting results of all fully connected layers are counted, and the category with the most votes is selected as the final result. For example, if there are three fully connected layers, two of which believe that "defects exist" and the other believes that "defects do not exist", the final voting result is "defects exist".

[0100] In some embodiments, the output probabilities of each layer can be weighted and summed according to the weight of each fully connected layer (the weight can be set according to the performance of the layer or experience). Finally, the category with the highest score is selected as the final voting result. For example, some fully connected layers contribute more to detection, so they can be given higher weights.

[0101] In some embodiments, the voting result can be a single category label, such as "defect exists" or "defect does not exist," or it can be a vector representing category probabilities, such as [0.8, 0.2], which means that the probability of defect existence is 80% and the probability of defect non-existence is 20%. By integrating the outputs of multiple fully connected layers into the voting layer, the instability caused by the judgment error of a single layer can be effectively avoided, and the reliability and robustness of the overall judgment can be improved. In particular, when the outputs of each layer are inconsistent, the voting layer can reach a more robust judgment through the "Most Common" mechanism, which will be explained in detail below.

[0102] For the convenience of example description, assume that the first fully connected layer has 3 first fully connected connectors, the second fully connected layer has 3 second fully connected connectors, the intermediate parameters output by the corresponding first fully connected connectors are input to the second fully connected connectors for processing, and the intermediate parameters output by the second fully connected connectors are processed by the activation function layer, and finally 3 probability distributions are obtained. The 3 probability distributions obtained are:

[0103] [0.8, 0.2], the probability of a defect is 80%, and the probability of no defect is 20%;

[0104] [0.3,0.7], the probability of a defect is 30%, and the probability of no defect is 70%;

[0105] [0.9,0.1], the probability of a defect is 90%, and the probability of no defect is 10%.

[0106] In some embodiments, in order to facilitate the voting process, each probability distribution can be mapped to a specific category label first. Usually, the category corresponding to the maximum probability in the probability distribution is taken as the judgment result of the probability distribution, so the results corresponding to the above three probability distributions are "defects exist", "defects do not exist" and "defects exist", respectively. In a specific implementation, after the voting layer collects the above results, it can use the "Most Co mmon" mechanism to perform majority voting and select the result with the most occurrences as the final voting result. For example, in the above three results, the number of "defects exist" is greater than the number of "defects do not exist", so the final voting result is "the photovoltaic module has a defect of missing film strips".

[0107] As can be seen from the above, the voting layer of the present invention processes the results of multiple probability distributions through the "Most Common" majority voting method to obtain the final detection judgment, which can comprehensively judge the processing results of each full connector and activation function layer and output a more accurate voting result. Therefore, the voting layer can significantly improve the robustness and stability of detection when processing complex photovoltaic module defect detection tasks.

[0108] The introduction of the voting layer makes the final decision of the model more robust. In the entire detection process, it plays the role of comprehensive judgment and final decision, ensuring that a robust judgment result can be obtained when the judgment results of multiple layers are inconsistent. According to the voting results, a definite detection result can be output. This result can be further used for quality control, control of automated production lines or other related decisions.

[0109] By integrating the outputs of multiple fully connected layers, the voting layer can effectively improve the consistency of decisions and avoid misjudgments at a single level. When faced with complex situations or scenarios with high uncertainty, the voting layer can provide more stable detection results and improve the overall robustness of the system. The voting layer can adjust the voting strategy according to different application scenarios and requirements. For example, it can use a simple majority voting method or a weighted voting method to adapt to different task complexities and accuracy requirements.

[0110] In summary, the voting layer of the present invention can effectively integrate the judgment results from multiple full connectors, making the detection process more stable and accurate, and helping to improve the overall performance of the system in complex detection tasks.

[0111] In some embodiments, the training process of the initial detection network may include:

[0112] First, multiple sample images are input into the initial detection network to obtain multiple initial defect prediction results;

[0113] Next, the loss function value of the target detection network is obtained according to each initial defect prediction result, the defect sample image, and the defect label of the defect sample image;

[0114] Finally, the model parameters of the initial detection network are iteratively updated according to the loss function value until the iteration termination condition is met to obtain the target detection network.

[0115] In an embodiment of the present invention, the target detection network is a detection network obtained by training the initial detection network. Both the defect sample image and the target defect sample image have corresponding defect labels. By inputting multiple defect sample images and multiple target defect sample images into the initial detection network, the parameters of the initial detection network are continuously adjusted so that the initial detection network can accurately detect defects on the input images.

[0116] Step 103: Input the product image of the product to be inspected into the target detection network, detect whether the film strip of the product to be inspected is missing, and obtain the target detection result.

[0117] In an embodiment of the present invention, after the processor obtains the target detection network, the missing film strip of the product to be detected can be detected through the target detection network. The processor can first obtain the product image of the product to be detected. The method of obtaining the product image may include but is not limited to acquisition through image acquisition equipment, database retrieval, network download, etc. Among them, the image acquisition equipment may include but is not limited to cameras, digital cameras, handheld cameras and other devices that can capture images. Then, the processor inputs the product image into the target detection network to obtain the defect detection result of the current product to be detected. Among them, the defect detection result may include but is not limited to: good, bad and specific defect types, etc.

[0118] In some embodiments, step 103 may include:

[0119] Outputting a first detection result of whether the product to be detected has a missing film band through the target detection network, and outputting a second detection result when the product to be detected has a missing film band;

[0120] The second test result includes the defect mark and position of the missing film tape of the product to be tested.

[0121] The target detection network of the present invention can not only identify whether there is a target in the image, but also locate the specific position of the defect of missing film strip in the image. The target detection network can be used to determine whether the product to be inspected has defects, and output the specific defect type and defect location if there are defects.

[0122] In some embodiments, the first test result can be a binary classification result, i.e., "defects exist" or "defects do not exist". If the model detects any type of defects in the product, the result output is "defects exist". If the model does not detect any defects or considers the product to be fully qualified, the result output is "defects do not exist". The first test result is used to make a preliminary judgment on the overall quality of the product, and can quickly identify products with obvious defects. For products without defects, they can be directly judged as qualified without further analysis.

[0123] In some embodiments, when the first detection result shows that "a defect exists", the second part of the target detection network outputs a "second detection result". This result contains detailed information about the defect, such as the type of defect (defect identification) and the specific location of the defect in the product. Figure 3 As shown in FIG. 1 , after the target detection network outputs the target detection result, the specific location where the film strip missing defect exists can be marked in the image of the photovoltaic module, and then the location can be marked and annotated, for example Figure 3In the figure, "MQDS111" is shown, where "MQDS" indicates the presence of a missing film strip to distinguish it from other types of defects. The following "111" is a generated random number to distinguish it from missing film strip defects at other locations.

[0124] As can be seen from the above, the target detection network of the present invention can effectively judge the overall quality of the product to be detected by integrating the first detection result and / or the second detection result, and provide detailed defect information when defects exist. This can not only quickly screen qualified products, but also accurately locate and mark defective products, thereby improving detection efficiency and accuracy.

[0125] In the embodiment of the present invention, the defective sample images and qualified sample images of photovoltaic modules are used to train the initial detection network to obtain the target detection network. The target detection network is used to detect the missing film strips of photovoltaic modules, and the multi-level features of the film strip images are automatically learned and extracted efficiently, and the key areas in the film strip images are focused on, which effectively enhances the perception of the subtle missing film strip areas, reduces missed detection and false detection, and thus can improve the accuracy and efficiency of the missing film strip detection of photovoltaic modules.

[0126] In a possible example, in order to better implement the film tape missing detection method of the photovoltaic module in the embodiment of the present invention, on top of the film tape missing detection method of the photovoltaic module, the embodiment of the present invention further provides a film tape missing detection method 200 of the photovoltaic module, such as Figure 4 As shown, the film tape missing detection 200 of the photovoltaic module includes:

[0127] The data acquisition module 201 is used to acquire sample images of photovoltaic modules; the sample images include a plurality of defective sample images and a plurality of qualified sample images, the defective sample images include a first cut image corresponding to a position where a film strip of the photovoltaic module is missing, and the qualified sample images include a second cut image corresponding to a position where a film strip of the photovoltaic module is not missing;

[0128] The model training module 202 inputs the sample image into the initial detection network, and trains the initial detection network to obtain a target detection network;

[0129] The defect detection module 203 is used to input the product image of the product to be detected into the target detection network, detect whether there is a film strip missing on the product to be detected, and obtain the target detection result.

[0130] In an embodiment of the present invention, the data acquisition module 201, the model training module 202 and the defect detection module 203 can be used to execute steps 101-103 of the aforementioned method embodiment respectively. For more detailed contents or specific implementation methods of each functional module, please refer to the description of the corresponding method steps, which will not be repeated here.

[0131] The film strip missing detection 200 of the photovoltaic module provided by the present invention first obtains a sample image of the photovoltaic module through the data acquisition module 201; the sample image includes a plurality of defective sample images and a plurality of qualified sample images, the defective sample image includes a first cut image corresponding to the position where the film strip of the photovoltaic module is missing, and the qualified sample image includes a second cut image corresponding to the position where the film strip of the photovoltaic module is not missing. Then, through the model training module 202, the sample image is input into the initial detection network, and the initial detection network is trained to obtain the target detection network. Finally, the product image of the product to be detected is input into the target detection network through the defect detection module 203, and the film strip missing of the product to be detected is detected to obtain the target detection result.

[0132] The embodiment of the present invention also provides a computer device. Figure 5 , Figure 5 This is a basic structural block diagram of the computer device in this embodiment.

[0133] The computer device includes a memory 310 and a processor 320 that are connected to each other through a system bus. It should be noted that the figure only shows a computer device with components 310-320, but it should be understood that it is not required to implement all the components shown, and more or fewer components can be implemented instead. Among them, those skilled in the art can understand that the computer device here is a device that can automatically perform numerical calculations and / or information processing according to pre-set or stored instructions, and its hardware includes but is not limited to microprocessors, application specific integrated circuits (Application Specific Integrated Circuit, ASIC), programmable gate arrays (Field-Programmable Gate Array, FPGA), digital processors (Digital Signal Processor, DSP), embedded devices, etc.

[0134] Computer devices can be computing devices such as desktop computers, notebooks, PDAs, and cloud servers. Computer devices can interact with users through keyboards, mice, remote controls, touch pads, or voice control devices.

[0135] The memory 310 includes at least one type of readable storage medium, and the readable storage medium includes a non-volatile memory or a volatile memory, for example, a flash memory, a hard disk, a multimedia card, a card-type memory (for example, an SD or DX memory, etc.), a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), a programmable read-only memory (PROM), a magnetic memory, a magnetic disk, an optical disk, etc., and the RAM may include a static RAM or a dynamic RAM. In some embodiments, the memory 310 may be an internal storage unit of a computer device, for example, a hard disk or a memory of the computer device. In other embodiments, the memory 310 may also be an external storage device of a computer device, such as a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (SecureDigital, SD) card or a flash card (Fla sh Card) equipped on the computer device. Of course, the memory 310 may also include both an internal storage unit of the computer device and its external storage device. In this embodiment, the memory 310 is generally used to store an operating system and various application software installed on the computer device, such as the program code of the above method. In addition, the memory 310 may also be used to temporarily store various types of data that have been output or are to be output. In some embodiments, the memory 310 may also be a register.

[0136] The processor 320 is generally used to perform the overall operation of the computer device. In this embodiment, the memory 310 is used to store program codes or instructions, the program code includes computer operation instructions, and the processor 320 is used to execute the program codes or instructions stored in the memory 310 or process data, such as running the program code of the above method.

[0137] In this article, the bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. The bus system can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, only one thick line is used in the figure, but it does not mean that there is only one bus or one type of bus.

[0138] Another embodiment of the present invention further provides a computer-readable medium, which may be a computer-readable signal medium or a computer-readable medium. A processor in a computer reads a computer-readable program code stored in the computer-readable medium, so that the processor can execute the functional actions specified in each step or a combination of steps in the above method; and generate a device for implementing the functional actions specified in each block or a combination of blocks in the block diagram.

[0139] Computer-readable media include but are not limited to electronic, magnetic, optical, electromagnetic, infrared memory or semiconductor systems, devices or apparatuses, or any appropriate combination of the foregoing, the memory is used to store program codes or instructions, the program codes include computer operating instructions, and the processor is used to execute the program codes or instructions of the above methods stored in the memory.

[0140] For the definitions of memory and processor, please refer to the description of the aforementioned computer device embodiment and will not be repeated here.

[0141] In the several embodiments provided by the present invention, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic, for example, the division of modules or units is only a logical function division, and there may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0142] Each functional unit or module in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional unit.

[0143] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, server, or network device, etc.) or a processor to perform all or part of the steps of the various embodiments of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), disk or optical disk, etc., and other media that can store program codes.

[0144] In the claims, any reference signs placed between brackets shall not be construed as limiting the claims. The word "comprising" described in the invention does not exclude the presence of elements or steps not listed in the claims. The word "a" or "an" preceding an element does not exclude the presence of a plurality of such elements. The present invention can be implemented with the aid of hardware comprising several different elements and with the aid of appropriately programmed computers. In a unit claim enumerating a number of devices, several of these devices may be embodied by the same hardware item. The use of first, second, and third etc. does not indicate any order, and these words may be interpreted as names. The steps in the above embodiments should not be understood as limitations on the order of execution unless otherwise specified.

[0145] The above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for detecting film tape missing of a photovoltaic module, characterized in that: The method comprises: Acquire sample images of photovoltaic modules; the sample images include a plurality of defective sample images and a plurality of qualified sample images, the defective sample images include a first cut image corresponding to a position where a film strip of the photovoltaic module is missing, and the qualified sample images include a second cut image corresponding to a position where a film strip of the photovoltaic module is not missing; Inputting the sample image into an initial detection network, and training the initial detection network to obtain a target detection network; The product image of the product to be detected is input into the target detection network, and the product to be detected is detected to see whether there is a missing film strip, so as to obtain a target detection result.

2. The method for detecting film tape missing of a photovoltaic module according to claim 1, characterized in that: The initial detection network includes a feature processing layer, and the feature processing layer includes a feature extraction layer and a feature splicing layer; The training of the initial detection network to obtain a target detection network includes: Inputting the sample image into the feature extraction layer, performing multi-scale feature extraction on the sample image through the feature extraction layer, and outputting multiple features of the sample image; Input multiple features of the sample image into the feature splicing layer to obtain a first splicing feature output by the feature splicing layer.

3. The method for detecting film tape missing of a photovoltaic module according to claim 2, characterized in that: The feature extraction layer includes a plurality of convolution kernels of different sizes; Performing multi-scale feature extraction on the sample image and outputting multiple features of the sample image includes: The sample image is input into a plurality of the convolution kernels, each of the convolution kernels outputs a feature of a corresponding scale, and features of a plurality of scales of the sample image are obtained.

4. The method for detecting film tape missing of a photovoltaic module according to claim 2, characterized in that: The initial detection network further includes an embedded coding layer, wherein the embedded coding layer includes an embedding layer, a coding layer and a de-embedding layer; The training of the initial detection network to obtain a target detection network includes: Inputting the first splicing feature into the embedding layer, mapping the first splicing feature into a low-dimensional vector through the embedding layer, and obtaining an embedded first splicing feature; Inputting the embedded first splicing feature into the encoding layer, performing multi-layer extraction on the embedded first splicing feature, and outputting a plurality of encoding feature vectors with richer feature information; The encoded feature vector is input into the de-embedding layer, the low-dimensional encoded feature vector is mapped and concatenated into a high-dimensional vector, and a second concatenated feature is output.

5. The method for detecting film tape missing of a photovoltaic module according to claim 4, characterized in that: The initial detection network also includes a first fully connected layer, a second fully connected layer and an activation function layer connected in sequence, each of the first fully connected layers includes a plurality of first fully connected layers, each of the second fully connected layers includes a plurality of second fully connected layers, and the number of the first fully connected layers is equal to the number of the second fully connected layers and both are odd numbers; The training of the initial detection network to obtain a target detection network includes: The second splicing feature is input into the first fully connected layer, and after being processed by the first fully connected layer, the second fully connected layer and the activation function layer, a probability distribution of whether the sample image has defects or does not have defects is output.

6. The method for detecting film tape missing of a photovoltaic module according to claim 5, characterized in that: The initial detection network also includes a voting layer; The training of the initial detection network to obtain a target detection network includes: Input each of the probability distributions into the voting layer to analyze the probability distributions and obtain a voting result of whether the product to be tested has a missing membrane strip; According to the voting result, a detection result of the initial detection network performing film tape missing detection on the photovoltaic module is determined.

7. The method for detecting film tape missing of a photovoltaic module according to claim 1, characterized in that: The step of inputting the product image of the product to be detected into the target detection network, detecting whether the film strip of the product to be detected is missing, and obtaining the target detection result includes: Outputting a first detection result of whether the product to be detected has a missing film band through the target detection network, and outputting a second detection result when the product to be detected has a missing film band; The second detection result includes the defect mark and position of the missing film tape of the product to be detected.

8. A device for detecting missing film strips of a photovoltaic module, characterized in that: The device comprises: A data acquisition module, used to acquire sample images of photovoltaic modules; the sample images include a plurality of defective sample images and a plurality of qualified sample images, the defective sample images include a first cut image corresponding to a position where a film strip of the photovoltaic module is missing, and the qualified sample images include a second cut image corresponding to a position where a film strip of the photovoltaic module is not missing; A model training module, inputting the sample image into an initial detection network, and training the initial detection network to obtain a target detection network; The defect detection module is used to input the product image of the product to be detected into the target detection network, detect whether there is any film strip missing on the product to be detected, and obtain the target detection result.

9. A computer device, characterized in that: include: Processor and memory; The memory is used to store computer programs; The processor is used to execute the computer program stored in the memory to implement the steps of the method for detecting film strip missing of a photovoltaic module as described in any one of claims 1-7.

10. A computer-readable storage medium, characterized in that: A computer program is stored, and the computer program is loaded by a processor to execute the steps of the method for detecting film strip missing of a photovoltaic module as described in any one of claims 1 to 7.

Images