Method for generating training gallery for visual AI detection of defective products of junction box

Abstract

A method for generating a training gallery for visual AI detection of defective products of a junction box relates to the field of photovoltaic technology, and uses an image processing algorithm to generate a defective product effect picture according to extracted features, each defect can have a plurality of different degrees and forms, and the visual AI detection of the defective products of the junction box can be realized by combining different defect features and iterative optimization. Various defective product effect pictures are generated, and it is ensured that the generated defective product effect picture library can cover various defect conditions possibly occurring in actual production. And integrating the generated defective product effect pictures into a library for subsequent visual AI model training. The defective product effect picture generated through image processing is used for replacing defective products in actual production, manpower, material resources and time cost are saved, meanwhile, the generated defective product effect picture library provides rich defect types and forms, and the robustness of the model and the convergence of the AI model are improved.

Description

Technical Field

[0001] The present invention relates to the technical field of photovoltaic modules, and particularly to a method for generating a training image library for visual AI detection of defective junction boxes. Background Art

[0002] In a photovoltaic power generation system, a junction box is a core component connecting solar cell modules to external circuits, undertaking key functions such as current regulation, hot spot protection, and sealing insulation. With the leap of global photovoltaic installed capacity, the market demand for photovoltaic junction boxes has soared to hundreds of millions of sets, and its quality control directly affects the safety and power generation efficiency of power stations.

[0003] A photovoltaic junction box includes a box body, diodes, conductive components, sealing rings, connectors, etc., and needs to meet strict performance indicators such as IP65 waterproof, -40~90°C temperature change resistance, and ultraviolet resistance. During production, due to processes such as resistance welding, crimping, and reflow soldering, the solder pads may be damaged, offset, or missing, resulting in defective products. Therefore, we have introduced AI visual inspection on the basis of the original manual inspection. However, AI visual inspection requires a large number of defective products in production. The pass rate of photovoltaic modules usually exceeds 99%, and real defective samples are scarce, resulting in insufficient training data and weak model generalization ability; defective products need to be sampled and calibrated manually, and the collection of defective product samples takes a long time, wasting a lot of manpower, material resources, and financial resources. Therefore, how to integrate the generated defective product effect diagrams into a library for subsequent visual AI model training is a technical problem that needs to be solved urgently in this case. Summary of the Invention

[0004] The present invention aims at the above problems and provides a method for generating a training image library for visual AI detection of defective junction boxes, which can enrich the types and forms of defects and improve the robustness and convergence of the model.

[0005] The technical solution of the present invention is as follows: A method for generating a training image library for visual AI detection of defective junction boxes, characterized by including the following steps: Step S1: Collect image data of junction boxes during production; Step S2: Perform image preprocessing, extract the defect features of the junction box, and encode these features to form a multi-dimensional feature array; Step S3: Use an image processing algorithm to generate simulated defective product effect diagrams with different categories and forms according to the defect features extracted in Step S2; Step S4: Form a diversified defective product effect diagram library by combining the parameters of different defect features and iteratively optimizing the generation process.

[0006] Specifically, after Step S4 is completed, Step S5 is executed; Step S5: Verify the generated defective product effect diagram, determine whether it meets the defect standards in actual production, and adjust the feature extraction or generation model parameters according to the verification results to complete the construction of the image library.

[0007] Specifically, the defect feature extraction described in step S2 includes: Locate the boundary of the solder pad offset through an edge detection algorithm, and calculate the offset angle and the pixel difference from the reference position. Use image segmentation technology to identify the area ratio of the damaged area, and determine the geometric features of the damaged shape in combination with morphological analysis. Analyze the RGB color difference between the missing area and the surrounding normal area, and quantify the color change threshold.

[0008] Specifically, after generating the simulated defective product effect diagram in step S3, further perform data enhancement operations on the image, including rotation, scaling, flipping, or color channel transformation, to improve the diversity of the training image library.

[0009] Specifically, the parameters for combining different defect features in step S4 include: Randomly combine the solder pad offset, damage, and missing features to generate a composite defective product effect diagram. Optimize the feature parameter combination through a genetic algorithm to maximize the coverage of defect scenarios in actual production.

[0010] Specifically, the verification in step S5 includes: Input the generated defective product effect diagram into a pre-trained AI detection model, and count the false detection rate and the missed detection rate. If the false detection rate exceeds the preset threshold, adjust the noise parameters or the feature weight distribution of the generation model. If the missed detection rate exceeds the preset threshold, increase the feature extraction dimension of specific defect types.

[0011] Specifically, after step S5 is completed, execute step S6; Step S6: After applying the generated image library to the training of the AI detection model, feedback to step S2 or step S3 according to the actual detection results, and dynamically optimize the feature extraction rules or the generation model structure.

[0012] Specifically, the image data in step S1 includes the junction box body area and the connector area.

[0013] Specifically, the junction box body area includes the box body positioning posts, conductors, solder pads, and diodes for casting diode modules.

[0014] Specifically, the connector area includes the inner ring basket, the sealing ring, and the connecting nut.

[0015] The image processing algorithm of the present invention uses a convolutional neural network to extract features from the preprocessed image, and designs a CNN model, which extracts features such as the offset, damage area, morphology, and missing color change of the solder pads from the image. After feature extraction, a generative adversarial network model is designed, which mainly receives a set of features as input and generates images with corresponding defects. The generator can learn how to convert features into specific image defects. By providing only a small number of defective product feature pictures, through feature extraction, a training picture library is generated, and through training methods such as feedback data collection, dynamic adjustment strategy of the picture library, optimization of feature extraction rules, adjustment of model structure, and closed-loop iterative process, an AI detection model is obtained to improve the detection accuracy; the features of the constructed model are diverse, capable of covering the situations that may occur in production, and the model is continuously tested and optimized, thereby enhancing the anti-interference ability and being suitable for complex defect scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 is a schematic flowchart of the present invention; Figure 2 is a front view of the intermediate box body junction box of a qualified product; Figure 3 is an effect diagram of the solder pad offset of the junction box generated by image processing; Figure 4 is an effect diagram of the solder pad damage of the junction box generated by image processing; In the figure, 1 is the intermediate box body junction box, 2-1 is the upper solder pad, 2-2 is the lower solder pad, 3-1 is the upper conductor, and 3-2 is the lower conductor. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0017] The embodiments of the present invention will be described in detail below. The examples of the embodiments are shown in the drawings, where the same or similar reference numerals represent the same or similar elements or elements with the same or similar functions throughout. The embodiments described below by referring to the drawings are exemplary and are only used to explain the present invention and should not be construed as a limitation of the present invention.

[0018] The present invention mainly collects the image data of the junction box in production, extracts the key features of the junction box by using image processing, encodes these features to form a multi-dimensional feature array. Using image processing algorithms (such as GAN, CNN, etc.), effect diagrams of defective products are generated according to the extracted features. Each defect can have multiple different degrees and morphologies. By combining different defect features and iterative optimization, a variety of effect diagrams of defective products are generated to solve the problem that the defective products in actual production cannot meet the sample requirements of the AI training model.

[0019] The specific technical solution is as follows: Step S1: Collect the image data of the junction box in production; Use an industrial camera (resolution ≥ 12 million pixels) on the production line to collect RGB images of normal junction boxes, covering different lighting conditions, angles, and background environments, with a single batch collection volume of ≥ 5000 images.

[0020] The image data includes the junction box body part and the connector part; The junction box body part includes the box body positioning posts, conductors, solder pads, and diodes for casting diode modules, etc. The connector part includes an inner ring basket, a sealing ring, a connecting nut, etc.

[0021] Step S2: Image preprocessing, extract the defect features of the junction box, encode these features, and form a multi-dimensional feature array; The defect feature extraction includes: Locate the boundary of the solder pad offset through an edge detection algorithm, and calculate the offset angle and the pixel difference from the reference position; Use image segmentation technology to identify the area ratio of the damaged area, and combine morphological analysis to determine the geometric features of the damaged shape; Analyze the RGB color difference between the missing area and the surrounding normal area, and quantify the color change threshold.

[0022] Specifically, image preprocessing includes adjusting the resolution, cropping, denoising, adjusting the brightness, contrast, saturation, and image sharpening, etc., to improve the image quality and authenticity. The tools used for image preprocessing are OPenCV + Pillow. For example: Resolution adjustment: Uniformly scale to 1024×1024 pixels, retaining edge details; Denoising processing: Use the non-local means (NLM) algorithm to eliminate image noise; Background cropping: Segment the junction box body through Mask R-CNN, retaining the solder pad area; Data augmentation: Randomly rotate the image (±15°), translate (±5%), and adjust the brightness (±10%) to expand the dataset to 20,000 images.

[0023] The defect features include the angle and position of the solder pad offset, the area and shape of the solder pad damage, the color change at the position of the solder pad missing, etc.

[0024] Specifically, the defect features include the angle and position of the solder pad offset, the area and shape of the solder pad damage, the color change at the position of the solder pad missing, etc. For example: Solder pad offset: Detect the solder pad contour through the Hough transform, and calculate the offset (unit: pixel) of the center point coordinate from the theoretical position and the angle deviation (unit: degree); Solder pad damage: Extract the damaged area based on a semantic segmentation model (U-Net), and calculate the area ratio (%) and morphological parameters (such as circularity, aspect ratio); Missing solder pads: Detect the color difference (mean square error of RGB three channels) and edge transition characteristics of the missing area.

[0025] Feature encoding: Normalize the above features into a 32-dimensional vector (e.g., [offset_x, offset_y, angular deviation, damage area, circularity...]) and construct a multi-dimensional feature array.

[0026] The multi-dimensional feature array is similar to a multi-dimensional array (N, B), where N represents the number of features and B represents the encoded feature types. For example: 1. Geometric structure (outer dimensions, hole position spacing, angular deviation, key dimension positions); 2. Surface quality (scratches, burrs, injection molding defects, electroplating spots); 3. Assembly integrity (diode soldering positions, conductors installed in place); 4. Electrical characteristics (oxidation spots on resistance welding metal contacts); 5. Identification recognition (clarity and position accuracy of component identifiers). By establishing a multi-dimensional feature array, construct a data structure for expressing various defective junction boxes and establish a dataset for objectively expressing various defective junction boxes.

[0027] Step S3: Using an image processing algorithm, generate simulated defective product effect diagrams with different categories and forms according to the defect features extracted in step S2. After generating the simulated defect effect diagram, further perform data augmentation operations on the image, including rotation, scaling, flipping, or color channel transformation, to enhance the diversity of the training image library.

[0028] The image processing algorithm includes a convolutional neural network (CNN) and a generative adversarial network (GAN). The standard model structure CNN model. In the field of computer vision, due to its powerful ability to extract local features of images and the parameter sharing feature, CNN has become the basic model architecture for visual detection (such as object detection, image segmentation, etc.). The earliest classic model can be traced back to 2012 and has been continuously improved by subsequent research. AlexNet (2012) Paper: Krizhevsky et al., ImageNet Classification with Deep Convolutional Neural Networks (NIPS 2012) Contribution: First surpassed traditional methods in large-scale image classification (ImageNet), promoting CNN to become the standard model for visual tasks.

[0029] Use a convolutional neural network (CNN) to extract features from the preprocessed images. The role of the convolutional layer is to extract features at different levels. The low-level convolution detects edges and textures, while the high-level convolutional layer detects more complex object parts or the whole. Then, the pooling layer is used to reduce the computational amount while maintaining the translational invariance of the features. The fully connected layer may be used for the final classification or regression task, outputting the position and category of the detection box. In this project, the main extracted features include geometric structures (specifically, the shape dimensions, angular deviations, and key dimension positions of the box body, solder pads, and positioning posts), surface quality (scratches on the conductor, burrs, injection molding defects, electroplating spots), assembly integrity (solder positions of diodes, conductors installed in place), electrical characteristics (oxidation spots on the resistance welding metal contacts), and identification recognition (clarity of the component identifier, position accuracy), etc. (the above 5-dimensional features), that is, the CNN model extracts 5-dimensional defect features of the junction box from the image (such as the offset, damage area, morphology, and missing color changes of the solder pads).

[0030] After the defect features are extracted, design a generative adversarial network (GAN) model. The DCGAN architecture model uses a discriminator with a convolutional structure to extract deep features for image classification / detection, generates fake samples after data augmentation for training the model to improve robustness. This model mainly receives a set of features as input and generates images with corresponding defects. The generator can learn how to convert features into specific image defects.

[0031] For example, design a U-Net structure generator. Input the feature vector, generate defect images through the transposed convolution layer, and then use the PatchGAN structure to judge the local consistency between the generated image and the real defect; alternately optimize the generator and the discriminator, with the initial learning rate of 1e-4 and a 10% decay every 50 rounds. Control the defect degree by adjusting the feature vector parameters (for example: offset ±10 pixels, damage area 5%-20%). Support the superposition of multiple defects (such as the combination of "offset + damage") to generate defect images with various shapes.

[0032] The generated images can be further post-processed (such as adjusting brightness, contrast, saturation, and performing image sharpening operations) to improve authenticity.

[0033] Step S4: By combining the parameters of different defect features and iteratively optimizing the generation process, form a diverse defect product effect picture library for subsequent visual AI model training.

[0034] The parameters for combining different defect features include: Randomly combine the solder pad offset, damage, and missing features to generate composite defect effect pictures; Optimize the feature parameter combination through genetic algorithms to maximize the coverage of defect scenarios in actual production.

[0035] The optimization process of the genetic algorithm includes coding strategies, the design of fitness functions, the selection of genetic operations, and parameter settings. The optimization criteria are divided into internal evaluation indicators (such as accuracy rate, feature dimension) and industry standards (such as relevant specifications of ISO and IEC). At the same time, combined with the actual visual inspection analysis of junction boxes, specific data comparison, parallel computing, and online optimization systems, as well as common problems in the solutions, are provided.

[0036] The iterative optimization mainly adopts: If the PSNR or misjudgment rate does not meet the standard, then adjust the adversarial loss weight of the GAN (increase the gradient penalty term); Increase the noise perturbation of the feature vector (Gaussian noise σ = 0.01) to enhance the generation diversity.

[0037] The multi-class feature images generated by S3, which form a multi-dimensional array, are iteratively optimized (such as tensors) to generate one or more feature effect diagrams. The optimization of feature extraction rules is combined with the previous genetic algorithm to update the feature selection online, including coding strategies, the design of fitness functions, the selection of genetic operations, and parameter settings. The optimization criteria are divided into internal evaluation indicators (such as accuracy rate, feature dimension) and industry standards (such as relevant specifications of ISO and IEC). At the same time, combined with the actual visual inspection analysis of junction boxes, specific data comparison, parallel computing, and online optimization systems, as well as common problems in the solutions, are provided.

[0038] Step S5: Verify the authenticity of the generated defective product effect diagram, determine whether it meets the defective standards in actual production, and adjust the feature extraction or generation model parameters according to the verification results to complete the construction of the picture library.

[0039] The authenticity verification mainly considers from two aspects: quantitative indicators and quality inspection review: Quantitative indicators: Input the generated defective product effect diagram into the pre-trained AI detection model, and count the false detection rate and missed detection rate; If the false detection rate exceeds the preset threshold, then adjust the noise parameters or feature weight distribution of the generation model; If the missed detection rate exceeds the preset threshold, then increase the feature extraction dimension of specific defective types.

[0040] Expert review: Three quality inspection personnel conduct blind tests on 200 random samples, and the misjudgment rate is required to be < 5%.

[0041] In this case, YOLOv7 is selected as the detection model. The generated images are input, and the positions and types of defects are marked. Training parameters: batch size of 32, initial learning rate of 3e-4, and optimized using the cosine annealing strategy. It is assumed that 100,000 defective images are generated in the image library and stored classified by defect type (offset, damage, missing, and mixed type). Each type of defect contains 5 degrees (slight, mild, moderate, medium, and extreme), covering 99% of the defect scenarios statistically counted in the production line.

[0042] The quality center makes various limit samples of multi-dimensional features (from geometric dimensions, surface quality, assembly integrity, electrical characteristics, and identification recognition), as well as various defective products collected by the quality center usually. Using a dedicated industrial camera, pictures are taken at a fixed imaging distance and image size and entered into the software verification library.

[0043] Images of various defective products and qualified images in the image library are combined in a certain proportion. For example: for the detection of defective products of junction boxes, taking the number of pictures of solder pad skew, solder pad missing, reverse diode, conductor color difference, conductor missing, positioning post damaged, poor spot welding, and qualified products as 14,000 as an example: 1000:1000:1000:1000:1000:1000:1000:7000 are combined. Among them, 70% is used as the training image library, and 30% is used as the verification image library. It is required that the qualification rate of the model reaches more than 99.5%.

[0044] The defect classification and judgment criteria in this case mainly include: 1. Critical Defects Defect Type Specific standards Allowable ratio Solder lug standards The solder piece is in the center of the tin storage tank, and the edge of the solder piece is 0~1mm from the tin storage tank. 0% Solder tab missing Is there solder in the tin storage tank? 0% Positioning column crushed Missing / broken positioning post 0% 2. Major Defects Defect Type Specific standards AQL value Solder pad displacement The edge of the solder piece is larger than the tin storage tank by 0~1mm ≤0.65% Solder piece skew The soldering lug is tilted more than ±5° ≤1.0% 3. Minor Defects Defect Type Specific standards AQL value Dirty surface The area of ​​the deposits on the soldering pad surface is greater than 5mm² and cannot be wiped off ≤2.5% Color deviation ΔE>1.5 (Compared with standard color plate) ≤1.5% Minor defects The defect area on the soldering lug surface is greater than 15% of the soldering lug area ≤3.0% At the same time, according to the verification results, the GAN model or feature extraction method is adjusted, and iterative training is carried out.

[0045] As Figure 2-4 shown, they are a normal junction box, a junction box with solder pad offset, and a junction box with solder pad damage respectively. Figure 2 As shown above, the upper solder pad has a rotating feature, and the lower solder pad has a translational feature.

[0046] Figure 3 As shown above, the upper solder pad has a regular damaged feature, and the lower solder pad has an irregular damaged feature.

[0047] Figure 4The color feature of the upper solder pad position shown changes from gray to the metallic copper color of the upper conductor. Only one combination is taken as an example in the figure, but it is not limited to one. Each type of defect can have multiple different degrees and forms. The finally generated defect product effect picture library provides a rich variety of defect types and forms. The diverse training data enables the AI model to converge faster, improving the detection efficiency and accuracy.

[0048] Figure 3 It shows the effect picture of solder pad offset. The upper solder pad has a rotational feature, and the lower solder pad has a translational feature. Taking the center of the solder storage groove as the origin, the rotation angle of the solder pad (such as ±5°) and the distance that the solder pad extends beyond the solder storage groove (such as ±1mm) are used for feature extraction. This feature is not limited to Figure 3 the features shown. Subsequently, combined iterative optimization is carried out (such as combining a rotation of 6° and extending 2mm beyond the solder storage groove) to generate defect product effect pictures containing one or more features, and finally they are integrated into a picture library.

[0049] Step S6: After applying the generated picture library to the training of the AI detection model, feedback according to the actual detection results to Step S2 or Step S3 to dynamically optimize the feature extraction rules or generate the model structure.

[0050] Feedback data collection: Design a mechanism to automatically collect the error samples of the model in actual detection, such as misdetection or missed detection samples. At the same time, consider the problem of data annotation, and a semi-automated manual annotation process is required.

[0051] Dynamic adjustment strategy of the picture library: including data augmentation, generating new samples (possibly using GAN) to supplement difficult-to-capture defect types. It may be necessary to train a generation model based on the feedback data to generate more relevant samples.

[0052] Optimization of feature extraction rules: Combining the previous genetic algorithm, online update the feature selection, and adjust the feature parameter combination based on the new data. At the same time, use the SHAP value or LIME method to consider the dynamic evaluation of feature importance.

[0053] Model structure adjustment: Automated Machine Learning (AutoML) technology, Neural Architecture Search (NAS) or model pruning. In addition, integrate an online learning mechanism so that the model can incrementally learn new data without forgetting old knowledge.

[0054] Closed-loop iterative process: The entire system needs to form a closed loop, automating from data collection to model update. Design monitoring metrics to determine whether to retrain the model or adjust the data.

[0055] Regarding the content disclosed in this case, the following points also need to be explained: (1). The attached drawings of the embodiments disclosed in this case only relate to the structures involved in the embodiments disclosed in this case. Other structures can refer to the general design; (2). Without conflict, the embodiments disclosed in this case and the features in the embodiments can be combined with each other to obtain new embodiments; The above are only the specific implementation manners disclosed in this case, but the protection scope of this disclosure is not limited thereto. The protection scope disclosed in this case shall be subject to the protection scope of the claims.

Claims

1. A method for generating a training image library for visual AI detection of defective junction boxes, characterized in that It includes the following steps: Step S1: Collect the image data of the junction box during production; Step S2: Perform image preprocessing, extract the defect features of the junction box, encode these features, and form a multi-dimensional feature array; Step S3: Use an image processing algorithm to generate simulated defective product effect diagrams containing different categories and morphologies according to the defect features extracted in Step S2; Step S4: Form a diverse defective product effect diagram library by combining the parameters of different defect features and iteratively optimizing the generation process.

2. A method for generating a training image library for visual AI detection of defective junction boxes according to claim 1, characterized in that After Step S4 is completed, execute Step S5; Step S5: Verify the generated defective product effect diagrams, determine whether they meet the defect standards in actual production, and adjust the feature extraction or generation model parameters according to the verification results to complete the construction of the library.

3. A method for generating a training image library for visual AI detection of defective junction boxes according to claim 1, characterized in that, The defect feature extraction described in Step S2 includes: Locate the boundary of the solder pad offset through an edge detection algorithm, and calculate the offset angle and the pixel difference from the reference position; Use image segmentation technology to identify the area ratio of the damaged area, and combine morphological analysis to determine the geometric features of the damaged shape; Analyze the RGB color difference between the missing area and the surrounding normal area, and quantify the color change threshold.

4. A method for generating a training image library for visual AI detection of defective junction boxes according to claim 1, characterized in that, After Step S3 generates the simulated defective effect diagram, further perform data enhancement operations on the image, including rotation, scaling, flipping, or color channel transformation, to enhance the diversity of the training library.

5. A method for generating a training image library for visual AI detection of defective junction boxes according to claim 1, characterized in that, The parameters for combining different defect features in Step S4 include: Randomly combine the solder pad offset, damage, and missing features to generate a composite defective effect diagram; Optimize the feature parameter combination through a genetic algorithm to maximize the coverage of the defect scenarios in actual production.

6. A method for generating a training image library for visual AI detection of defective junction boxes according to claim 2, characterized in that, The verification in Step S5 includes: Input the generated defective product effect diagrams into a pre-trained AI detection model, and count the false detection rate and the missed detection rate; If the false detection rate exceeds the preset threshold, adjust the noise parameter or the feature weight distribution of the generation model; If the missed detection rate exceeds the preset threshold, increase the feature extraction dimension of specific defect types.

7. A method for generating a training image library for visual AI detection of defective junction boxes according to claim 2, characterized in that, After Step S5 is completed, execute Step S6; Step S6: After applying the generated library to the training of the AI detection model, feedback to Step S2 or Step S3 according to the actual detection results, and dynamically optimize the feature extraction rules or the generation model structure.

8. A method for generating a training image library for visual AI detection of defective junction boxes according to claim 1, characterized in that, The image data in Step S1 includes the junction box body area and the connector area.

9. A method for generating a training image library for visual AI detection of defective junction boxes according to claim 8, characterized in that, The junction box body area includes the box positioning posts, conductors, solder pads, and diodes for casting diode modules.

10. A method for generating a training image library for visual AI detection of defective junction boxes according to claim 8, characterized in that, The connector area contains an inner ring basket, a sealing ring, and a connecting nut.

Images