A deep learning-based laser welding visual inspection method for junction boxes

Abstract

The application belongs to the technical field of deep learning and welding visual detection, and particularly relates to a kind of terminal box laser welding visual detection methods based on deep learning, its characterized in that: including the following steps: terminal box overall positioning;Training first level deep learning model and corresponding discrimination to detection object;First level image mask processing, first level deep learning model optimization training;Positioning the center coordinates of bus bar;Through the center coordinates, laser welding is carried out;Training second level deep learning model and discriminating detection object;Second level image mask processing;Positioning the center coordinates of scar;Using the center coordinates of scar to cut out the welding scar graph, training third level deep learning model and determining the OK / NG result of welding scar, third level deep learning model can participate in the discrimination of detection object of second level deep learning model at the same time.The application improves the detection efficiency, production efficiency and yield of photovoltaic terminal box laser welding.

Description

Technical Field

[0001] This invention belongs to the fields of deep learning technology and welding visual inspection technology, specifically relating to a deep learning-based visual inspection method for junction box laser welding. Background Technology

[0002] The information disclosed in the background section of this invention is intended only to enhance the understanding of the overall background of the invention and is not necessarily to be construed as an admission or in any way implying that such information constitutes prior art known to those skilled in the art.

[0003] A photovoltaic (PV) junction box is a connecting device between a solar cell array (composed of solar cell modules) and a solar charging control device. Its main function is to connect and protect the solar photovoltaic modules, connect the electricity generated by the solar cells to external power lines, and conduct the current generated by the photovoltaic modules. Therefore, PV junction boxes play a crucial role in the application of solar cell modules. As a connecting device, the structure of a PV junction box includes a box body, cover, connectors, terminals, and diodes. The connectors connect to external electrical equipment, and the terminals connect the photovoltaic module leads (busbars) to the connectors. Therefore, the welding quality between the terminals and the photovoltaic module leads (busbars) significantly affects the quality of photovoltaic power generation, potentially causing open circuits or power loss in the module. Welding inspection has always been a challenge in visual inspection; without accurate and complete welding inspection, a high rate of defective welds can easily occur, and product quality cannot be effectively guaranteed. Summary of the Invention

[0004] To address the aforementioned shortcomings in existing technologies, this invention provides a deep learning-based visual inspection method for laser welding of junction boxes.

[0005] To solve the above-mentioned technical problems, the present invention adopts the following technical solution:

[0006] A deep learning-based visual inspection method for laser welding of junction boxes includes the following steps:

[0007] S1, Junction Box Overall Positioning: Locate the junction box within the entire field of view of the camera and take a screenshot;

[0008] S2, Select the edge region of the busbar in the overall screenshot of the junction box in the previous step, and use the overall position of the junction box and the edge region to train the basic large model to obtain a first-level deep learning model.

[0009] The discrimination module of the first-level deep learning model is used to quickly find the edge region of the busbar and to distinguish the edge regions of the junction box as a whole and the busbar, and output the corresponding signal.

[0010] S3, the edge region of the busbar falling within the set area is reduced and limited by a first-level image mask;

[0011] The first-level image, after being processed by the first-level image mask, is input into the first-level optimized training set to optimize and train the first-level deep learning model.

[0012] S4. Find the edges of the busbars in the image after the first level of scaling down, and then locate the center coordinates of the busbars by the center of the busbar edges;

[0013] S5, laser welding is performed on the busbar by using the center coordinates of the busbar as the position coordinates for laser welding;

[0014] S6. Take a picture of the welded busbar, select the weld scar area in the image and take a screenshot. Use the weld scar area to train the basic large model to obtain the second-level deep learning model.

[0015] The discrimination module of the aforementioned secondary deep learning model is used to quickly locate and discriminate weld scar regions and output corresponding signals.

[0016] S7, The weld scar area falling within the set area is subjected to secondary reduction and limitation through secondary image mask processing;

[0017] The image after secondary image masking is input into the secondary optimization training set to optimize and train the secondary deep learning model.

[0018] S8. Find the edge of the scar in the image after secondary reduction and limitation, and then locate the center coordinate of the scar through the center of the scar edge;

[0019] S9. Based on the center coordinates of the scar, the scar area is cropped to obtain the weld scar image. The weld scar image is used to train the second-level deep learning model to obtain the third-level deep learning model.

[0020] The discrimination module of the three-level deep learning model is used to quickly determine the OK / NG result of the weld spatter. At the same time, in step S6, the three-level deep learning model can be used to make discrimination at the same time as the two-level deep learning model. For objects with a discrimination result of OK, a qualified signal is directly output.

[0021] Image masking processing includes:

[0022] S10, Create Mask: Create a binary image with the same dimensions as the original image of the busbar edge region map or weld spatter region map; set the area to be processed in the mask to 1, and the other areas to 0.

[0023] S20, Apply Mask: Apply a mask to the original image;

[0024] S30, Resulting image: A new image is obtained by combining the original image and the mask, where only the corresponding regions in the mask that are set to 1 are affected by the operation.

[0025] Specifically, the underlying large model is Faster R-CNN, which includes:

[0026] Convolutional Neural Networks: They are used to extract features from images;

[0027] Region Proposal Network: A lightweight fully convolutional neural network used to scan feature maps and efficiently generate candidate regions where a target object may exist;

[0028] ROI Pooling Layer and Discriminant Module: The ROI pooling layer is used to convert proposals of different sizes and shapes into fixed-size feature maps; these fixed-size feature maps are then processed through a series of fully connected layers, which constitute the discriminant module.

[0029] Specifically, the convolutional neural network is ResNet;

[0030] The discrimination module has two outputs: one is a classification output, which represents the probability that each proposed region belongs to each category; the other is a bounding box regression output, which is used to optimize the coordinates of the proposal to more accurately surround the target object.

[0031] Specifically, in step S2, the issues to be identified include: inspection of the junction box and inspection of the busbar;

[0032] The inspection of junction boxes includes: detection of junction box misalignment exceeding the range, and detection of incorrect installation of the positive and negative terminals and their positions.

[0033] The busbar inspection includes: busbar length inspection, busbar twisting inspection, and busbar not being pressed inspection.

[0034] Specifically, in step S6, the issues to be determined include: whether there are bursts in the weld spatter, whether the weld spatter is twisted, and the effective number of weld wires.

[0035] As a preferred approach, the discrimination output signal in S6 and the discrimination output signal in S2 are combined for analysis, and the analysis results are output.

[0036] As a preferred option, the SGD optimization algorithm is used to optimize the first-level deep learning model, the second-level deep learning model, and the third-level deep learning model respectively;

[0037] The image masking process is an image segmentation process used to divide specific regions of an image.

[0038] Preferably, the discrimination output signals in S6 and S2 are sent to the screening control system simultaneously.

[0039] Beneficial effects: This invention improves the ability to quickly find the edge region of the busbar through a first-level deep learning model, speeding up the edge finding efficiency. It also distinguishes the edge regions of the junction box as a whole and the busbar. Once there is a serious positional deviation of the junction box or busbar, the output signal can be directly sent to the screening control system. The screening control system controls the corresponding screening device to remove it, avoiding subsequent wasted operations and time, and improving the welding production efficiency and yield of photovoltaic junction boxes.

[0040] The two-level deep learning system can quickly locate the weld scar area, so as to quickly identify the weld scar area, improve the efficiency of post-weld weld scar positioning and detection, and thus improve the welding production efficiency of photovoltaic junction boxes.

[0041] The three-level deep learning model can quickly determine the OK / NG result of the weld spatter. At the same time, for high-quality weld spatter, while the two-level deep learning model is making a judgment, the three-level deep learning model can make a judgment on the object itself. For objects with an OK judgment result, a qualified signal is directly output, which improves the detection efficiency.

[0042] This invention uses three deep learning models to progressively change the detection objects, gradually detecting the entire junction box, busbars, and weld spatters from large to small, resulting in higher positioning accuracy and more accurate detection results, thereby improving the yield rate of photovoltaic junction box welding. Attached Figure Description

[0043] Figure 1 This is a schematic diagram of the overall method flow of the present invention. Detailed Implementation

[0044] To enable those skilled in the art to better understand the present invention, the technical solution of the present invention will be further described below in conjunction with the accompanying drawings and embodiments.

[0045] Figure 1 A deep learning-based visual inspection method for laser welding of junction boxes is presented, comprising the following steps:

[0046] S1, Junction Box Overall Positioning: Locate the junction box within the entire field of view of the camera and take a screenshot;

[0047] S2, Select the edge region of the busbar in the overall screenshot of the junction box in the previous step, and use the overall position of the junction box and the edge region to train the basic large model to obtain a first-level deep learning model.

[0048] The discrimination module of the first-level deep learning model is used to quickly find the edge region of the busbar and to distinguish the edge regions of the junction box as a whole and the busbar, and output the corresponding signal.

[0049] S3, the edge region of the busbar falling within the set area is reduced and limited by a first-level image mask;

[0050] The first-level image, after being processed by the first-level image mask, is input into the first-level optimized training set to optimize and train the first-level deep learning model.

[0051] S4. Find the edges of the busbars in the image after the first level of scaling down, and then locate the center coordinates of the busbars by the center of the busbar edges;

[0052] S5, laser welding is performed on the busbar by using the center coordinates of the busbar as the position coordinates for laser welding;

[0053] S6. Take a picture of the welded busbar, select the weld scar area in the image and take a screenshot. Use the weld scar area to train the basic large model to obtain the second-level deep learning model.

[0054] The discrimination module of the aforementioned secondary deep learning model is used to quickly locate and discriminate weld scar regions and output corresponding signals.

[0055] S7, The weld scar area falling within the set area is subjected to secondary reduction and limitation through secondary image mask processing;

[0056] The image after secondary image masking is input into the secondary optimization training set to optimize and train the secondary deep learning model.

[0057] S8. Find the edge of the scar in the image after secondary reduction and limitation, and then locate the center coordinate of the scar through the center of the scar edge;

[0058] S9. Based on the center coordinates of the scar, the scar area is cropped to obtain the weld scar image. The weld scar image is used to train the second-level deep learning model to obtain the third-level deep learning model.

[0059] The discrimination module of the three-level deep learning model is used to quickly determine the OK / NG result of the weld spatter. At the same time, in step S6, the three-level deep learning model can perform discrimination simultaneously with the two-level deep learning model. For objects with a discrimination result of OK, a qualified signal is directly output. In other words, the three-level deep learning model can participate in the discrimination of the objects detected by the two-level deep learning model.

[0060] Specifically, the underlying large model is Faster R-CNN, which includes:

[0061] Convolutional Neural Network (CNN): The first part of Faster R-CNN is a convolutional neural network that is used to extract features from an image. These feature maps will be used for subsequent region proposal and object detection.

[0062] Region Proposal Network (RPN): The RPN is the second key component of Faster R-CNN. It uses feature maps previously extracted by the CNN to generate candidate regions where a target object may exist. These candidate regions are called "proposals" or "anchors". The RPN is a fully convolutional network that outputs a set of rectangular proposals, each with a score representing the probability that the region may contain the target.

[0063] ROI Pooling Layer and Discriminant Module: The ROI pooling layer is used to convert proposals of different sizes and shapes into fixed-size feature maps; these fixed-size feature maps are then processed through a series of fully connected layers, which constitute the discriminant module.

[0064] The entire process of Faster R-CNN is as follows:

[0065] 1. Use a convolutional neural network to extract features from the input image;

[0066] 2. Use a Region Proposal Network (RPN) to generate candidate regions on the feature map;

[0067] 3. Use ROI pooling layers to convert these candidate regions into fixed-size feature maps;

[0068] 4. These feature maps are classified and bounding box regressed using the discrimination module.

[0069] The convolutional neural network can be ResNet;

[0070] The discrimination module has two outputs: one is a classification output, which represents the probability that each proposed region belongs to each category; the other is a bounding box regression output, which is used to optimize the coordinates of the proposal to more accurately surround the target object.

[0071] In another embodiment, in step S2, the issues to be determined include: detection of the junction box and detection of the busbar;

[0072] The inspection of the junction box includes:

[0073] Junction box misalignment out-of-range detection: If the corresponding product cannot be found in the corresponding template inspection area, the misalignment is considered to be out of range.

[0074] Junction box polarity and position installation error detection: Detection is performed by checking for mismatch with the template;

[0075] The busbar inspection includes: busbar length inspection, busbar twisting inspection, and busbar not being pressed inspection.

[0076] In another embodiment, the issues to be determined in step S6 include: whether there are weld spatters, whether the weld spatters have bursts, whether the weld spatters are twisted, and the effective number of weld wires.

[0077] In another embodiment, the discrimination output signal in S6 and the discrimination output signal in S2 are combined and analyzed, and the analysis result is output. For example, if the junction box misalignment detection is normal, the busbar length detection and busbar twist detection are also normal, but no weld spatter is detected, it is very likely that the laser welding machine is faulty or abnormal and no laser is emitted. Therefore, a signal of laser welding machine fault or abnormality is output.

[0078] In another embodiment, the SGD optimization algorithm is used to optimize the first-level deep learning model, the second-level deep learning model, and the third-level deep learning model respectively;

[0079] The specific steps of SGD are as follows:

[0080] 1. Initialization: Randomly initialize the model's parameters. This can be done by randomly selecting small values.

[0081] 2. Sample Selection: In standard gradient descent, you would use the entire dataset to compute the gradient of the loss function. However, in SGD, you only randomly select one sample (or a small batch of samples, called a mini-batch) at a time to compute the gradient. This greatly speeds up the computation process.

[0082] 3. Calculate the gradient: Calculate the loss function L for the model parameters. The gradient of the loss function. This gradient is the slope of the loss function with the current parameter settings, and it indicates how we should update the parameters to reduce the loss.

[0083] 4. Update parameters: Update parameters using the following rules:

[0084]

[0085] in It is the learning rate (a positive number) used to control the step size of our parameter updates along the gradient direction. It is the gradient of the loss function with respect to the parameters.

[0086] 5. Repeat: Repeat steps 2-4 until the stopping criteria are met, such as reaching the maximum number of iterations or the change in the loss function is less than a certain threshold.

[0087] Note that a key parameter of SGD is the learning rate. Setting an appropriate learning rate is crucial. If the learning rate is too large, the algorithm may oscillate around the minimum and fail to converge. If the learning rate is too small, convergence may be very slow.

[0088] Regarding learning rate For the method of determining the value, please refer to the following:

[0089] Initial testing: Before training the model, you can conduct preliminary tests using different learning rates (e.g., 0.1, 0.01, 0.001, 0.0001) to observe the model's performance. Based on the model's training loss and validation performance, choose a learning rate that performs well as a starting point.

[0090] Learning rate decay: Gradually reducing the learning rate during training is generally a good strategy. Starting with a higher learning rate can speed up convergence, while gradually decreasing the learning rate as training progresses helps stabilize the training and find finer solutions. This strategy is often referred to as learning rate decay or learning rate scheduling.

[0091] Adaptive learning rate: Some optimization algorithms, such as Adam, Adagrad, and RMSprop, can automatically adjust the learning rate based on historical gradient information. These algorithms typically do not require manually setting a very fine learning rate.

[0092] Using learning rate search techniques: Techniques such as grid search, random search, or Bayesian optimization can be used to search for the optimal learning rate.

[0093] Learning Rate Finder: A technique called learning rate range test or learning rate finder involves gradually increasing the learning rate over a training cycle while monitoring the validation loss. Plotting the learning rate against the loss can help find a suitable learning rate range.

[0094] Typically, the learning rate The value can be selected between 0.1 and 0.0001, and in this embodiment, it can be selected between 0.001 and 0.005.

[0095] The image masking process is an image segmentation process used to divide specific regions of an image.

[0096] In another embodiment, the discrimination output signals in S6 and S2 are simultaneously sent to the screening control system.

[0097] This invention improves the ability to quickly identify the edge region of the busbar using a first-level deep learning model, thus accelerating the edge finding efficiency. It also distinguishes the edge regions of the junction box as a whole and the busbar. If there is a serious positional deviation in the junction box or busbar, the output signal can be directly sent to the screening control system. The screening control system controls the corresponding screening device to remove it, avoiding subsequent wasted operations and time, and improving the welding production efficiency and yield of photovoltaic junction boxes.

[0098] The two-level deep learning system can quickly locate the weld scar area, so as to quickly identify the weld scar area, improve the efficiency of post-weld weld scar positioning and detection, and thus improve the welding production efficiency of photovoltaic junction boxes.

[0099] The three-level deep learning model can quickly determine the OK / NG result of the weld spatter. At the same time, for high-quality weld spatter, while the two-level deep learning model is making a judgment, the three-level deep learning model can make a judgment on the object itself. For objects with an OK judgment result, a qualified signal is directly output, which improves the detection efficiency.

[0100] This invention uses three deep learning models to progressively change the detection objects, gradually detecting the entire junction box, busbars, and weld spatters from large to small, resulting in higher positioning accuracy and more accurate detection results, thereby improving the yield rate of photovoltaic junction box welding.

[0101] The above descriptions are merely embodiments of the present invention. Commonly known structures and characteristics of the solutions are not described in detail here. Those skilled in the art are aware of all common technical knowledge in the field prior to the application date or priority date, are aware of all existing technologies in that field, and have the ability to apply conventional experimental methods prior to that date. Those skilled in the art can, under the guidance of this application, improve and implement this solution in combination with their own capabilities. Some typical known structures or methods should not be obstacles for those skilled in the art to implement this application. It should be noted that those skilled in the art can make several modifications and improvements without departing from the structure of the present invention. These should also be considered within the scope of protection of the present invention, and will not affect the effectiveness of the implementation of the present invention or the practicality of the patent.

Claims

1. A deep learning-based visual inspection method for laser welding of a junction box, characterized by: The method comprises the following steps: S1, overall positioning of the junction box: find the position of the junction box in the entire field of view of the camera and take a screenshot; S2, select the edge region of the busbar in the screenshot of the overall junction box in the previous step, and obtain a first deep learning model by training a basic large model using the overall position of the junction box and the edge region; The discrimination module of the first deep learning model is used to quickly find the edge region of the busbar and discriminate the overall junction box and the edge region of the busbar, and output corresponding signals; S3, the busbar edge region falling within the set region is subjected to first image mask processing for first reduction and limitation; The first image after the first image mask processing is input into a first optimization training set to optimize and train the first deep learning model; S4, find the busbar edge in the image after the first reduction and limitation, and then locate the center coordinates of the busbar through the center of the busbar edge; S5, laser weld the busbar by taking the center coordinates of the busbar as the position coordinates of laser welding; S6, take a photo of the welded busbar, select the weld scar region in the image and take a screenshot, and obtain a second deep learning model by training a basic large model using the weld scar region; The discrimination module of the second deep learning model is used to quickly find the weld scar region and discriminate the weld scar region, and output corresponding signals; S7, the weld scar region falling within the set region is subjected to second image mask processing for second reduction and limitation; The image after the second image mask processing is input into a second optimization training set to optimize and train the second deep learning model; S8, find the edge of the scar in the image after the second reduction and limitation, and then locate the center coordinates of the scar through the center of the scar edge; S9, take a screenshot of the scar region according to the center coordinates of the scar, take a weld scar image, and obtain a third deep learning model by training the second deep learning model using the weld scar image; The discrimination module of the third deep learning model is used to quickly determine the OK / NG result of the weld scar, and simultaneously, the third deep learning model can discriminate simultaneously with the second deep learning model in step S6.

2. The deep learning-based junction box laser welding visual detection method according to claim 1, wherein: The basic large model is a Faster R-CNN, which comprises: a convolutional neural network for extracting features of an image; a region proposal network, which is a lightweight fully convolutional neural network for scanning feature maps and efficiently generating candidate regions where target objects may exist; an ROI pooling layer and a discrimination module, wherein the ROI pooling layer is used to convert proposals of different sizes and shapes into fixed-size feature maps; and the fixed-size feature maps are then processed through a series of fully connected layers, which constitute the discrimination module.

3. The deep learning-based junction box laser welding visual detection method according to claim 2, wherein: The convolutional neural network is a ResNet; The discrimination module has two outputs: one is a classification output, which is used to represent the probability of each proposal region belonging to each category. Secondly, the bounding box regression output is used to optimize the proposed coordinates to more accurately surround the target object.

4. The laser welding visual inspection method for junction boxes based on deep learning according to claim 1, characterized in that: In step S2, the problems to be distinguished include: detection of the junction box and detection of the bus bar; The detection of the junction box includes: detection of the junction box skewing out of range, detection of the positive and negative electrode direction and position installation error of the junction box; The detection of the bus bar includes: detection of the length of the bus bar, detection of the distortion of the bus bar, and detection of the non-pressing of the bus bar.

5. The laser welding visual inspection method for junction boxes based on deep learning according to claim 4, characterized in that: In step S6, the problems to be distinguished include: whether the welding scar has a burst point, whether the welding scar has distortion, and the effective number of welding lines.

6. The laser welding visual inspection method for junction boxes based on deep learning according to claim 5, characterized in that: The discrimination output signals in S6 and S2 are combined and analyzed, and the analysis results are output.

7. The laser welding visual inspection method for junction boxes based on deep learning according to claim 6, characterized in that: The first-level deep learning model, the second-level deep learning model, and the third-level deep learning model are respectively optimized by using the SGD optimization algorithm; The image mask processing is an image segmentation processing for segmenting a specific region of an image.

8. The method of claim 7, wherein the method further comprises: The discrimination output signals in S6 and S2 are sent to the screening control system.

Images