A photovoltaic junction box defect detection method and system based on regional constraint two-stage visual identification
By adopting a two-stage visual recognition method based on regional constraints, the problems of unstable positioning and inaccurate defect identification in photovoltaic module junction box inspection were solved, achieving accurate detection and parameterized quantification of junction boxes, and improving the stability and interpretability of the inspection.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- JOLYWOOD SUZHOU SUNWATT
- Filing Date
- 2026-04-03
- Publication Date
- 2026-07-10
AI Technical Summary
In the current photovoltaic module manufacturing process, junction box inspection suffers from problems such as unstable positioning, inaccurate identification of local defects, insufficient parameter quantification, and weak interpretability of defect judgment. In particular, it is difficult to accurately identify and quantify defects under background complexity, scale changes, and shooting position fluctuations.
A two-stage visual recognition method based on region constraints is adopted. Through one-stage target detection and two-stage instance segmentation, combined with spatial prior constraints, region candidate scoring, deduplication of overlapping similar masks, workstation attribution, and parameterized quantization, accurate detection of junction boxes is achieved.
It improves the stability and accuracy of junction box positioning, can identify camera offset anomalies, enhances the engineering interpretability of defect output, and provides quantitative parameters that can be used for production line process control, supporting quality inspection review and traceability.
Smart Images
Figure CN122367908A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the fields of industrial visual inspection, photovoltaic manufacturing inspection, and intelligent image analysis, and in particular to a method and system for detecting defects in photovoltaic junction boxes based on region-constrained two-stage visual recognition. Background Technology
[0002] During the manufacturing process of photovoltaic modules, the junction box area typically requires inspection of the junction box body, leads, tinned areas, and residual adhesive. Existing inspection methods generally have the following problems:
[0003] When performing single-stage detection directly on the entire image, unstable positioning is easily caused by the complexity of the background, changes in the size of the junction box, and fluctuations in the shooting position.
[0004] The lack of spatial priors in judging defects in local structures makes it difficult to reliably distinguish between left, middle, and right junction boxes, as well as the left and right components within each junction box.
[0005] Multi-class instance segmentation results are prone to issues such as duplicate detection of the same type, misjudgment of quantity, and inconsistency in quantification during engineering deployment.
[0006] The test results only indicate whether there is an abnormality, lacking quantifiable parameters such as width and area for process control.
[0007] Ordinary vision algorithms are weak in perceiving shooting problems such as camera offset, making it difficult to provide timely feedback on the production line status.
[0008] Therefore, it is necessary to provide a photovoltaic junction box inspection scheme that combines spatial prior constraints, two-stage depth recognition, regional candidate scoring, overlapping and deduplication of similar masks, station assignment, parameterized quantization, regularized defect judgment, and inverse mapping fusion to improve the inspection accuracy, stability, and interpretability. Summary of the Invention
[0009] The purpose of this invention is to provide a photovoltaic junction box defect detection method based on regional constraint two-stage visual recognition, so as to solve the problems of unstable junction box positioning, inaccurate local defect identification, insufficient parameter quantification, and weak interpretability of defect judgment in the prior art.
[0010] To achieve the above objectives, the present invention adopts the following technical solution: a photovoltaic junction box defect detection method based on region-constrained two-stage visual recognition, comprising the following steps:
[0011] Step 1: Obtain the original image to be detected. The original image includes at least three photovoltaic junction box imaging areas.
[0012] Step 2: Input the original image into the one-stage target detection model to obtain at least one junction box candidate target. Each candidate target includes at least a bounding box, center coordinates, and confidence score.
[0013] Step 3: Divide the original image into left, middle and right regions according to the original image width, and determine the region to which each candidate target belongs based on the center x-coordinate. Use the region constraint candidate scoring algorithm to select the optimal junction box target in each region. When no valid junction box target is obtained in any region, output the camera shift defect.
[0014] Step 4: Perform boundary expansion and image cropping on each optimal junction box target to obtain multiple local images of the junction box;
[0015] Step 5: Input the local images of each junction box into the two-stage instance segmentation model to obtain the segmentation results of the lead wire target, the tinned target, and the residual adhesive target;
[0016] Step 6: Perform overlap deduplication processing based on the overlap coefficient of the same type of mask for lead-out targets, tinned targets and residual adhesive targets respectively. Retain targets of the same type with larger area or higher confidence, and do not remove the overlap relationship between different types of targets.
[0017] Step 7: Based on the normalized horizontal position of the deduplicated lead wire target and the tinning target in the local image of each junction box, execute the left and right station assignment algorithm to assign them as the left station target and the right station target, respectively.
[0018] Step 8: Calculate the width of each lead-out target, the area of each tinned target, and the total area of residual adhesive targets in each junction box local image based on the segmentation results;
[0019] Step 9: Compare the width and area with the preset thresholds, and output at least one of the following defects: missing lead, insufficient soldering, lead too short, lead too long, no soldering, and residual adhesive.
[0020] Step 10: Perform inverse mapping fusion on the junction box positioning results, segmentation results, quantification indicators, and defect results according to the cropping offset corresponding to each local image to generate the detection result image.
[0021] Among them, based on the center x-coordinate of the candidate target With the original image width The region to which the candidate target belongs is determined according to the following formula:
[0022]
[0023] in, Indicates the left region. Indicates the middle region, Indicates the right region;
[0024] The region constraint candidate scoring algorithm calculates the candidate target score according to the following formula:
[0025]
[0026] in Indicates the first Confidence level of each candidate target Indicates the first The area of each candidate target This represents the maximum area of candidate targets within the same region. Indicates the x-coordinate of the candidate target center. The x-coordinate of the center of the region to which the candidate target belongs is represented, and W represents the width of the original image. This represents the width of a single region after dividing the original image into three equal parts, used to normalize the lateral offset of the candidate target center relative to the center of its respective region. , and is the weighting coefficient; the candidate target with the highest score in each region is selected as the optimal junction box target.
[0027] Among them, the deduplication process for overlapping elements of the same type includes:
[0028] Convert targets of the same type into binary masks;
[0029] Sort similar targets by area and confidence level from largest to smallest;
[0030] Calculate the mask overlap coefficient between the current target and the previously retained targets of the same type, using the following formula:
[0031]
[0032] in, and These represent binary masks for two targets of the same type. and These represent the effective number of pixels in the corresponding mask; when the mask overlap coefficient is greater than or equal to the preset overlap threshold, the target with the larger area is retained; when the areas are the same, the target with the higher confidence is retained; when the mask overlap coefficient is less than the preset overlap threshold, the current target is retained.
[0033] Overlapping is allowed between different types of targets, and lead-out targets, tinning targets, and residual adhesive targets are independently involved in subsequent quantification and defect determination.
[0034] Specifically, each junction box local image is divided into a left half and a right half according to its width, and workstation assignment is performed based on the normalized ratio of the target center's horizontal coordinate to the local image width. The normalized ratio is calculated according to the following formula:
[0035]
[0036] in, Indicates the x-coordinate of the target center. Indicates the first The width of a partial image of a junction box; when When the target is assigned to the left workstation target, The target will then be assigned to the right workstation.
[0037] The width of the lead-out target is calculated using the span of its segmented mask in the lateral direction, as shown in the following formula:
[0038]
[0039] in, This represents the maximum x-coordinate of the leader line segmentation mask. The minimum x-coordinate of the leader line segmentation mask;
[0040] The tinned area and residual adhesive area are calculated based on the number of effective pixels in the corresponding segmentation mask.
[0041] The preset thresholds include at least the minimum lead width threshold, the maximum lead width threshold, the minimum tinning area threshold, and the maximum residual adhesive area threshold, and the thresholds can be configured according to the camera resolution, inspection station, and product specifications.
[0042] The defect determination includes a parameterized defect determination matrix based on the workstation index, and the defect determination matrix includes at least:
[0043] When no lead wire is detected in the left or right workstation of a junction box in a partial image, it is determined that a lead wire is missing.
[0044] When no soldering target is identified in the left or right workstation of a junction box in a partial image, it is determined to be lacking solder.
[0045] When the width of the lead wire is less than the minimum width threshold of the lead wire, it is determined that the lead wire is too short;
[0046] When the width of the lead wire exceeds the maximum width threshold of the lead wire, it is determined that the lead wire is too long;
[0047] When the tinning area is less than the minimum tinning area threshold, it is determined to be untinned;
[0048] When the total area of residual adhesive exceeds the maximum area threshold of residual adhesive, it is determined that there is residual adhesive.
[0049] The inverse mapping fusion includes: recording the cropping offset of each junction box local image relative to the original image. The bounding box coordinates, segmentation mask coordinates, and defect annotation coordinates in the local image are then mapped back to the original image coordinate system using the following formula:
[0050]
[0051] in, Represents the coordinates of any pixel or labeled area in a local image. This represents the coordinates of the original image after mapping.
[0052] A photovoltaic junction box defect detection system includes: an image acquisition module, a region constraint candidate scoring module, a local cropping module, a two-stage segmentation module, a same-type mask overlap deduplication module, a workstation assignment module, a parameter calculation module, a defect judgment module, and an inverse mapping fusion output module. The image acquisition module acquires the original image; the region constraint candidate scoring module performs region assignment and joint scoring filtering on candidate targets for the junction box; the local cropping module generates a local image of the junction box; the two-stage segmentation module performs instance segmentation on leads, tinning, and residual adhesive; the same-type mask overlap deduplication module performs overlap deduplication based on mask overlap coefficients on targets of the same type; the workstation assignment module assigns lead targets and tinning targets to the left and right workstations of the local image; the parameter calculation module calculates the lead width, tinning area, and residual adhesive area; the defect judgment module outputs defect results according to a configured threshold; and the inverse mapping fusion output module maps and overlays the results onto the original image and outputs the detection result image.
[0053] Compared with the prior art, the present invention has at least the following beneficial effects:
[0054] 1. This solution improves the stability of overall junction box positioning and detailed defect identification through a two-stage structure of "one-stage whole-map positioning and two-stage local fine segmentation".
[0055] 2. This scheme incorporates the spatial distribution of junction boxes into the detection process by combining left, middle and right three-part area constraints with candidate target joint scoring, and can directly identify camera displacement anomalies caused by shooting offset.
[0056] 3. This scheme uses a local image left and right workstation attribution algorithm to bind structural recognition with workstation semantics, thereby improving the engineering interpretability of defect output;
[0057] 4. This scheme uses a deduplication algorithm based on mask overlap coefficients to suppress duplicate identification of the same type while preserving the cross-type overlap relationships that may exist between lead wires, soldering, and residual adhesive in the actual process.
[0058] 5. This scheme quantifies lead wire width, tinning area, and residual adhesive area and introduces them into a parameterized defect judgment matrix, so that the detection results can be used for production line process control, rather than just for simple sorting;
[0059] 6. By using the inverse mapping fusion algorithm, the local segmentation results, quantization results, and defect labels are written back to the whole image, which facilitates quality inspectors to review, archive, and trace. Attached Figure Description
[0060] Figure 1 Diagram of dual-stage visual recognition technology for photovoltaic junction box defect detection.
[0061] Figure 2 Application diagram of two-stage visual recognition for photovoltaic junction box defect detection.
[0062] Figure 3 Deployment diagram for two-stage visual recognition photovoltaic junction box defect detection. Detailed Implementation
[0063] The present invention will be further described below with reference to the accompanying drawings and embodiments, but the scope of protection of the present invention is not limited to the following embodiments.
[0064] Example 1: See Figures 1-3 A defect detection method for photovoltaic junction boxes based on region-constrained two-stage visual recognition.
[0065] Overall process:
[0066] In this embodiment, an industrial camera captures a raw image containing three photovoltaic junction box areas. After receiving the image, the industrial computer performs the following steps.
[0067] Step S101 Acquisition of the original image
[0068] Obtain the original image to be detected. Its width is The height is .
[0069] Step S102: First-stage junction box positioning
[0070] Original image Inputting a one-stage target detection model yields several candidate target sets for junction boxes:
[0071]
[0072] Each candidate target should include at least the bounding box, center coordinates, confidence level, and area information.
[0073] Step S103 Region Constraint Filtering
[0074] The original image is divided into three regions based on its width:
[0075] Left region:
[0076] Central region:
[0077] Right region:
[0078] For each candidate target According to its central x-coordinate Calculate the region:
[0079]
[0080] Within each region, a joint score is calculated for the candidate targets:
[0081]
[0082] The junction box with the highest score is selected as the optimal target for the region. If there is no valid target in any region, the camera displacement defect of the corresponding region is output.
[0083] The technical advantage of this step is that even if the first-stage model outputs multiple candidate targets, it can combine the fixed spatial distribution pattern of junction boxes in the actual production line to impose engineering constraints on the results and improve positioning stability.
[0084] Step S104: Partial cropping
[0085] The bounding box of the optimal target in each region is expanded to obtain the expanded bounding box, and the corresponding local junction box image is cropped from the original image:
[0086]
[0087] in, These correspond to partial images of the left, middle, and right junction boxes, respectively.
[0088] Step S105: Two-stage instance segmentation
[0089] Each local image Input a two-stage instance segmentation model and output three target classes:
[0090] Lead-out target set
[0091] Tin-based target set
[0092] Residual adhesive target set
[0093] Each target should include at least an instance mask, bounding box, center coordinates, and confidence score.
[0094] Step S106: Remove duplicates of the same type.
[0095] Deduplication is performed independently for each type of target in each local image. Take the leader line target as an example:
[0096] Sets based on area and confidence level Target sorting in the middle;
[0097] The mask overlap coefficient is calculated between the sorted targets and the retained targets in turn.
[0098] For any two masks of the same type The overlap coefficient is calculated using the following formula:
[0099]
[0100] If the overlap coefficient is greater than or equal to the preset overlap threshold, then targets with smaller areas or lower confidence levels are deleted.
[0101] If the overlap coefficient is less than the preset overlap threshold, the current target is retained.
[0102] The same strategy is used for both the tin-treated target set and the residual adhesive target set. It is important to emphasize that:
[0103] Remove duplicate leads from each other;
[0104] Remove duplicates between tin-plated products;
[0105] Remove excess adhesive from each other;
[0106] Lead wires, tinning, and residual adhesive are not removed in a mutually exclusive manner.
[0107] This design avoids artificially high numbers of similar items due to repeated model segmentation, while preserving the real relationship that different process phenomena may coexist on the image.
[0108] Step S107: Assignment of left and right workstations
[0109] For each local image Let its width be Divide it into a left half and a right half, and determine the target center x-coordinate. Calculate the normalized horizontal coordinate:
[0110]
[0111] when When returning to the left workstation; The time is returned to the right workstation.
[0112] The lead-out wire and tinning processes are assigned to left and right workstations independently. If a workstation is missing its corresponding target, it can be identified as a defect.
[0113] Step S108 Parameter Quantization
[0114] Lead width
[0115] For any lead-out mask Its width is defined as the horizontal span:
[0116]
[0117] Tinning area
[0118] For any tinning mask Its area is defined as the number of effective pixels:
[0119]
[0120] Residual adhesive area
[0121] For the set of residual adhesive targets in a local image The total residual adhesive area is:
[0122]
[0123] Step S109 Defect Rule Determination
[0124] Set configurable thresholds:
[0125] Minimum width threshold for lead wires
[0126] Maximum width threshold of lead wire
[0127] Minimum area threshold for tinning
[0128] Maximum area threshold of residual adhesive
[0129] The decision logic is as follows:
[0130] If there is no target for the lead wire at the left or right workstation, it is determined that the lead wire is missing.
[0131] If the left or right workstation is found to be without solder, it is considered to be in a state of solder shortage.
[0132] If the lead wire width If so, it is determined that the lead wire is too short;
[0133] If the lead wire width If so, it is determined that the lead wire is too long;
[0134] If the tinning area If so, it is determined to be untinned;
[0135] If the total area of residual glue If so, it is determined that there is residual glue.
[0136] Step S110: Inverse mapping and superposition output
[0137] The segmentation contours, workstation labels, width / area values, and defect text from each local image are inversely mapped back to the original image using a cropping offset, generating a result image with detection boxes, segmented regions, and defect information. For any coordinates in the local image... Based on local clipping offset Write it back using the following formula:
[0138]
[0139] This ensures that the local recognition results are strictly aligned with the coordinate system of the original image.
[0140] It should be noted that the above embodiments are not intended to limit the scope of protection of the present invention. Equivalent transformations or substitutions made based on the above technical solutions all fall within the scope of protection of the claims of the present invention.
Claims
1. A method for detecting defects in photovoltaic junction boxes based on region-constrained two-stage visual recognition, characterized in that, Includes the following steps: Step 1: Obtain the original image to be detected. The original image includes at least three photovoltaic junction box imaging areas. Step 2: Input the original image into the one-stage target detection model to obtain at least one junction box candidate target. Each candidate target includes at least a bounding box, center coordinates, and confidence score. Step 3: Divide the original image into left, middle and right regions according to the original image width, and determine the region to which each candidate target belongs based on the center x-coordinate. Use the region constraint candidate scoring algorithm to select the optimal junction box target in each region. When no valid junction box target is obtained in any region, output the camera shift defect. Step 4: Perform boundary expansion and image cropping on each optimal junction box target to obtain multiple local images of the junction box; Step 5: Input the local images of each junction box into the two-stage instance segmentation model to obtain the segmentation results of the lead wire target, the tinned target, and the residual adhesive target; Step 6: Perform overlap deduplication processing based on the overlap coefficient of the same type of mask for lead-out targets, tinned targets and residual adhesive targets respectively. Retain targets of the same type with large area and high confidence, and do not remove the overlap relationship between different types of targets. Step 7: Based on the normalized horizontal position of the deduplicated lead wire target and the tinning target in the local image of each junction box, execute the left and right station assignment algorithm to assign them as the left station target and the right station target, respectively. Step 8: Calculate the width of each lead-out target, the area of each tinned target, and the total area of residual adhesive targets in each junction box local image based on the segmentation results; Step 9: Compare the width and area with the preset thresholds, and output at least one of the following defects: missing lead, insufficient soldering, lead too short, lead too long, no soldering, and residual adhesive. Step 10: Perform inverse mapping fusion on the junction box positioning results, segmentation results, quantification indicators, and defect results according to the cropping offset corresponding to each local image to generate the detection result image.
2. The photovoltaic junction box defect detection method based on region-constrained two-stage visual recognition according to claim 1, characterized in that, Based on the center x-coordinate of the candidate target With the original image width The region to which the candidate target belongs is determined according to the following formula: in, Indicates the left region. Indicates the middle region, Indicates the right region; The region constraint candidate scoring algorithm calculates the candidate target score according to the following formula: in Indicates the first Confidence level of each candidate target Indicates the first The area of each candidate target. This represents the maximum area of candidate targets within the same region. Indicates the x-coordinate of the candidate target center. The x-coordinate of the center of the region to which the candidate target belongs is represented, and W represents the width of the original image. This represents the width of a single region after dividing the original image into three equal parts, used to normalize the lateral offset of the candidate target center relative to the center of its respective region. , and is the weighting coefficient; the candidate target with the highest score in each region is selected as the optimal junction box target.
3. The photovoltaic junction box defect detection method based on region-constrained two-stage visual recognition according to claim 2, characterized in that, The same type of overlapping deduplication process includes: Convert targets of the same type into binary masks; Sort similar targets by area and confidence level from largest to smallest; Calculate the mask overlap coefficient between the current target and the previously retained targets of the same type, using the following formula: in, and These represent binary masks for two targets of the same type. and These represent the effective number of pixels in the corresponding mask; when the mask overlap coefficient is greater than or equal to the preset overlap threshold, the target with the larger area is retained; when the areas are the same, the target with the higher confidence is retained; when the mask overlap coefficient is less than the preset overlap threshold, the current target is retained. Overlapping is allowed between different types of targets, and lead-out targets, tinning targets, and residual adhesive targets are independently involved in subsequent quantification and defect determination.
4. The photovoltaic junction box defect detection method based on region-constrained two-stage visual recognition according to claim 3, characterized in that, Each junction box local image is divided into a left half and a right half according to its width. Workstation assignment is performed based on the normalized ratio of the target center's horizontal coordinate to the local image width. This normalized ratio is calculated using the following formula: in, Indicates the x-coordinate of the target center. Indicates the first The width of a partial image of a junction box; when When the target is assigned to the left workstation target, The target will then be assigned to the right workstation.
5. The photovoltaic junction box defect detection method based on region-constrained two-stage visual recognition according to claim 4, characterized in that, The width of the lead-out target is calculated using the span of its segmented mask in the lateral direction, as shown in the formula: in, This represents the maximum x-coordinate of the leader line segmentation mask. The minimum x-coordinate of the leader line segmentation mask; The tinned area and residual adhesive area are calculated based on the number of effective pixels in the corresponding segmentation mask.
6. The photovoltaic junction box defect detection method based on region-constrained two-stage visual recognition according to claim 5, characterized in that, The preset thresholds include at least the minimum lead width threshold, the maximum lead width threshold, the minimum tinning area threshold, and the maximum residual adhesive area threshold, and the thresholds can be configured according to the camera resolution, inspection station, and product specifications.
7. The photovoltaic junction box defect detection method based on region-constrained two-stage visual recognition according to claim 6, characterized in that, The defect determination includes a parameterized defect determination matrix based on the workstation index, and the defect determination matrix includes at least: When no lead wire is detected in the left or right workstation of a junction box in a partial image, it is determined that a lead wire is missing. When no soldering target is identified in the left or right workstation of a junction box in a partial image, it is determined to be lacking solder. When the width of the lead wire is less than the minimum width threshold of the lead wire, it is determined that the lead wire is too short; When the width of the lead wire exceeds the maximum width threshold of the lead wire, it is determined that the lead wire is too long; When the tinning area is less than the minimum tinning area threshold, it is determined to be untinned; When the total area of residual adhesive exceeds the maximum area threshold of residual adhesive, it is determined that there is residual adhesive.
8. The photovoltaic junction box defect detection method based on region-constrained two-stage visual recognition according to claim 7, characterized in that, The inverse mapping fusion includes: recording the cropping offset of each junction box local image relative to the original image. The bounding box coordinates, segmentation mask coordinates, and defect annotation coordinates in the local image are then mapped back to the original image coordinate system using the following formula: in, Represents the coordinates of any pixel or labeled area in a local image. This represents the coordinates of the original image after mapping.
9. A photovoltaic junction box defect detection system, characterized in that, To implement the photovoltaic junction box defect detection method based on region-constrained two-stage visual recognition as described in any one of claims 1-8, the system comprises: an image acquisition module, a region-constrained candidate scoring module, a local cropping module, a two-stage segmentation module, a same-type mask overlap deduplication module, a workstation assignment module, a parameter calculation module, a defect judgment module, and an inverse mapping fusion output module; wherein, the image acquisition module is used to acquire the original image, the region-constrained candidate scoring module is used to perform region assignment and joint scoring screening on candidate junction box targets, and the local cropping module is used to generate... The junction box partial image includes a two-stage segmentation module for instance segmentation of lead wires, tinning, and residual adhesive; a same-type mask overlap deduplication module for performing overlap deduplication based on mask overlap coefficients on same-type targets; a station assignment module for assigning lead wire targets and tinning targets to the left and right stations of the partial image; a parameter calculation module for calculating lead wire width, tinning area, and residual adhesive area; a defect judgment module for outputting defect results according to a configured threshold; and an inverse mapping fusion output module for mapping and superimposing the results onto the original image and outputting the detection result image.
10. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the photovoltaic junction box defect detection method based on region-constrained two-stage visual recognition as described in any one of claims 1 to 8.