Weld mark labeling method, device, equipment and storage medium

Abstract

The application provides a solder joint marking method, device and equipment and a storage medium. The method comprises the following steps: cutting a to-be-marked solder joint image by using a fixed-size sliding window to obtain an initial segmentation sub-image with a preset overlap rate; removing an illumination component of the initial segmentation sub-image by using an image processing theory algorithm to obtain an illumination normalized sub-image; performing spatial pyramid downsampling and primary solder joint marking on the illumination normalized sub-image to obtain a first marked sub-image carrying a confidence level; screening a first target sub-image and a second target sub-image to be secondarily marked from the first marked sub-image through the confidence level; performing a multi-inflation rate convolution operation, a channel-space attention cascade, small target detection and secondary solder joint marking on the second target sub-image to obtain a second marked sub-image; and determining a solder joint marking result of the to-be-marked solder joint image through the second marked sub-image and the first target sub-image, so as to improve the efficiency, recall rate and accuracy of solder joint marking.

Description

Technical Field

[0001] This application relates to the field of computer vision technology, and in particular to a method, apparatus, device and storage medium for marking solder joints. Background Technology

[0002] Current mainstream annotation methods for small target detection in high-resolution images typically rely on direct manual annotation of the original high-resolution images. While this method is feasible in general scenarios, its limitations become particularly prominent when faced with complex industrial vision tasks such as weld spatter detection.

[0003] During solder spatter detection, the spattered solder joints exhibit a wide distribution area and irregular spatial arrangement in the image. Furthermore, these joints contain numerous tiny, often irregularly shaped, independent spatter points, posing a significant challenge to the annotation process. In addition, high-resolution images of solder spatter are prone to localized overexposure or uneven brightness due to the reflective metallic background, further weakening the contrast between the tiny spatter targets and the background. Therefore, annotators must continuously identify a large number of extremely small targets in a noisy, highly reflective, and complex background. This high-intensity, high-precision visual search and localization task easily leads to visual fatigue and decreased attention, resulting not only in low annotation efficiency but, more seriously, a high rate of missed and incorrect annotations. Summary of the Invention

[0004] In view of this, this application aims to propose a method, apparatus, equipment, and storage medium for solder joint marking, to solve the problem that current manual marking of solder joint spatter not only leads to low marking efficiency, but more seriously, it also causes a high rate of missed marking and incorrect marking. The specific technical solution is as follows: According to a first aspect of this application, a solder joint marking method is provided, the method comprising: A fixed-size sliding window is used to cut the image of the solder joint to be labeled, resulting in an initial segmented sub-image with a preset overlap rate; The illumination component of the initial segmented sub-image is removed using image processing theory and algorithms to obtain an illumination-normalized sub-image; The illumination-normalized sub-image is subjected to spatial pyramid downsampling and initial solder joint annotation to obtain a first annotated sub-image carrying confidence. The first target sub-image and the second target sub-image to be further annotated are selected from the first labeled sub-image based on the confidence level. The second target sub-image is subjected to multi-dilation rate convolution, channel-spatial attention cascade, small target detection, and secondary annotation of solder joints to obtain the second annotated sub-image. The solder joint annotation result of the solder joint image to be annotated is determined by the second annotated sub-image and the first target sub-image.

[0005] Optionally, the step of using image processing theory algorithms to remove the illumination component of the initial segmented sub-image to obtain an illumination-normalized sub-image further includes: The pixel values ​​of the initial segmented sub-image are obtained, and the multiplication relationship between the pixel values ​​and the illumination and reflection components of the initial segmented sub-image is determined using image processing theory algorithms. The multiplication relation is converted into an addition relation in the logarithmic field; Gaussian blur is applied to the initial segmented sub-image to estimate the illumination components of the initial segmented sub-image; The reflection component of the initial segmented sub-image is estimated using the additive relationship and the illumination component. The new pixel values ​​of the initial segmented sub-image are determined by the reflection component of the initial segmented sub-image; The illumination-normalized sub-image is obtained by using the new pixel values ​​of the initial segmented sub-image.

[0006] Optionally, before performing spatial pyramid downsampling and initial solder joint annotation on the illumination-normalized sub-image to obtain the first annotated sub-image carrying confidence, the method further includes: Collect several original images of the solder joints to be labeled; For any original solder joint image to be labeled, the solder joints in the original solder joint image to be labeled are labeled to obtain the first solder joint labeling information; Obtain the first target detection model by introducing the spatial pyramid downsampling module; The original image of the solder joint to be labeled is used as input, and the first solder joint labeling information is used as the training target to train the first target detection model to obtain a coarse-grained labeling model. The coarse-grained labeling model is used to perform spatial pyramid downsampling of the image and initial labeling of the solder joints.

[0007] Optionally, before performing spatial pyramid downsampling and initial solder joint annotation on the illumination-normalized sub-image to obtain the first annotated sub-image carrying confidence, the method further includes: Obtain the size of the illumination-normalized sub-image; The non-maximum suppression cross-union ratio threshold of the coarse-grained annotation model is dynamically adjusted based on the stated size.

[0008] Optionally, the step of performing spatial pyramid downsampling and initial solder joint annotation on the illumination-normalized sub-image to obtain a first annotated sub-image carrying confidence scores includes: The illumination-normalized sub-image is input into the coarse-grained annotation model to perform spatial pyramid downsampling and initial annotation of solder joints, thereby determining candidate annotation boxes and the targets to which the solder joints within the candidate annotation boxes belong. Based on the target to which the solder joint belongs within the candidate annotation box, the candidate annotation boxes are grouped to obtain several candidate annotation box sets; For any set of candidate annotation boxes, obtain the target confidence level of the target to which the solder joint belongs in each candidate annotation box in the set, and filter out the first candidate annotation box and several second candidate annotation boxes based on the target confidence level; Obtain the intersection-union ratio (IUU) of the first candidate bounding box and each second candidate bounding box; By comparing the intersection-union ratio (CUNR) with the non-maximum suppression CUNR threshold, the second candidate bounding box to be deleted is determined. The illumination-normalized sub-image is labeled using the deleted second candidate label box and the first candidate label box to obtain the first labeled sub-image.

[0009] Optionally, the step of filtering the first target sub-image and the second target sub-image to be re-annotated from the first labeled sub-image based on the confidence level further includes: If the confidence level is less than the first confidence threshold, then delete the first labeled sub-image; If the confidence level is greater than the second confidence threshold, then the first labeled sub-image is determined to be the first target sub-image; If the confidence level is greater than or equal to the first confidence threshold and less than or equal to the second confidence threshold, then the first labeled sub-image is determined to be the second target sub-image to be labeled again.

[0010] Optionally, after filtering the first target sub-image and the second target sub-image to be re-annotated from the first labeled sub-image using the confidence level, the process includes: The original image of the weld points to be labeled is divided into equal parts to obtain original sub-images with a preset overlap rate; For any original sub-image, the solder joints in the original sub-image are labeled to obtain the second solder joint labeling information; Obtain a second object detection model that incorporates a receptive field module, a convolutional block attention module, and a second pyramid-level detection layer; The original sub-image is used as input, and the second solder joint annotation information is used as the training target to train the second target detection model, thereby obtaining a fine-grained annotation model. The fine-grained annotation model is used to perform multi-dilation rate convolution operation, channel-spatial attention cascade, small target detection, and secondary annotation of solder joints on the image.

[0011] According to a second aspect of this application, a solder joint marking device is provided, the device comprising: The first cutting module is used to cut the image of the weld point to be marked using a sliding window of a fixed size to obtain an initial segmented sub-image with a preset overlap rate; The illumination component removal module is used to remove the illumination component of the initial segmented sub-image using image processing theoretical algorithms, so as to obtain the illumination-normalized sub-image. The initial annotation module is used to perform spatial pyramid downsampling and initial annotation of solder joints on the illumination normalized sub-image to obtain a first annotated sub-image carrying confidence. The filtering module is used to filter out a first target sub-image and a second target sub-image to be further labeled from the first labeled sub-image based on the confidence level. The secondary annotation module is used to perform multi-dilation rate convolution operation, channel-spatial attention cascade, small target detection and secondary annotation of solder joints on the second target sub-image to obtain the second annotated sub-image; The determination module is used to determine the solder joint annotation result of the solder joint image to be annotated by using the second annotation sub-image and the first target sub-image.

[0012] According to another aspect of this application, an electronic device is also provided, comprising: processor; Memory used to store the processor's executable instructions; The processor is configured to execute the instructions to implement the solder joint marking method as described above.

[0013] According to another aspect of this application, a computer-readable storage medium is also provided, wherein instructions are stored therein, which, when executed on a computer, cause the computer to implement any of the solder joint marking methods of the first aspect described above.

[0014] The solder joint annotation method provided in this application firstly uses a fixed-size sliding window to segment the image of the solder joints to be annotated, obtaining an initial segmented sub-image with a preset overlap rate. This operation makes tiny solder joints, which originally occupied only a few pixels in the entire image of the solder joints to be annotated, relatively prominent targets in the sub-image, greatly improving the effective resolution and feature richness of the targets. The introduction of the preset overlap rate effectively avoids the solder joints from being segmented or lost due to being at the cutting boundary, thus eliminating the problem of missed detection caused by this. Using image processing theory algorithms, the illumination component of the sub-image is removed to obtain an illumination-normalized sub-image, effectively suppressing local overexposure or uneven brightness caused by reflection, avoiding missed solder joints due to illumination interference. The illumination-normalized sub-image is then subjected to spatial pyramid downsampling and initial solder joint annotation to obtain a first annotated sub-image carrying confidence. The application of spatial pyramid downsampling improves the detection capability of solder joints. The confidence is used to filter out the first target sub-image and the second target sub-image to be annotated a second time from the first annotated sub-image. This method not only quickly preserves high-quality annotation results to improve efficiency, but also accurately identifies difficult samples requiring secondary annotation. The second target sub-image undergoes multi-dilation rate convolution, channel-spatial attention cascade, small target detection, and secondary solder joint annotation to obtain a second annotated sub-image. The multi-dilation rate convolution enhances multi-scale perception of solder joints of different sizes, the channel-spatial attention cascade effectively suppresses metal reflection interference, focuses on key spatial regions, and achieves adaptive feature fusion. Small target detection improves the detection capability of solder joints in the image. The solder joint annotation result is determined by using the second annotated sub-image and the first target sub-image. In summary, this application solves the problems of low efficiency, high missed annotation rate, and high mis-annotation rate in manual annotation caused by complex conditions such as extremely small and numerous spatter points, irregular shapes, and high background reflectivity and noise, thus improving the efficiency, recall, and accuracy of solder joint annotation.

[0015] The above description is only an overview of the technical solution of this application. In order to better understand the technical means of this application and to implement it in accordance with the contents of the specification, and to make the above and other objects, features and advantages of this application more obvious and understandable, specific embodiments of this application are given below. Attached Figure Description

[0016] Various other advantages and benefits will become apparent to those skilled in the art upon reading the following detailed description of preferred embodiments. The accompanying drawings are for illustrative purposes only and are not intended to limit the scope of this application. Furthermore, the same reference numerals denote the same parts throughout the drawings. In the drawings: Figure 1 This is a flowchart illustrating the steps of a solder joint marking method provided in this application; Figure 2 yes Figure 1 The flowchart shown is a dual-model screening process in a solder joint annotation method provided in this application. Figure 3 This is a schematic diagram of the structure of a solder joint marking device provided in this application; Figure 4 This is a schematic diagram of the structure of an electronic device provided in this application. Detailed Implementation

[0017] To make the objectives, technical solutions, and advantages of this application clearer, the various embodiments of this application will be described in detail below with reference to the accompanying drawings. However, those skilled in the art will understand that many technical details have been provided in the various embodiments of this application to facilitate a better understanding of the application. However, the technical solutions claimed in this application can be implemented even without these technical details and with various variations and modifications according to the following embodiments. The division of the various embodiments below is for ease of description and should not constitute any limitation on the specific implementation of this application. The various embodiments can be combined with and referenced by each other without contradiction.

[0018] Currently, the main annotation method for small target detection in high-resolution images is direct full-image annotation. However, manually annotating the original high-resolution image requires repeated zooming to check details, which is time-consuming and prone to omissions or inaccuracies. Furthermore, for complex industrial vision tasks such as solder spatter detection, other problems arise, such as the large and irregular distribution area of ​​solder spatter, the small and numerous individual spatters with irregular shapes, the reflective nature of the metal background, and the curved or L-shaped surfaces of some detection areas causing blurring and defocusing of spatter points in certain areas of the image. These factors further contribute to omissions or inaccuracies. Based on these problems, this application proposes a solder spatter annotation method. (Refer to...) Figure 1 The diagram illustrates a flowchart of a solder joint marking method provided in this application, the method comprising: Step 101: Use a sliding window of fixed size to cut the image of the solder joint to be labeled to obtain an initial segmented sub-image with a preset overlap rate.

[0019] This application employs a fixed-size sliding window to segment the image of the solder joints to be annotated. First, a fixed-size n×n window is defined and slides across the original image from left to right and from top to bottom with a specified step size. After each slide, the image region below the window is extracted as an initial segmented sub-image. This application requires a preset overlap rate between the initial segmented sub-images to ensure the integrity of cross-boundary targets and avoid missed detections caused by segmentation. Therefore, a preset overlap rate needs to be set, for example, to 10% (this value can be adjusted adaptively according to requirements). Then, the step size = n × (1 - preset overlap rate). For example, when n = 100 and the preset overlap rate = 10%, the step size = 100 × 0.9 = 90. Using a fixed-size sliding window to segment the image of the solder joints to be annotated allows for the removal of a large amount of irrelevant background information in subsequent processes, reducing computational resources.

[0020] It should be understood that because the image of the solder joints to be labeled is segmented, the total number of pixels in the initial segmented sub-images is less than the total number of pixels in the image of the solder joints to be labeled, meaning the pixel count of the sub-images is reduced. However, in order to better preserve the shape, edges, and other key features of the solder joint spatter in the sub-images, the clarity of the sub-images will not be reduced.

[0021] Step 102: Use image processing theory and algorithms to remove the illumination component of the initial segmented sub-image to obtain the illumination-normalized sub-image.

[0022] This application considers that solder joint spatter images, due to the reflective metal background, are prone to local overexposure or uneven brightness during imaging. Therefore, image processing algorithms are used to eliminate the problem of local overexposure or uneven brightness caused by metal reflection, resulting in an illumination-normalized sub-image. Thus, step 102 specifically includes the following sub-steps: Sub-step 1021: Obtain the pixel values ​​of the initial segmented sub-image, and use image processing theory algorithms to determine the multiplication relationship between the pixel values ​​and the illumination and reflection components of the initial segmented sub-image.

[0023] Sub-step 1022 converts the multiplication relation into an addition relation in the logarithmic field.

[0024] Sub-step 1023: Perform Gaussian blur operation on the initial segmented sub-image to estimate the illumination components of the initial segmented sub-image.

[0025] Sub-step 1024 estimates the reflection component of the initial segmented sub-image using additive relationships and illumination components.

[0026] Sub-step 1025: Determine the new pixel values ​​of the initial segmented sub-image using the reflection component of the initial segmented sub-image.

[0027] Sub-step 1026: Obtain the illumination-normalized sub-image using the new pixel values ​​of the initially segmented sub-image.

[0028] The image processing algorithm in question refers to the Retinex algorithm, which determines the multiplicative relationship between pixel values ​​and the illumination and reflection components in the initial segmented sub-image. The multiplicative relationship between pixel values ​​and the illumination and reflection components of the initial segmented sub-image is: pixel value = illumination component × reflection component, where pixel value can be represented as I, illumination component as L, and reflection component as R, so the multiplicative relationship can be expressed as I = L * R. The illumination component represents the lighting conditions in the scene, which usually changes slowly and is related to the intensity and direction of the light source and the geometry of the scene. It corresponds to low-frequency information in the image (such as large areas of light and dark). The reflection component represents the inherent properties of the object itself, such as its color, material, and texture. It usually occurs abruptly at the edges of the object, corresponding to high-frequency information in the image (such as details and contours). The logarithmic addition relationship means taking the natural logarithm (log) of both sides of the multiplication relationship, resulting in log(I) = log(L * R) = log(L) + log(R). Gaussian blur is a low-pass filter that removes high-frequency details (i.e., noise and edges) from an image, retaining only the slowly changing low-frequency components. Since the illumination component is slowly changing low-frequency information, while the reflection component is high-frequency information containing details, the high-frequency log(R) part of the image log(I) is greatly weakened after Gaussian blur, and the resulting blur is an estimate of the illumination component. By subtracting the estimated illumination component from the additive relationship, theoretically, what remains is the reflection component, representing the object's inherent properties. This step can remove the influence of uneven illumination and extract pure object detail information. Converting the estimated reflection component back to the real number domain (through the inverse operation of logarithmic transformation—exponential operation) yields new pixel values ​​for the sub-image. The image obtained using these new pixel values ​​is the illumination-normalized sub-image that eliminates local overexposure or uneven brightness caused by metallic reflections.

[0029] The above steps transform the complex multiplicative illumination-reflection model into a tractable additive model through logarithmic transformation, and use the characteristics of Gaussian low-pass filtering to estimate and separate the low-frequency component representing the background illumination. Finally, by subtracting this component, an illumination-normalized image with enhanced details and balanced contrast is obtained, thereby significantly improving the visual quality and usability of the image under non-uniform illumination conditions, and providing a more stable and reliable foundation for subsequent tasks such as solder joint recognition and annotation in the image.

[0030] Step 103: Perform spatial pyramid downsampling and initial solder joint annotation on the illumination normalized sub-image to obtain the first annotated sub-image with confidence level.

[0031] This application employs a coarse-grained annotation model to perform spatial pyramid downsampling and initial annotation of solder joints on illumination-normalized sub-images. The coarse-grained annotation model refers to an improved YOLOv11 model that incorporates a Spatial Pyramid Downsampling Convolution (SPD-Conv) module. This effectively reduces feature map resolution and expands the receptive field while preserving and recombining all information from the previous layer to the maximum extent, avoiding information loss. Here, the receptive field refers to the size of the region in the input image corresponding to a point in the feature map; a larger receptive field results in a larger region in the input image corresponding to a point in the feature map. This means that even in deep feature maps, the model can still "see" the fine features of the solder joints, greatly improving the model's sensitivity and ability to capture small targets. The first annotated sub-image output by the coarse-grained annotation model can carry the image's confidence level, facilitating subsequent application of different measures to images with different confidence levels.

[0032] This application utilizes a coarse-grained annotation model to perform preliminary screening of illumination-normalized sub-images, specifically detecting and annotating solder joints within these sub-images. This allows for the subsequent removal of illumination-normalized sub-images that do not contain solder joints. However, the coarse-grained annotation model requires training before use. The training process specifically includes the following steps: Collect several original images of the solder joints to be labeled; For any original solder joint image to be labeled, the solder joints in the original solder joint image are labeled to obtain the first solder joint labeling information; Obtain the first target detection model by introducing the spatial pyramid downsampling module; The original image of the solder joint to be labeled is used as input, and the first solder joint labeling information is used as the training target to train the first target detection model, thereby obtaining a coarse-grained labeling model. The coarse-grained labeling model is used to perform spatial pyramid downsampling of the image and initial labeling of the solder joints.

[0033] The original images of the solder joints to be labeled must be representative to ensure that the subsequently trained model can learn the diverse characteristics of the solder joints under different lighting conditions, angles, welding processes (such as cold solder joints, missing solder joints, and solder joints that are too large or too small), and background interference. This ensures the model's generalization ability and avoids overfitting. Simultaneously, image quality is crucial; high resolution is required to ensure that the detailed features of the solder joints are clearly discernible. Labeling the solder joints in the original images can be done using an image annotation tool like Labelimg, drawing rectangular boxes around the areas where solder spatter accumulates in the original images.

[0034] The above steps overcome the inherent defect of traditional models losing details during the downsampling process by using a spatial pyramid downsampling module specifically optimized for small targets. This enables the training of a robust coarse-grained annotation model that is extremely sensitive to solder joint features using high-quality labeled data, significantly reducing the probability of missed and false detections of small targets such as solder joints, and fundamentally improving the accuracy and reliability of automatic solder joint detection.

[0035] The illumination-normalized sub-images of this application overlap, so when identifying targets in the illumination-normalized sub-images, multiple bounding boxes may be identified for the same target. Therefore, it is necessary to select the most accurate one and remove redundant and highly overlapping bounding boxes. To achieve this, an appropriate non-maximum suppression cross-union ratio (CUI) threshold needs to be set. Then, during the non-maximum suppression process, the bounding boxes are first sorted by confidence level, and the bounding box H with the highest confidence level is retained. Next, the CUI of other bounding boxes with bounding box H is calculated. If the CUI exceeds the CUI threshold, these bounding boxes are considered to point to the same target as bounding box H and are discarded. If the CUI is lower than the CUI threshold, these bounding boxes are retained. In this way, the bounding box with the highest confidence level can be retained while highly overlapping redundant boxes are removed, preventing excessive merging of bounding boxes. This application adaptively adjusts the CUI threshold by adjusting the size of the illumination-normalized sub-images. The steps involved are as follows: Obtain the size of the illumination-normalized sub-image; The non-maximum suppression cross-union ratio threshold of the coarse-grained annotation model is dynamically adjusted based on the size.

[0036] The above steps establish a dynamic correlation between image size and the non-maximum suppression cross-union ratio (CUI) threshold, moving away from mechanically using a fixed parameter and optimizing prediction results based on the size of the currently processed image. This dynamic adjustment significantly enhances robustness to different scenarios: it effectively suppresses redundant predictions on large images, improving efficiency and accuracy; and it effectively prevents missed detections and ensures recall in small images or densely populated areas. Ultimately, this results in more stable and reliable performance for solder joint detection across images with different resolutions, shooting distances, and layouts.

[0037] When obtaining the first labeled sub-image using a coarse-grained annotation model, this application first uses a spatial pyramid downsampling module to extract and compress the features of the illumination-normalized sub-image at multiple scales while preserving more details, gradually constructing a feature map rich in semantic information. Then, using the detection head of the coarse-grained annotation model, based on the semantic information on the feature map, bounding box regression technology is used to identify and predict the center point coordinate offset and width / height scaling factors of possible solder joints. The actual bounding box coordinates of the solder joints on the illumination-normalized sub-image are calculated using the center point coordinate offset and width / height scaling factors, thereby generating a large number of candidate annotation boxes. Then, a deeper semantic judgment is performed on the content within these boxes, not only identifying them as solder joints but also further distinguishing the specific targets they belong to. The classification is based on the specific component target to which the solder joints within the candidate boxes belong. For example, all bounding boxes predicted as "belonging to solder point 1" are grouped into set A, and all bounding boxes predicted as "belonging to solder point 2" are grouped into set B. The model outputs candidate bounding boxes and the target to which the solder point belongs within each candidate bounding box, as well as the target confidence score of that candidate bounding box, representing the model's confidence that the target exists within the box and that its corresponding solder point is correctly predicted. For any set of candidate bounding boxes, the candidate bounding box with the highest target confidence score is selected as the first candidate bounding box, and the remaining candidate bounding boxes are selected as the second candidate bounding boxes. The spatial overlap between the first candidate bounding box and each second candidate bounding box is quantified by calculating the intersection-union ratio (IU / R) between them. If the IU / R is greater than the non-maximum suppression IU / R threshold, the corresponding second candidate bounding box is determined to be a redundant box (i.e., competing with the first candidate bounding box to label the same solder point) and is marked for deletion. After deleting all... After a second candidate box is marked as redundant, the remaining boxes, excluding the first candidate boxes, repeat the above filtering steps (i.e., selecting the candidate box with the highest confidence and calculating the intersection-union ratio (IU / R) with the remaining candidate boxes, comparing the IU / R with the non-maximum suppression IU / R threshold, and continuing to delete redundant boxes) until only one candidate box remains in the candidate box set, ending the iteration operation. At this point, the first candidate box and the remaining second candidate box selected each time are considered as the boxes to be retained. The first candidate box represents the best prediction for a certain solder joint, while the second candidate box that was not deleted represents the successful prediction of other different solder joints in the same group. After this process is applied iteratively to all candidate box sets, all the retained boxes together constitute the complete annotation of the entire sub-image, i.e., the first annotated sub-image. Based on the above, step 103 specifically includes the following sub-steps: Sub-step 1031: Input the illumination-normalized sub-image into the coarse-grained annotation model, perform spatial pyramid downsampling and initial annotation of the solder joints, and determine the candidate annotation boxes and the targets to which the solder joints in the candidate annotation boxes belong.

[0038] Sub-step 1032: Group the candidate annotation boxes based on the target to which the weld point belongs within the candidate annotation box to obtain a set of several candidate annotation boxes.

[0039] Sub-step 1033: For any set of candidate annotation boxes, obtain the target confidence level corresponding to the target to which the solder joint belongs in each candidate annotation box in the set of candidate annotation boxes, and filter out the first candidate annotation box and several second candidate annotation boxes based on the target confidence level.

[0040] Sub-step 1034: Obtain the intersection-union ratio of the first candidate bounding box and each second candidate bounding box.

[0041] Sub-step 1035: By comparing the cross-union ratio and the non-maximum suppressed cross-union ratio threshold, the second candidate bounding box to be deleted is determined.

[0042] Sub-step 1036: The illumination-normalized sub-image is labeled using the deleted second candidate label box and the first candidate label box to obtain the first labeled sub-image.

[0043] The above steps, by first grouping solder joints according to their target, and then performing non-maximum suppression within each group using an adaptive non-maximum suppression cross-union ratio threshold, greatly optimize the accuracy and reliability of annotation. It not only effectively eliminates redundant predictions for the same solder joint, but also intelligently protects the correct annotation of adjacent solder joints on different components, significantly improving the recall and accuracy of solder joint annotation.

[0044] After screening candidate bounding boxes by non-maximum suppression cross-union ratio thresholding, this application will also locate the detection boxes of all first-labeled sub-images in the original image, aggregate and deduplicate the detection results of overlapping areas, and further screen the bounding boxes.

[0045] Step 104: Select the first target sub-image and the second target sub-image to be annotated again from the first labeled sub-image based on confidence level.

[0046] The coarse-grained annotation model of this application outputs a confidence score for the first annotated sub-image along with the first annotated sub-image. Then, based on pre-set first and second confidence thresholds, it determines the processing method for the first annotated sub-image. Specifically, the process of selecting the second target sub-image for secondary annotation from the first annotated sub-image based on the confidence score includes the following sub-steps: Sub-step 1041: If the confidence level is less than the first confidence threshold, then delete the first labeled sub-image.

[0047] Sub-step 1042: If the confidence level is greater than the second confidence level threshold, then the first labeled sub-image is determined to be the first target sub-image.

[0048] Sub-step 1043: If the confidence level is greater than or equal to the first confidence threshold and less than or equal to the second confidence threshold, then the first labeled sub-image is determined to be the second target sub-image to be labeled again.

[0049] The first confidence threshold is less than the second confidence threshold. For example, if the first confidence threshold is set to 0.3 and the second confidence threshold is set to 0.7, then if the confidence of a certain first-labeled sub-image is 0.25, it is deleted because 0.25 < 0.3. If the confidence of a certain first-labeled sub-image is 0.8, it is retained because 0.8 > 0.7. If the confidence of a certain first-labeled sub-image is 0.5, it is determined as the second target sub-image to be re-labeled because 0.3 < 0.5 < 0.7.

[0050] The above steps, by introducing a three-level decision-making mechanism based on confidence, achieve intelligent quality sorting and process optimization of automatic annotation results. Low-quality results are decisively discarded to ensure data purity, while high-quality results are directly adopted to improve efficiency. The second target sub-images that need secondary annotation are accurately selected. Thus, while ensuring the overall high quality of the final annotation dataset, the automation level and resource utilization efficiency of the entire annotation process are significantly improved.

[0051] Step 105: Perform multi-dilation rate convolution operation, channel-spatial attention cascade, small target detection, and secondary annotation of solder joints on the second target sub-image to obtain the second annotated sub-image.

[0052] This application employs a fine-grained annotation model to perform multi-dilation rate convolution operations, channel-spatial attention cascades, small target detection, and secondary annotation of weld points on the second target sub-image. The fine-grained annotation model incorporates a receptive field block (RFB), a convolutional block attention module (CBAM), and a second pyramid-level detection layer. The receptive field block performs multi-dilation rate convolution operations on the input second target sub-image, introducing multiple dilated convolutions with different dilation rates in parallel or cascaded manner. This allows for the capture of multi-scale contextual information at the same level, significantly enhancing the model's ability to perceive targets of different sizes (especially small targets, corresponding to weld points in this application). The convolutional block attention module performs channel-spatial attention cascades on the input second target sub-image, connecting (cascading) the channel attention and spatial attention modules. First, channel attention selects important feature types in the second target sub-image, then spatial attention locates important positions within the second target sub-image. The combination of these two approaches achieves precise focusing, suppressing metallic reflection interference, focusing on key spatial regions, and adaptive feature fusion. The second pyramid-level detection layer can perform small target detection on the input second target sub-image, specifically on the high-resolution feature layer (P2 detection layer) of the feature pyramid. In the feature pyramid network or pyramid architecture, "P" represents a level of the pyramid. P2 specifically refers to the earliest and highest-resolution feature layer from the backbone network (usually 1 / 4 or 1 / 8 the size of the input image). Enabling the P2 detection layer means that the model will directly utilize this high-resolution feature map rich in fine details (such as edges and corners) for prediction, which greatly improves the ability to detect extremely small targets in the image. In this application, small targets refer to solder joints in the image. The above module can significantly improve the detection accuracy of dense and blurred solder joints.

[0053] Before performing secondary annotation on the second target sub-image to be annotated, this application requires training the fine-grained annotation model. The training process is as follows: The original image of the weld points to be labeled is divided into equal parts to obtain original sub-images with a preset overlap rate; For any given original sub-image, the solder joints in the original sub-image are labeled to obtain the second solder joint labeling information; Obtain a second object detection model that incorporates a receptive field module, a convolutional block attention module, and a second pyramid-level detection layer; The original sub-image is used as input, and the second solder joint annotation information is used as the training target to train the second target detection model, resulting in a fine-grained annotation model. The fine-grained annotation model is used to perform multi-dilation rate convolution operation, channel-spatial attention cascade, small target detection, and secondary annotation of solder joints on the image.

[0054] The preset overlap rate of the original sub-images obtained after segmenting the original solder joint image to be labeled during training is consistent with the preset overlap rate of the sub-images obtained after segmenting the original solder joint image to be labeled in actual application. This can be set to 10% or other values; this application does not impose specific limitations on this. Since the coarse-grained annotation model has already been trained using the original solder joint image to be labeled, and subsequent training of the fine-grained annotation model requires improved accuracy, the original solder joint image to be labeled is segmented to avoid loss of detail and detection difficulties due to the target solder joints being too small or too dense. By equally dividing the large image into smaller, more easily processed image patches (sub-images), the relative size of each solder joint in the image can be effectively magnified, making its features more prominent. When labeling the solder joints in the original sub-images, the image annotation tool Labelimg can be used for precise single-point labeling of the solder joints.

[0055] The above steps cleverly solve the inherent problem of high-density small target detection through the "equal division and overlap" strategy. On this basis, using high-quality fine-labeled data, a fine-grained labeling model integrating advanced technologies such as multi-scale receptive field, intelligent attention mechanism and high-resolution detection layer was trained. It has an extremely sensitive perception ability and amazing identification accuracy for tiny solder joint defects, and improves the recall and accuracy of single-point labeling of complex small targets such as solder joint spatter.

[0056] Step 106: Determine the solder joint annotation result of the solder joint image to be annotated by using the second annotation sub-image and the first target sub-image.

[0057] When this application outputs the second labeled sub-image using the fine-grained annotation model, it also outputs the confidence level of the second labeled sub-image. A confidence threshold is set, and second labeled sub-images with confidence levels below the threshold are discarded, indicating that no valid solder joints were detected in these discarded second labeled sub-images. After processing, the first target sub-image retained by the coarse-grained annotation model and the second labeled sub-image retained by the fine-grained annotation model are taken as the final valid images. Because the solder joints in these images have already been annotated during the modeling process, the solder joint annotation results for the image to be annotated can be determined based on the annotation status of these images, improving annotation efficiency and accuracy. Alternatively, the image annotation tool Labelimg can be used to re-annotate the solder joints in these images to obtain the solder joint annotation results for the image to be annotated. Because the number of images requiring annotation is reduced after filtering, even re-annotating using the image annotation tool Labelimg can improve annotation efficiency.

[0058] The flowchart for image filtering using a dual-model approach in this application is as follows: Figure 2 As shown, the image of solder joints to be labeled is segmented to obtain sub-images with a preset overlap rate. Then, the confidence level of the output first labeled sub-image is determined by a coarse-grained labeling model. First labeled sub-images with a confidence level less than the first confidence level threshold are directly discarded, while first labeled sub-images with a confidence level greater than the second confidence level threshold are directly retained. First labeled sub-images with a confidence level between the first and second confidence levels are determined as the second target sub-images and input into a fine-grained labeling model. The fine-grained labeling model detects and labels the input second target sub-images. If a solder joint is detected in the input second target sub-image, the image is retained and determined as the second labeled sub-image. If no solder joint is detected in the input second target sub-image, the image is deleted.

[0059] The solder joint annotation method provided in this application firstly uses a fixed-size sliding window to segment the image of the solder joints to be annotated, obtaining an initial segmented sub-image with a preset overlap rate. This operation makes tiny solder joints, which originally occupied only a few pixels in the entire image of the solder joints to be annotated, relatively prominent targets in the sub-image, greatly improving the effective resolution and feature richness of the targets. The introduction of the preset overlap rate effectively avoids the solder joints from being segmented or lost due to being at the cutting boundary, thus eliminating the problem of missed detection caused by this. Using image processing theory algorithms, the illumination component of the sub-image is removed to obtain an illumination-normalized sub-image, effectively suppressing local overexposure or uneven brightness caused by reflection, avoiding missed solder joints due to illumination interference. The illumination-normalized sub-image is then subjected to spatial pyramid downsampling and initial solder joint annotation to obtain a first annotated sub-image carrying confidence. The application of spatial pyramid downsampling improves the detection capability of solder joints. The confidence is used to filter out the first target sub-image and the second target sub-image to be annotated a second time from the first annotated sub-image. This method not only quickly preserves high-quality annotation results to improve efficiency, but also accurately identifies difficult samples requiring secondary annotation. The second target sub-image undergoes multi-dilation rate convolution, channel-spatial attention cascade, small target detection, and secondary solder joint annotation to obtain a second annotated sub-image. The multi-dilation rate convolution enhances multi-scale perception of solder joints of different sizes, the channel-spatial attention cascade effectively suppresses metal reflection interference, focuses on key spatial regions, and achieves adaptive feature fusion. Small target detection improves the detection capability of solder joints in the image. The solder joint annotation result is determined by using the second annotated sub-image and the first target sub-image. In summary, this application solves the problems of low efficiency, high missed annotation rate, and high mis-annotation rate in manual annotation caused by complex conditions such as extremely small and numerous spatter points, irregular shapes, and high background reflectivity and noise, thus improving the efficiency, recall, and accuracy of solder joint annotation.

[0060] Reference Figure 3 The diagram shows a schematic representation of a solder joint marking device provided in this application. The device includes: The first cutting module 201 is used to cut the image of the weld point to be marked using a sliding window of a fixed size to obtain an initial segmented sub-image with a preset overlap rate.

[0061] The illumination component removal module 202 is used to remove the illumination component of the initial segmented sub-image using image processing theory and algorithms to obtain an illumination-normalized sub-image.

[0062] The initial annotation module 203 is used to perform spatial pyramid downsampling and initial annotation of the solder joints on the illumination normalized sub-image to obtain the first annotated sub-image carrying confidence.

[0063] The filtering module 204 is used to filter out the first target sub-image and the second target sub-image to be further annotated from the first labeled sub-image based on confidence level.

[0064] The secondary annotation module 205 is used to perform multi-dilation rate convolution operation, channel-spatial attention cascade, small target detection and secondary annotation of weld points on the second target sub-image to obtain the second annotated sub-image.

[0065] The determination module 206 is used to determine the solder joint annotation result of the solder joint image to be annotated by using the second annotation sub-image and the first target sub-image.

[0066] Optionally, the illumination component rejection module 202 includes: The first acquisition submodule is used to acquire the pixel values ​​of the initial segmented sub-image and use image processing theory algorithms to determine the multiplication relationship between the pixel values ​​and the illumination and reflection components of the initial segmented sub-image.

[0067] The transformation submodule is used to convert multiplication relations into addition relations in the logarithmic field.

[0068] The first estimation submodule is used to perform Gaussian blur operation on the initial segmented subimage and estimate the illumination components of the initial segmented subimage.

[0069] The second estimation submodule is used to estimate the reflection component of the initial segmented sub-image using additive relationships and illumination components.

[0070] The first determining submodule is used to determine the new pixel values ​​of the initial segmented subimage through the reflection component of the initial segmented subimage.

[0071] The second acquisition submodule is used to acquire the illumination-normalized subimage through the new pixel values ​​of the initial segmented subimage.

[0072] Optionally, the solder joint marking device also includes: The acquisition module is used to acquire several original images of the solder joints to be labeled.

[0073] The first annotation module is used to annotate the solder joints in any original solder joint image to obtain the first solder joint annotation information.

[0074] The first acquisition module is used to acquire the first target detection model that incorporates the spatial pyramid downsampling module.

[0075] The first training module is used to train the first target detection model by taking the original image of the solder joint to be labeled as input and the first solder joint labeling information as the training target, and obtaining a coarse-grained labeling model. The coarse-grained labeling model is used to perform spatial pyramid downsampling of the image and initial labeling of the solder joints.

[0076] The second acquisition module is used to acquire the size of the illumination-normalized sub-image.

[0077] The adjustment module is used to dynamically adjust the non-maximum suppression cross-union ratio threshold of the coarse-grained annotation model based on the size.

[0078] Optionally, the initial annotation module 203 includes: The first annotation submodule is used to input the illumination normalized sub-image into the coarse-grained annotation model, perform spatial pyramid downsampling and initial annotation of solder joints, and determine the candidate annotation box and the target to which the solder joint in the candidate annotation box belongs.

[0079] The grouping module is used to group candidate annotation boxes based on the target to which the weld point belongs within the candidate annotation box, resulting in a set of several candidate annotation boxes.

[0080] The third acquisition submodule is used to acquire the target confidence of the target to which the solder joint belongs in each candidate annotation box in any candidate annotation box set, and to filter out the first candidate annotation box and several second candidate annotation boxes based on the target confidence.

[0081] The fourth acquisition submodule is used to obtain the intersection-union ratio of the first candidate annotation box and each second candidate annotation box.

[0082] The second determination submodule is used to determine the second candidate bounding box to be deleted by comparing the cross-union ratio and the non-maximum suppression cross-union ratio threshold.

[0083] The second annotation submodule is used to annotate the illumination-normalized subimage using the deleted second candidate annotation box and the first candidate annotation box to obtain the first annotated subimage.

[0084] Optionally, the filtering module 204 includes: The delete submodule is used to delete the first labeled sub-image if the confidence level is less than the first confidence level threshold.

[0085] The third determination submodule is used to determine the first labeled subimage as the first target subimage if the confidence level is greater than the second confidence level threshold.

[0086] The fourth determination submodule is used to determine the first labeled sub-image as the second target sub-image to be labeled if the confidence level is greater than or equal to the first confidence threshold and less than or equal to the second confidence threshold.

[0087] Optionally, the solder joint marking device also includes: The second cutting module is used to cut the original image of the weld points to be labeled into equal parts to obtain the original sub-images with a preset overlap rate.

[0088] The second annotation module is used to annotate the solder joints in any original sub-image to obtain the second solder joint annotation information.

[0089] The third acquisition module is used to acquire the second object detection model that incorporates the receptive field module, the convolutional block attention module, and the second pyramid-level detection layer.

[0090] The second training module is used to train the second target detection model by taking the original sub-image as input and the second solder joint annotation information as training target, so as to obtain a fine-grained annotation model. The fine-grained annotation model is used to perform multi-dilation rate convolution operation, channel-spatial attention cascade, small target detection and secondary annotation of solder joints on the image.

[0091] The solder joint annotation device provided in this application firstly segments the image of the solder joint to be annotated using a sliding window of fixed size, obtaining an initial segmented sub-image with a preset overlap rate. This operation makes tiny solder joints, which originally occupied only a few pixels in the entire image of the solder joint to be annotated, relatively prominent targets in the sub-image, greatly improving the effective resolution and feature richness of the targets. The introduction of the preset overlap rate effectively avoids the solder joints being segmented or lost due to being at the cutting boundary, thus eliminating the problem of missed detection caused by this. The illumination component of the sub-image is removed using image processing theory algorithms to obtain an illumination-normalized sub-image, effectively suppressing local overexposure or uneven brightness caused by reflection, avoiding missed solder joints due to illumination interference. The illumination-normalized sub-image is then subjected to spatial pyramid downsampling and initial solder joint annotation to obtain a first annotated sub-image carrying confidence. The application of spatial pyramid downsampling improves the detection capability of solder joints. The first target sub-image and the second target sub-image to be annotated for a second time are selected from the first annotated sub-image based on the confidence. This method not only quickly preserves high-quality annotation results to improve efficiency, but also accurately identifies difficult samples requiring secondary annotation. The second target sub-image undergoes multi-dilation rate convolution, channel-spatial attention cascade, small target detection, and secondary solder joint annotation to obtain a second annotated sub-image. The multi-dilation rate convolution enhances multi-scale perception of solder joints of different sizes, the channel-spatial attention cascade effectively suppresses metal reflection interference, focuses on key spatial regions, and achieves adaptive feature fusion. Small target detection improves the detection capability of solder joints in the image. The solder joint annotation result is determined by using the second annotated sub-image and the first target sub-image. In summary, this application solves the problems of low efficiency, high missed annotation rate, and high mis-annotation rate in manual annotation caused by complex conditions such as extremely small and numerous spatter points, irregular shapes, and high background reflectivity and noise, thus improving the efficiency, recall, and accuracy of solder joint annotation.

[0092] Reference Figure 4 This application also provides an electronic device, such as Figure 4As shown, it includes a processor 301, a communication interface 302, a memory 303, and a communication bus 304, wherein the processor 301, the communication interface 302, and the memory 303 communicate with each other through the communication bus 304. Processor 301, memory 303 for storing processor-executable instructions; The processor 301 is configured to execute the instructions to implement the solder joint marking method as described above: A fixed-size sliding window is used to cut the image of the solder joint to be labeled, resulting in an initial segmented sub-image with a preset overlap rate; The illumination component of the sub-image is removed using image processing theory and algorithms to obtain an illumination-normalized sub-image; The illumination-normalized sub-image is subjected to spatial pyramid downsampling and initial solder joint annotation to obtain a first annotated sub-image carrying confidence. The first target sub-image and the second target sub-image to be further annotated are selected from the first labeled sub-image based on the confidence level. The second target sub-image is subjected to multi-dilation rate convolution, channel-spatial attention cascade, small target detection, and secondary annotation of solder joints to obtain the second annotated sub-image. The solder joint annotation result of the solder joint image to be annotated is determined by the second annotated sub-image and the first target sub-image.

[0093] The communication bus mentioned above can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. This communication bus can be divided into address bus, data bus, control bus, etc. For ease of illustration, only one thick line is used to represent it in the diagram, but this does not mean that there is only one bus or one type of bus.

[0094] The communication interface is used for communication between the aforementioned terminal and other devices.

[0095] The memory may include random access memory (RAM) or non-volatile memory, such as at least one disk storage device. Optionally, the memory may also be at least one storage device located remotely from the aforementioned processor.

[0096] The processors mentioned above can be general-purpose processors, including central processing units (CPUs), network processors (NPs), etc.; they can also be digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0097] In another embodiment provided in this application, a computer-readable storage medium is also provided, which stores instructions that, when executed on a computer, cause the computer to perform any of the solder joint marking methods described in the above embodiments.

[0098] In the above embodiments, implementation can be achieved, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented in software, it can be implemented, in whole or in part, as a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium accessible to a computer or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., a solid-state drive (SSD)).

[0099] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0100] The various embodiments in this specification are described in a related manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the system embodiments are basically similar to the method embodiments, so the description is relatively simple; relevant parts can be referred to the descriptions of the method embodiments.

[0101] The above description is merely a preferred embodiment of this application and is not intended to limit the scope of protection of this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application are included within the scope of protection of this application.

Claims

1. A method for marking solder joints, characterized in that, The method includes: A fixed-size sliding window is used to cut the image of the solder joint to be labeled, resulting in an initial segmented sub-image with a preset overlap rate; The illumination component of the initial segmented sub-image is removed using image processing theory and algorithms to obtain an illumination-normalized sub-image; The illumination-normalized sub-image is subjected to spatial pyramid downsampling and initial solder joint annotation to obtain a first annotated sub-image carrying confidence. The first target sub-image and the second target sub-image to be further annotated are selected from the first labeled sub-image based on the confidence level. The second target sub-image is subjected to multi-dilation rate convolution, channel-spatial attention cascade, small target detection, and secondary annotation of solder joints to obtain the second annotated sub-image. The solder joint annotation result of the solder joint image to be annotated is determined by the second annotated sub-image and the first target sub-image.

2. The method according to claim 1, characterized in that, The step of using image processing theory and algorithms to remove the illumination component of the initial segmented sub-image to obtain an illumination-normalized sub-image further includes: The pixel values ​​of the initial segmented sub-image are obtained, and the multiplication relationship between the pixel values ​​and the illumination and reflection components of the initial segmented sub-image is determined using image processing theory algorithms. The multiplication relation is converted into an addition relation in the logarithmic field; Gaussian blur is applied to the initial segmented sub-image to estimate the illumination components of the initial segmented sub-image; The reflection component of the initial segmented sub-image is estimated using the additive relationship and the illumination component. The new pixel values ​​of the initial segmented sub-image are determined by the reflection component of the initial segmented sub-image; The illumination-normalized sub-image is obtained by using the new pixel values ​​of the initial segmented sub-image.

3. The method according to claim 1, characterized in that, Before performing spatial pyramid downsampling and initial solder joint annotation on the illumination-normalized sub-image to obtain the first annotated sub-image carrying confidence, the method further includes: Collect several original images of the solder joints to be labeled; For any original solder joint image to be labeled, the solder joints in the original solder joint image to be labeled are labeled to obtain the first solder joint labeling information; Obtain the first target detection model by introducing the spatial pyramid downsampling module; The original image of the solder joint to be labeled is used as input, and the first solder joint labeling information is used as the training target to train the first target detection model to obtain a coarse-grained labeling model. The coarse-grained labeling model is used to perform spatial pyramid downsampling of the image and initial labeling of the solder joints.

4. The method according to claim 3, characterized in that, Before performing spatial pyramid downsampling and initial solder joint annotation on the illumination-normalized sub-image to obtain the first annotated sub-image carrying confidence, the method further includes: Obtain the dimensions of the illumination-normalized sub-image; The non-maximum suppression cross-union ratio threshold of the coarse-grained annotation model is dynamically adjusted based on the stated dimensions.

5. The method according to claim 4, characterized in that, The step of performing spatial pyramid downsampling and initial solder joint annotation on the illumination-normalized sub-image to obtain a first annotated sub-image carrying confidence scores includes: The illumination-normalized sub-image is input into the coarse-grained annotation model to perform spatial pyramid downsampling and initial annotation of solder joints, thereby determining candidate annotation boxes and the targets to which the solder joints within the candidate annotation boxes belong. Based on the target to which the solder joint belongs within the candidate annotation box, the candidate annotation boxes are grouped to obtain several candidate annotation box sets; For any set of candidate annotation boxes, obtain the target confidence level of the target to which the solder joint belongs in each candidate annotation box in the set, and filter out the first candidate annotation box and several second candidate annotation boxes based on the target confidence level; Obtain the intersection-union ratio (IUU) of the first candidate bounding box and each second candidate bounding box; By comparing the intersection-union ratio (CUNR) with the non-maximum suppression CUNR threshold, the second candidate bounding box to be deleted is determined. The illumination-normalized sub-image is labeled using the deleted second candidate label box and the first candidate label box to obtain the first labeled sub-image.

6. The method according to claim 1, characterized in that, The step of filtering the first target sub-image and the second target sub-image to be re-annotated from the first labeled sub-image using the confidence level further includes: If the confidence level is less than the first confidence threshold, then delete the first labeled sub-image; If the confidence level is greater than the second confidence threshold, then the first labeled sub-image is determined to be the first target sub-image; If the confidence level is greater than or equal to the first confidence threshold and less than or equal to the second confidence threshold, then the first labeled sub-image is determined to be the second target sub-image to be labeled again.

7. The method according to claim 3, characterized in that, After filtering out the first target sub-image and the second target sub-image to be re-annotated from the first labeled sub-image using the confidence level, the process includes: The original image of the solder joints to be labeled is segmented to obtain original sub-images with a preset overlap rate; For any original sub-image, the solder joints in the original sub-image are labeled to obtain the second solder joint labeling information; Obtain a second object detection model that incorporates a receptive field module, a convolutional block attention module, and a second pyramid-level detection layer; The original sub-image is used as input, and the second solder joint annotation information is used as the training target to train the second target detection model, thereby obtaining a fine-grained annotation model. The fine-grained annotation model is used to perform multi-dilation rate convolution operation, channel-spatial attention cascade, small target detection, and secondary annotation of solder joints on the image.

8. A solder joint marking device, characterized in that, The device includes: The first cutting module is used to cut the image of the weld point to be marked using a sliding window of a fixed size to obtain an initial segmented sub-image with a preset overlap rate; The illumination component removal module is used to remove the illumination component of the initial segmented sub-image using image processing theoretical algorithms, so as to obtain the illumination-normalized sub-image. The initial annotation module is used to perform spatial pyramid downsampling and initial annotation of solder joints on the illumination normalized sub-image to obtain a first annotated sub-image carrying confidence. The filtering module is used to filter out a first target sub-image and a second target sub-image to be further labeled from the first labeled sub-image based on the confidence level. The secondary annotation module is used to perform multi-dilation rate convolution operation, channel-spatial attention cascade, small target detection and secondary annotation of solder joints on the second target sub-image to obtain the second annotated sub-image; The determination module is used to determine the solder joint annotation result of the solder joint image to be annotated by using the second annotation sub-image and the first target sub-image.

9. An electronic device, characterized in that, include: processor; Memory used to store processor-executable instructions; The processor is configured to execute the instructions to implement the solder joint marking method as described in any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the solder joint marking method as described in any one of claims 1 to 7.

Images