Solder wheel profile extraction method, device, equipment and storage medium

Abstract

The application discloses a soldering contour extraction method, device, equipment and storage medium. The method first acquires a two-dimensional image and a three-dimensional height map of a target detection object; performs image segmentation on the two-dimensional image to obtain an initial mask; modifies the initial mask based on height information in the three-dimensional height map to obtain a target mask; extracts an edge of the target mask to obtain a closed contour point sequence; performs geometric fitting on the closed contour point sequence to convert the closed contour point sequence into an initial geometric primitive sequence containing line segments and arc line segments; performs primitive position alignment and intersection reconstruction on the initial geometric primitive sequence based on a preset geometric rule to obtain a target geometric primitive sequence; and generates soldering contour data of the target detection object based on the target geometric primitive sequence. The application can improve the precision of the extracted contour data and thus improve the accuracy of SPI detection.

Description

Technical Field

[0001] This application relates to the fields of computer vision and industrial intelligent inspection technology, and in particular to a method, apparatus, equipment and storage medium for solder contour extraction. Background Technology

[0002] In the surface mount technology (SMT) manufacturing process, solder paste inspection (SPI) equipment is typically used to inspect quality indicators such as the height, area, volume, and offset of solder or solder paste deposits on the PCB. Traditional SPI programming heavily relies on computer-aided manufacturing standard data files, such as Gerber files and CAD design data, to parse the shape, size, and coordinates of the pads and generate inspection windows accordingly. However, in actual production lines, standard data files are often unavailable in a timely manner due to reasons such as data confidentiality, missing information, or frequent product changes. This makes programming without standard data files a crucial requirement for rapid implementation.

[0003] In related technologies, programming schemes without standard data files have two major drawbacks: First, image segmentation based on a single feature is easily affected by interference, resulting in missed detections and false detections; second, directly using bounding rectangles or original discrete contours as detection objects cannot accurately represent the true geometric boundaries and is prone to introducing boundary noise, resulting in poor accuracy of the extracted contour data, which in turn seriously reduces the accuracy of SPI detection. Summary of the Invention

[0004] This application aims to address at least one of the technical problems existing in the prior art. To this end, this application proposes a solder contour extraction method, apparatus, device, and storage medium, which can improve the accuracy of the extracted contour data, thereby improving the accuracy of SPI detection.

[0005] In a first aspect, embodiments of this application provide a solder contour extraction method, including: Acquire the 2D image and 3D height map of the target object; The two-dimensional image is segmented to obtain an initial mask; The initial mask is corrected based on the height information in the three-dimensional height map to obtain the target mask; Extract the edges of the target mask to obtain a closed contour point sequence; Geometric fitting is performed on the closed contour point sequence to convert it into an initial geometric primitive sequence containing line segments and arc segments; Based on preset geometric rules, the initial geometric primitive sequence is aligned and its intersection points are reconstructed to obtain the target geometric primitive sequence. Based on the target geometric primitive sequence, solder contour data of the target detection object is generated.

[0006] According to some embodiments of this application, the step of performing image segmentation on the two-dimensional image to obtain an initial mask includes: The two-dimensional image is input into a pre-trained deep learning segmentation model, which outputs a preliminary prediction mask. Extract the color features of the area covered by the preliminary predicted mask; Within the entire area of ​​the two-dimensional image, color similarity is calculated based on the color features to extract a color-assisted segmentation mask; The preliminary prediction mask is fused with the color-assisted segmentation mask to generate the initial mask.

[0007] According to some embodiments of this application, the step of correcting the initial mask based on the height information in the three-dimensional height map to obtain the target mask includes: Obtain the height value of each pixel point contained in the initial mask in the three-dimensional height map. If the height value is lower than a preset height threshold, remove the corresponding pixel point from the initial mask to obtain the first modified mask. For each pixel within the first correction mask, a local window centered on that pixel is constructed, and the height gradient value within each local window is calculated. If the height gradient value of the local window meets the preset gradient anomaly threshold condition, the corresponding pixel is removed from the first correction mask to obtain the target mask.

[0008] According to some embodiments of this application, the step of extracting the edge of the target mask to obtain a closed contour point sequence includes: Perform connected component analysis on the target mask to determine at least one candidate target region and remove noisy connected components with an area smaller than a preset area threshold; The selected candidate target regions are processed by internal hole filling and morphological opening and closing operations, and the outer contour of each candidate target region after processing is extracted to obtain an initial contour point sequence. The initial contour point sequence is resampled and smoothed to obtain the closed contour point sequence.

[0009] According to some embodiments of this application, the step of geometrically fitting the closed contour point sequence to convert the closed contour point sequence into an initial geometric primitive sequence containing line segments and arc segments includes: The closed contour point sequence is segmented using a polyline segmentation algorithm to generate an initial line segment sequence consisting of multiple initial line segments connected end to end. Traverse the initial line segment sequence, extract local line segment combinations consisting of at least two connected initial line segments in sequence, and obtain the target contour point subset corresponding to the local line segment combination; Calculate the first fitting residual from the subset of target contour points to the combination of local line segments, and the second fitting residual generated when fitting the subset of target contour points to candidate arc segments. If the second fitting residual is less than the first fitting residual, and the candidate arc segment satisfies the preset arc quantization constraint, then the local line segment combination in the initial line segment sequence is replaced with the candidate arc segment to obtain the initial geometric primitive sequence; wherein, the preset arc quantization constraint includes at least one of the following: The arc height of the candidate arc segment is greater than or equal to the preset minimum arc height threshold. The ratio of the arc height to the chord length of the candidate arc segment is greater than or equal to a preset ratio threshold. When the local line segment combination is located between two parallel initial line segments, the absolute value of the difference between the ideal radius of the candidate arc segment and half the distance between the two parallel initial line segments is less than or equal to a first tolerance threshold, and the absolute value of the difference between the arc height of the candidate arc segment and the ideal radius is less than or equal to a second tolerance threshold.

[0010] According to some embodiments of this application, the step of performing primitive position alignment and intersection reconstruction on the initial geometric primitive sequence based on preset geometric rules to obtain the target geometric primitive sequence includes: The orientation angles of each line segment in the initial geometric primitive sequence are extracted and clustered to construct multiple reference baseline support lines. Project the endpoints of each line segment onto the reference datum support line that matches the direction, so that each line segment snaps onto the corresponding reference datum support line, thus completing the alignment of the primitive position. Calculate the intersection point of two adjacent reference datum support lines, and use the intersection point as a common endpoint to connect the line segments attached to the two adjacent reference datum support lines to obtain the target geometric primitive sequence.

[0011] According to some embodiments of this application, the step of performing primitive position alignment and intersection reconstruction on the initial geometric primitive sequence based on preset geometric rules to obtain the target geometric primitive sequence further includes: Identify the target arc segment located between two parallel reference datum support lines in the initial geometric primitive sequence; Based on the positions of the two parallel reference datum support lines, the center and radius parameters of the target arc segment are corrected so that the two endpoints of the corrected target arc segment are respectively connected to the two parallel reference datum support lines. Identify the offset short line segment located between the two intersecting reference support lines as a corner artifact, delete the corner artifact, and extend the two intersecting reference support lines to the intersection.

[0012] Secondly, embodiments of this application provide a solder contour extraction device, comprising: The acquisition module is used to acquire the two-dimensional image and three-dimensional height map of the target object. An image segmentation module is used to segment the two-dimensional image to obtain an initial mask; The height correction module is used to correct the initial mask based on the height information in the three-dimensional height map to obtain the target mask; The contour extraction module is used to extract the edges of the target mask to obtain a sequence of closed contour points. The geometric fitting module is used to perform geometric fitting on the closed contour point sequence and convert the closed contour point sequence into an initial geometric primitive sequence containing line segments and arc segments. The primitive reconstruction module is used to perform primitive position alignment and intersection reconstruction on the initial geometric primitive sequence based on preset geometric rules to obtain the target geometric primitive sequence; The generation module is used to generate solder contour data of the target detection object based on the target geometric primitive sequence.

[0013] Thirdly, embodiments of this application provide an electronic device, including: a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the solder contour extraction method as described in any one of the embodiments of the first aspect of this application.

[0014] Fourthly, embodiments of this application provide a computer-readable storage medium storing a program that is executed by a processor to implement the solder contour extraction method as described in any one of the embodiments of the first aspect of this application.

[0015] This application proposes a solder contour extraction method, apparatus, electronic device, and storage medium. The method first acquires a two-dimensional image and a three-dimensional height map of the target object; performs image segmentation on the two-dimensional image to obtain an initial mask; performs correction processing on the initial mask based on the height information in the three-dimensional height map to obtain a target mask; extracts the edges of the target mask to obtain a closed contour point sequence; performs geometric fitting on the closed contour point sequence to convert it into an initial geometric primitive sequence containing line segments and arc segments; performs primitive position alignment and intersection reconstruction on the initial geometric primitive sequence based on preset geometric rules to obtain a target geometric primitive sequence; and generates solder contour data of the target object based on the target geometric primitive sequence.

[0016] In contrast to the problems in the background art, the solder contour extraction method provided in this application firstly acquires a two-dimensional image and a three-dimensional height map of the target object. Based on the height information in the three-dimensional height map, the initial mask obtained from the two-dimensional image segmentation is corrected. By combining multi-dimensional data, misjudged areas that may occur in the single two-dimensional image segmentation are eliminated, resulting in a more accurate target mask. Secondly, the edges of the target mask are extracted to obtain a closed contour point sequence, and the closed contour point sequence is geometrically fitted to convert it into an initial geometric primitive sequence containing line segments and arc segments. This transforms discrete pixels into structured geometric features, reducing edge interference caused by local pixel fluctuations. Finally, based on preset geometric rules, primitive position alignment and intersection reconstruction are performed on the initial geometric primitive sequence to obtain a target geometric primitive sequence after correcting positional deviations. The above steps work synergistically. Through the gradual processing of height information correction, geometric fitting, and rule reconstruction, the impact of image noise and discretization errors is reduced, improving the accuracy of the extracted contour data and thus enhancing the accuracy of SPI detection.

[0017] Additional aspects and advantages of this application will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of this application. Attached Figure Description

[0018] The above and / or additional aspects and advantages of this application will become apparent and readily understood from the description of the embodiments taken in conjunction with the following drawings, in which: Figure 1 This is a flowchart of a solder contour extraction method provided in an embodiment of this application.

[0019] Figure 2 This is a flowchart of a solder contour extraction method provided in another embodiment of this application.

[0020] Figure 3 This is a flowchart of a solder contour extraction method provided in another embodiment of this application.

[0021] Figure 4 This is a flowchart of a solder contour extraction method provided in another embodiment of this application.

[0022] Figure 5 This is a flowchart of a solder contour extraction method provided in another embodiment of this application.

[0023] Figure 6 This is a flowchart of a solder contour extraction method provided in another embodiment of this application.

[0024] Figure 7 This is a flowchart of a solder contour extraction method provided in another embodiment of this application.

[0025] Figure 8 This is a schematic diagram of a solder contour extraction device provided in one embodiment of this application.

[0026] Figure 9 This is a schematic diagram of the hardware structure of the electronic device provided in the embodiments of this application. Detailed Implementation

[0027] The embodiments of this application are described in detail below. Examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain this application, and should not be construed as limiting this application.

[0028] In the description of this application, "several" means one or more, "more than" means two or more, "greater than," "less than," and "exceeding" are understood to exclude the stated number, while "above," "below," and "within" are understood to include the stated number. The use of "first" and "second" in the description is merely for distinguishing technical features and should not be construed as indicating or implying relative importance, or implicitly indicating the number of indicated technical features, or implicitly indicating the order of the indicated technical features.

[0029] In the description of this application, it should be understood that the orientation descriptions, such as up, down, left, right, front, and back, are based on the orientation or positional relationship shown in the accompanying drawings. They are only for the convenience of describing this application and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on this application.

[0030] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "illustrative embodiment," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of this application. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0031] In the description of this application, it should be noted that, unless otherwise explicitly defined, terms such as "setting," "installation," and "connection" should be interpreted broadly. Those skilled in the art can reasonably determine the specific meaning of the above terms in this application based on the specific content of the technical solution. Furthermore, the identification of specific steps in the following text does not imply a limitation on the order of steps or execution logic. The execution order and logic between each step should be understood and inferred from the content described in the embodiments.

[0032] In the surface mount technology (SMT) manufacturing process, solder paste inspection (SPI) equipment is typically used to inspect quality indicators such as the height, area, volume, and offset of solder or solder paste deposits on the PCB. Traditional SPI programming heavily relies on computer-aided manufacturing standard data files, such as Gerber files and CAD design data, to parse the shape, size, and coordinates of the pads and generate inspection windows accordingly. However, in actual production lines, standard data files are often unavailable in a timely manner due to reasons such as data confidentiality, missing information, or frequent product changes. This makes programming without standard data files a crucial requirement for rapid implementation.

[0033] In related technologies, programming schemes without standard data files have two major drawbacks: First, image segmentation based on a single feature is easily affected by interference, resulting in missed detections and false detections; second, directly using bounding rectangles or original discrete contours as detection objects cannot accurately represent the true geometric boundaries and is prone to introducing boundary noise, resulting in poor accuracy of the extracted contour data, which in turn seriously reduces the accuracy of SPI detection.

[0034] This application aims to address at least one of the technical problems existing in the prior art. To this end, this application proposes a solder contour extraction method, apparatus, electronic device, and storage medium, which can improve the accuracy of the extracted contour data, thereby improving the accuracy of SPI detection.

[0035] Reference Figure 1The solder contour extraction method provided in the embodiments of this application may include, but is not limited to, the following steps 101 to 106: Step 101: Obtain the two-dimensional image and three-dimensional height map of the target object.

[0036] Step 102: Perform image segmentation on the two-dimensional image to obtain the initial mask.

[0037] Step 103: Based on the height information in the 3D height map, the initial mask is corrected to obtain the target mask.

[0038] Step 104: Extract the edges of the target mask to obtain a closed contour point sequence.

[0039] Step 105: Perform geometric fitting on the closed contour point sequence to convert the closed contour point sequence into an initial geometric primitive sequence containing line segments and arc segments.

[0040] Step 106: Based on preset geometric rules, perform primitive position alignment and intersection reconstruction on the initial geometric primitive sequence to obtain the target geometric primitive sequence.

[0041] Step 107: Generate solder contour data of the target detection object based on the target geometric primitive sequence.

[0042] In step 101 of some embodiments, the target detection object is typically the area of ​​the circuit board to be inspected after the solder printing process in surface mount technology. In actual inspection scenarios, since single image data often cannot fully reflect the true physical form of the solder, this step simultaneously acquires a two-dimensional image and a three-dimensional height map of the target detection object at the same physical location. The two-dimensional image mainly records the planar visual features of the target detection object's surface, such as the contrast between the solder and the circuit board substrate, texture, and contour; while the three-dimensional height map records the physical height data of various spatial points on the target detection object's surface in the vertical direction. By simultaneously acquiring these two different dimensions of data, a sufficient data foundation can be provided for the subsequent comprehensive and accurate extraction of the actual solder area, avoiding the limitations of single-dimensional data.

[0043] In step 102 of some embodiments, image segmentation aims to divide pixels in a two-dimensional image into target areas belonging to solder and other areas belonging to the background. By analyzing and classifying the pixel features in the two-dimensional image, areas predicted to be solder can be extracted to form mask data for marking target locations, i.e., an initial mask. This initial mask typically exists in the form of a binarized image, with pixels within the mask representing initially identified solder areas. It should be noted that since two-dimensional images only contain planar visual information, they are easily affected by factors such as light source reflections, shadows from surrounding components, or similar substrate colors in actual production environments. Therefore, the initial mask obtained solely based on two-dimensional image segmentation is often a preliminary prediction result, and its edges may have certain deviations or contain some misjudged non-solder areas.

[0044] Please see Figure 2 In some embodiments, step 102 may include, but is not limited to, steps 201 to 204.

[0045] Step 201: Input the two-dimensional image into the pre-trained deep learning segmentation model and output a preliminary prediction mask.

[0046] Step 202: Extract the color features of the preliminarily predicted mask coverage area.

[0047] Step 203: Within the entire area of ​​the two-dimensional image, calculate the color similarity based on color features and extract the color-assisted segmentation mask.

[0048] Step 204: The preliminary prediction mask and the color-assisted segmentation mask are fused to generate the initial mask.

[0049] In step 201 of some embodiments, the pre-trained deep learning segmentation model is trained using a large amount of historical image data containing solder samples, and is capable of identifying the visual features of solder areas from complex circuit board backgrounds. The previously acquired two-dimensional image of the target object is used as input data, and after forward propagation and feature extraction through the model's internal network layers, each pixel in the two-dimensional image is classified and predicted. The final output of the model is a preliminary prediction mask, which indicates the pixel areas that the model initially identifies as solder at the semantic level. It should be noted that, due to the limitations of the deep learning model during actual inference due to differences in the data distribution of the training set or specific lighting changes on-site, the preliminary prediction mask may have missed or misjudged areas at the edges of the solder pads.

[0050] In step 202 of some embodiments, to compensate for potential local edge biases in a single prediction by the deep learning model, this step utilizes the aforementioned preliminary prediction mask as a region guide to locate the corresponding mask-covered area in the original two-dimensional image. Subsequently, specific color information of pixels within this covered area is extracted, such as obtaining statistical data on the color distribution and mean of pixels in a specific color space (e.g., RGB or HSV), thereby forming color features that can objectively characterize the true appearance of solder on the current target detection object. This process utilizes the high-confidence core region predicted by the model to dynamically extract the solder reference color attributes under the current actual working conditions and lighting conditions.

[0051] In step 203 of some embodiments, after obtaining the reference color features characterizing the current solder surface, these features are used as a reference standard to globally traverse all pixels of the entire two-dimensional image. The distance or similarity between the color data of each pixel in the two-dimensional image and the aforementioned reference color features is calculated. For pixels whose color similarity meets a set threshold condition, they are determined to be regions with solder color attributes, and these pixels are combined to generate another set of binary data independent of the model prediction, namely a color-assisted segmentation mask. This auxiliary mask can capture solder regions with the same color features that may be missed by the deep learning model based on simple optical color consistency.

[0052] In step 204 of some embodiments, the preliminary prediction mask based on model semantic feature extraction and the color-assisted segmentation mask based on physical appearance feature extraction are superimposed and logically operated on in spatial coordinates. By fusing the two, the robustness of deep learning in overall structure recognition and the sensitivity of color similarity calculation in capturing local details can be combined. The mask data obtained after fusion integrates the advantages of the two two-dimensional planar features mentioned above, thereby generating a more complete and accurate initial mask, which can be output to subsequent steps for further height correction.

[0053] Through steps 201 to 204 above, this embodiment first uses a deep learning model to obtain a preliminary predicted region of the target object, then adaptively extracts the actual color features of the region and performs color similarity matching globally to generate an auxiliary mask, and finally fuses the model prediction result with the color auxiliary result. This cascaded processing method, which combines deep learning semantic segmentation with traditional image color feature matching, overcomes the segmentation defects that single detection models are prone to when facing complex circuit board backgrounds, interference from similar substrates, or edge reflections, improves the integrity and accuracy of the two-dimensional image segmentation stage, and provides a high-quality two-dimensional initial mask benchmark for subsequent correction based on three-dimensional physical height.

[0054] In step 103 of some embodiments, to overcome the limitations of the aforementioned two-dimensional image segmentation, this step introduces objective physical dimensions from the three-dimensional height map as a verification condition. Specifically, the initial mask is aligned with the three-dimensional height map in spatial coordinates to obtain the actual height information of each pixel covered by the initial mask in the three-dimensional height map. Since real solder has a certain physical thickness on the circuit board, and the color interference items of reflective points or planes in the two-dimensional image usually lack corresponding thickness features, by evaluating this height information, it is possible to effectively identify and eliminate forged areas or misjudged pixels in the initial mask that do not conform to the solder height characteristics. After the height information is constrained and corrected, the remaining mask area can more realistically reflect the actual coverage of the solder, thereby obtaining a more accurate target mask.

[0055] Please see Figure 3 In some embodiments, step 103 may include, but is not limited to, steps 301 to 303.

[0056] Step 301: Obtain the height value of each pixel in the initial mask in the three-dimensional height map. If the height value is lower than the preset height threshold, remove the corresponding pixel from the initial mask to obtain the first modified mask.

[0057] Step 302: For each pixel in the first correction mask, construct a local window centered on that pixel and calculate the height gradient value within each local window.

[0058] Step 303: If the height gradient value of the local window meets the preset gradient anomaly threshold condition, the corresponding pixel is removed from the first correction mask to obtain the target mask.

[0059] In step 301 of some embodiments, the initial mask obtained from the two-dimensional image segmentation in the preceding steps is mapped and aligned with the three-dimensional height map in spatial coordinates. In actual production scenarios, the silkscreen markings, exposed copper foil, or reflective areas on the substrate of the printed circuit board may have colors and textures very similar to solder in the two-dimensional image, thus being incorrectly included in the initial mask; however, these interference items are merely two-dimensional surfaces in physical space, with almost no thickness. Therefore, this step extracts the actual physical height value corresponding to each pixel in the initial mask in the three-dimensional height map and compares it with a preset height threshold. For pixels with height values ​​lower than the height threshold, they are determined to be reflective surfaces of the two-dimensional plane or thin non-solder layer interference, thus being removed from the effective range of the initial mask. After this initial screening based on height, the output mask data is the first corrected mask, which effectively eliminates the planar misjudgment areas caused by two-dimensional image segmentation.

[0060] In step 302 of some embodiments, the height gradient value is used to objectively reflect the steepness or height change rate of the target surface morphology within a local spatial range. In the acquired 3D height map, besides the planar misjudgment area in step 301, there may also be defects caused by solder stickiness during demolding, or local high-frequency noise from the 3D scanning device itself. These anomalous features often have high height values ​​and cannot be eliminated in step 301, but they manifest as extremely steep abrupt changes within the local space. To identify these features, for each pixel retained by the first correction mask, a local window with a specific size (e.g., a 3×3 or 5×5 pixel matrix) centered on that pixel is defined on the 3D height map. Subsequently, a differential algorithm is used to calculate the height difference change between adjacent pixels within this local window, thereby quantifying the height gradient value of the corresponding central pixel.

[0061] In step 303 of some embodiments, the preset gradient anomaly threshold condition is a rejection criterion based on prior knowledge of the physical smoothness of the standard printed solder surface. For a normally printed solder paste surface, the height change in a local area is usually relatively gentle, and the gradient value is within a reasonable distribution range. However, for the aforementioned spike defects or three-dimensional morphological noise, the height gradient value within the local window will be abnormally drastic, exhibiting a large numerical jump. Therefore, a series of height gradient values ​​calculated in step 302 are compared with the gradient anomaly threshold condition. When the height change rate of a pixel exceeds the physical law of a normal solder surface, i.e., meets the preset gradient anomaly threshold condition, the pixel is determined to belong to three-dimensional morphological interference such as spikes or probe noise, and is thus removed from the first correction mask. After the above screening, the set of retained pixels constitutes the final target mask.

[0062] Through steps 301 to 303 above, this embodiment first uses the absolute height threshold in the three-dimensional height map to eliminate the interference of two-dimensional planes with almost no thickness in the initial mask and obtain the first corrected mask; then, by constructing a local window and calculating the height gradient value, the hidden sharpening defects or three-dimensional local noise in the first corrected mask are eliminated by using the height change rate of the physical surface, the misjudged areas left by the two-dimensional image segmentation and the three-dimensional local abnormal noise are cleared, and the final output target mask can closely fit the coverage of the real solder.

[0063] In step 104 of some embodiments, the edges of the target mask are extracted to obtain a closed contour point sequence. After obtaining an accurate target mask, in order to further obtain the geometry of the solder, the data format needs to be converted from a region representation to a boundary representation. Specifically, the outer boundary of the target mask is traversed and extracted to identify the edge pixels located at the junction of the foreground and background of the target mask. Subsequently, these discrete edge pixels are arranged continuously according to their topological connection order in physical space (e.g., clockwise or counterclockwise direction) to construct an ordered set of pixel coordinates connected end to end. This set is the closed contour point sequence, which completely outlines the outer contour shape of the target mask and provides the basic discrete point data for subsequent vectorization calculations.

[0064] Please see Figure 4 In some embodiments, step 104 may include, but is not limited to, steps 401 to 403.

[0065] Step 401: Perform connected component analysis on the target mask to determine at least one candidate target region and remove noisy connected components with an area smaller than a preset area threshold.

[0066] Step 402: Perform internal hole filling and morphological opening and closing operations on the screened candidate target regions, and extract the outer contour of each candidate target region after processing to obtain the initial contour point sequence.

[0067] Step 403: Resample and smooth the initial contour point sequence to obtain a closed contour point sequence.

[0068] In step 401 of some embodiments, connected component analysis is performed on the target mask obtained in the aforementioned steps. This involves dividing the set of pixels in the mask that have the same pixel state (e.g., all marked as foreground pixels) and are spatially adjacent into independent connected regions, thereby identifying at least one candidate target region in the entire image space. Due to random noise during the acquisition process of the imaging device, or minor misjudgments in the preceding multimodal mask correction stage, isolated small patches often exist in the target mask that are outside the main solder area. Therefore, the pixel area size contained in each candidate target region is calculated and compared with a preset area threshold. For small connected regions with an area smaller than the preset area threshold, they are determined to be noise regions that do not possess the true physical scale of solder, and their pixel states are removed. This operation effectively filters out spatially discrete interference and retains candidate target regions that can characterize the true shape of the solder body.

[0069] In step 402 of some embodiments, after removing area noise, the remaining candidate target areas may contain scattered non-target pixels, i.e., internal holes, due to abnormal local solder reflection or uneven thickness; simultaneously, their outer boundaries may also have burrs or small local gaps. By performing internal hole filling processing, background pixels completely surrounded by candidate target areas can be identified and converted into target pixels, thereby repairing structural defects within the area and ensuring the integrity of the solder area. Furthermore, morphological opening and closing operations are used to perform morphological correction on the area, which can effectively eliminate small isolated protrusions on the boundary and fill small depressions on the boundary. After completing the above internal repair and boundary smoothing, the outer boundary pixels of each candidate target area are tracked and extracted, and these boundary pixel points are continuously recorded according to spatial topological order, thereby obtaining an initial contour point sequence reflecting the initial boundary morphology of each area.

[0070] In step 403 of some embodiments, although the initial contour point sequence has been extracted in the previous step, due to the inherent characteristics of digital image discretization sampling, the boundary points of the pixel grid are often unevenly distributed in space and have local step-like pixel jitter. Therefore, this step first resamples the initial contour point sequence, re-interpolates and redistributes the contour points according to the set step size, so that adjacent discrete points in the sequence maintain a uniform and reasonable interval in physical space, avoiding data redundancy caused by excessively dense points or feature loss caused by excessively sparse points in subsequent calculations. Subsequently, smoothing processing is used to reduce the high-frequency jitter caused by pixel discretization, making the contour trend smoother. The point set after resampling and homogenization processing is the final closed contour point sequence, which, as standardized high-quality discrete coordinate data, is output to the next process for vectorized geometric fitting.

[0071] Through steps 401 to 403 described above, this embodiment first removes isolated noise in the mask space using connected region area filtering. Then, it repairs internal defects and boundary burrs in the solder area through internal hole filling and morphological operations. Finally, it performs pixel pitch homogenization and jitter reduction on the extracted boundary pixels through resampling and smoothing. This series of morphological and image standardization cleaning processes effectively eliminates residual structural defects and image discretization interference in the two-dimensional mask, providing a high-quality, uniform, and smooth discrete contour data benchmark for subsequent geometric primitive fitting.

[0072] In step 105 of some embodiments, since the extracted closed contour point sequence is composed of dense discrete pixels, it not only has a large data volume but also easily preserves the jagged edges of the image itself. Therefore, this step uses geometric fitting to reduce the dimensionality and structure the sequence. Specifically, according to a specific fitting algorithm, the continuously distributed discrete contour points are fitted into a standard mathematical geometric shape. Considering that the solder shape on the standard pads on the circuit board is usually composed of straight edges and rounded corners, the target shape for fitting is set as line segments and arc segments. After fitting, the closed contour, which was originally composed of a large number of pixels, is transformed into a vectorized sequence composed of multiple line segments and arc segments spliced ​​together end to end, i.e., the initial geometric primitive sequence.

[0073] Please see Figure 5 In some embodiments, step 105 may include, but is not limited to, steps 501 to 504.

[0074] Step 501: The closed contour point sequence is segmented using a polyline segmentation algorithm to generate an initial line segment sequence consisting of multiple initial line segments connected end to end.

[0075] Step 502: Traverse the initial line segment sequence, extract local line segment combinations consisting of at least two connected initial line segments in sequence, and obtain the target contour point subset corresponding to the local line segment combinations.

[0076] Step 503: Calculate the first fitting residual from the target contour point subset to the local line segment combination, and the second fitting residual generated when fitting the target contour point subset to the candidate arc segment.

[0077] Step 504: If the second fitting residual is less than the first fitting residual, and the candidate arc segment satisfies the preset arc quantization constraint, then the local line segment combination in the initial line segment sequence is replaced with the candidate arc segment to obtain the initial geometric primitive sequence.

[0078] In step 501 of some embodiments, the closed contour point sequence extracted in the aforementioned steps is a series of dense discrete pixel coordinate points. To initially structure it and reduce the dimensionality of subsequent calculations, this step introduces a piecewise linear segmentation algorithm (such as the Douglas-Peucker algorithm) to approximate the discrete point set into a polygon. This algorithm, by setting a distance tolerance, divides the overall closed contour point sequence into multiple line segments according to the spatial distance variation trend, thereby converting the originally large number of discrete pixels into a coarse polygonal contour composed of multiple line segments connected end-to-end. The dataset output by this transformation is the initial line segment sequence, which achieves dimensionality reduction from discrete image features to initial geometric features, providing basic units for subsequent classification and evaluation of lines and arcs.

[0079] In step 502 of some embodiments, since the corners or specific areas of the actual solder pads are usually composed of arcs in their physical form, and the above-mentioned polyline segmentation algorithm forcibly divides these arcs into multiple polylines during the approximation process, this step performs a global traversal of the initial line segment sequence to identify these arc regions. It selects adjacent initial line segments one by one as local line segment combinations to be evaluated. Simultaneously, based on the position mapping relationship, it backtracks to the original closed contour point sequence and extracts the original discrete pixels originally covered by these local line segment combinations, forming a corresponding subset of target contour points. This subset of target contour points serves as the original verification data for subsequent comparison of the fitting residuals between line segments and arc segments.

[0080] In step 503 of some embodiments, the sum of squared vertical distances from each discrete point in the target contour point subset to the corresponding local line segment combination is first calculated as the first fitting residual to characterize the error of the polyline model. Then, assuming that the target contour point subset originally belongs to a circular arc, the subset is calculated using a circular fitting algorithm to obtain a candidate arc segment that best approximates it. Similarly, the radial error from each discrete point in the point set to the candidate arc segment is calculated as the second fitting residual to characterize the error of the circular arc model.

[0081] In step 504 of some embodiments, if the second fitting residual is less than the first fitting residual, and the candidate arc segment satisfies the preset arc quantization constraint, then the local line segment combination in the initial line segment sequence is replaced with the candidate arc segment to obtain the initial geometric primitive sequence. In this step, even if the second fitting residual is smaller, it only indicates that the mathematical approximation of the circular arc model is higher. To prevent misjudging straight line edges with burrs as circular arcs, it is also necessary to introduce arc quantization constraints combined with physical engineering priors for rigorous verification. This is the core difference between this embodiment and the conventional polyline approximation algorithm at the algorithm level.

[0082] Specifically, the preset arc quantization constraints include: The arc height of the candidate arc segment is greater than or equal to a preset minimum arc height threshold. In specific implementations, the minimum arc height threshold is preferably 2 to 3 pixels, and its actual value needs to be adaptively adjusted according to the resolution of the acquired image. The higher the resolution, the higher the threshold should be set. This constraint ensures that the identified arc has substantial physical curvature.

[0083] The ratio of the arc height to the chord length of the candidate arc segment is greater than or equal to a preset ratio threshold. Let the arc height of the candidate arc segment be h, the chord length be c, and the preset ratio threshold be... That is, it is required to meet the following conditions. In a preferred embodiment, The value ranges from 0.02 to 0.05. Furthermore, for larger pads or solder paste targets, The optimum value is 0.02 to 0.03; for small targets or images with significant noise,

[0084] The ratio is preferably between 0.04 and 0.05. If the ratio is less than the specified threshold, it indicates that the candidate arc is too flat relative to its chord length. Even if the arc fitting residual is small, it is very likely just a slight jitter of the straight line boundary. Therefore, it is rejected as an arc segment and retained as a line segment.

[0085] For oblong pads, capsule-shaped solder paste, or semi-circular end structures, when a local line segment combination is located between two parallel initial line segments, let the physical distance between the two parallel lines be . Then the ideal radius of the semi-circular end structure is approximately: At this point, dual proportional tolerance requirements must be met simultaneously: (1) The ideal radius R of the candidate arc segment is half the distance between the two parallel initial line segments. The absolute value of the difference between the two must be less than or equal to a first tolerance threshold. This first tolerance threshold is set as follows: The proportionality coefficient The preferred value is between 0.10 and 0.20. That is, it satisfies the formula: .

[0086] (2) The absolute value of the difference between the arc height h of the candidate arc segment and the ideal radius R must be less than or equal to the second tolerance threshold. This second tolerance threshold is set as follows: The proportionality coefficient The preferred value is between 0.10 and 0.25. That is, it satisfies the formula: .

[0087] By using the above joint proportional constraints, we can accurately identify and preserve the true semi-circular end structure, while effectively suppressing the pseudo-circular arc caused by local edge noise.

[0088] Through steps 501 to 504 above, this embodiment first employs a piecewise linear segmentation algorithm for data dimensionality reduction, and then, based on residual comparison, integrates an arc quantization constraint mechanism based on image resolution, target size, and geometric structure. This not only avoids misidentifying approximate straight lines or noise fluctuations as arcs, but also effectively preserves true rounded corners, semi-circular ends, and arc-shaped solder boundaries, improving the stability and geometric reliability of subsequent output primitives.

[0089] In step 106 of some embodiments, despite preliminary geometric fitting, line segments and arc segments in the initial geometric primitive sequence still exhibit slight tilts or positional deviations due to limitations in image resolution or local noise, failing to fully conform to actual industrial design specifications. In this step, preset geometric rules represent prior specifications for standard pad designs, such as edges being horizontal or vertical. Based on these rules, the poses of each primitive in the initial geometric primitive sequence are calibrated, forcing slightly tilted line segments to align with the prescribed directions. Simultaneously, due to the movement of primitive positions, the original connections between adjacent primitives are broken; therefore, it is necessary to extend or reduce adjacent primitives, recalculate their intersection positions, and establish connections—i.e., intersection point reconstruction. After position alignment and intersection point reconstruction, not only are the local distortions caused by image discretization eliminated, but the corners and straight edges of the solder contour are also restored, thereby outputting a standardized target geometric primitive sequence.

[0090] Please see Figure 6 In some embodiments, step 106 may include, but is not limited to, steps 601 to 603.

[0091] Step 601: Extract the orientation angles of each line segment in the initial geometric primitive sequence, perform cluster analysis, and construct multiple reference baseline support lines.

[0092] Step 602: Project the endpoints of each line segment onto the reference datum support line that matches the direction, so that each line segment snaps to the corresponding reference datum support line, thus completing the alignment of the graphic elements.

[0093] Step 603: Calculate the intersection of two adjacent reference datum support lines, and use the intersection as the common endpoint to connect the line segments that are attached to the two adjacent reference datum support lines to obtain the target geometric primitive sequence.

[0094] In step 601 of some embodiments, the orientation angle of each line segment is first extracted, and these line segments are clustered according to a set orientation tolerance. For further normalization, for each orientation cluster, it can be attempted to be snapped to a specific orientation according to industry standards, such as 0°, 45°, 90°, or 135°. Subsequently, based on the midpoint position of the clustered line segments and the target orientation, a corresponding reference baseline support line is constructed. It should be noted that the reference baseline support line is not a solid line segment with a finite length, but rather an ideal geometric baseline extending infinitely in a specific direction, constructed based on the orientation, position, and engineering priors of one or more line segments. This support line characterizes the physical straight-line position that each line segment should ideally be in, providing a geometric correction benchmark for subsequent operations such as endpoint correction, merging adjacent line segments, and joint reconnection.

[0095] In step 602 of some embodiments, after obtaining the ideal reference support line, this step projects the two endpoints of each line segment in the initial geometric primitive sequence onto its corresponding reference support line along an orthogonal direction. This projection snapping mechanism forces endpoints that have slightly deviated from the straight line due to underlying image segmentation noise or pixel discretization to revert to the ideal straight line trajectory. This endpoint snapping process effectively eliminates high-frequency jitter at the boundary, achieves precise alignment of primitives in local spatial positions, and also makes it easier to merge discrete line segments in the same direction in subsequent processing.

[0096] In step 603 of some embodiments, for two spatially adjacent line segments, if both have been successfully constructed and snapped to their respective reference support lines, the mathematical intersection point of these two spatial support lines is directly calculated. Then, this mathematical intersection point is used as the common endpoint of the two adjacent line segments, and they are stretched or trimmed to this intersection point for connection. The purpose of this step is to ensure that adjacent line segments achieve true intersection and closure in geometric topology, rather than merely approximation in numerical coordinates. For the rectangular, L-shaped, T-shaped edges or straight-edge structures in rounded rectangles commonly found in integrated circuit pads, this joint reconnection operation can improve the geometric consistency of corner points, thereby generating a more accurate sequence of target geometric primitives.

[0097] Please see Figure 7 In some embodiments, step 106 may also include, but is not limited to, steps 701 to 703.

[0098] Step 701: Identify the target arc segment located between two parallel reference datum support lines in the initial geometric primitive sequence.

[0099] Step 702: Based on the positions of the two parallel reference datum support lines, correct the center and radius parameters of the target arc segment so that the two endpoints of the corrected target arc segment are connected to the two parallel reference datum support lines respectively.

[0100] Step 703: Identify the offset short line segment located between the two intersecting reference support lines as corner artifacts, delete the corner artifacts, and extend the two intersecting reference support lines to the intersection.

[0101] In step 701 of some embodiments, when traversing the primitive sequence, specific topological combinations are detected. When an arc primitive is found to be located between two parallel line segments in spatial topology, and the reference support directions of these two adjacent line segments are consistent, it is determined whether the radius of the arc and the spatial distance between these two parallel support lines approximately satisfy a semicircular connection relationship. If so, the arc is identified as a semicircular bridging structure, i.e., the target arc segment described in this step.

[0102] In step 702 of some embodiments, once the semicircular bridging structure is confirmed, the original local fitting parameters of the arc are discarded. Instead, two parallel reference support lines are used as absolute physical constraints to recalculate the center coordinates, radius, start point, end point, and start and end angles of the target arc segment. Through this forced reconstruction, it is ensured that the two ends of the corrected target arc segment can accurately fall on the two parallel support lines, and its overall shape conforms to the standard semicircular connection specification between the two parallel lines. This step corrects local primitives through global geometric constraints and is particularly suitable for high-precision extraction of typical SMT engineering structures such as oblong pads, capsule-shaped solder paste, and semicircular end pads.

[0103] In step 703 of some embodiments, during the low-level processing of image segmentation and contour extraction, due to pixel jaggedness, contour sampling errors, or interference from local optical noise, areas that should have right-angled intersections often exhibit additional burrs or fine diagonal lines. Therefore, when a short line segment is detected between two approximately orthogonal main line segments, and the direction angle of this short line segment is neither close to the preceding line segment nor the following line segment, the algorithm classifies it as a corner artifact short line caused by noise. During processing, the theoretical intersection point of the reference support lines corresponding to the preceding and following main line segments is directly calculated. The endpoint of the preceding line segment and the starting point of the following line segment are extended and stretched to this theoretical intersection point to complete the closure, and the corner artifact short line located in the middle is directly deleted. This corner refinement mechanism effectively removes local burrs, ensuring that the output geometric primitive sequence has engineering rationality at the corners.

[0104] In step 107 of some embodiments, after the primitive extraction and regularization correction are completed, the final target geometric primitive sequence and its structured parameters such as vertex coordinates, line segment lengths, arc center and radius are summarized and formatted. The output data is highly accurate solder contour data, which can be directly sent to the downstream inspection process for calculating key quality indicators such as solder area, volume or offset. Specifically, for line segments, the system outputs straight line boundaries or linear interpolation commands; for circular arc segments, it outputs circular arc primitives and their corresponding center, radius, start and end angles and directions; for elliptical arc segments, it outputs elliptical arc approximations or discretizes them into multiple arcs / line segments with a set precision. The objects generated by the above conversion can be directly used as solder inspection windows, pad candidate areas, solder paste measurement areas, measurement boundaries for area / volume / height offsets of SPI devices, or automatically programmed templates that can be manually edited and reused.

[0105] Furthermore, in some embodiments, to ensure the absolute reliability of the generated detection objects in scenarios without Gerber programming, after outputting the target geometric primitive sequence, a step of quality evaluation and backtracking of the generated target geometric primitive sequence is included. Specifically, the geometric quality of the current primitive is evaluated through a series of quantitative indicators, including but not limited to: the numerical difference between the area of ​​the segmented region and the area of ​​the primitive's closed region, the average or maximum distance from the original discrete contour to the fitted primitive boundary, the number of finally generated primitive segments, the arc fitting residual, the closure endpoint error, and the stability of the corner points. Simultaneously, its matching degree with typical pad prior shapes is also evaluated. When one or more of the above quality evaluation indicators fall below a set safety threshold, the system automatically triggers a backtracking strategy. For example, the initial image segmentation threshold is adaptively adjusted before re-extracting the contour, the smoothing intensity of the contour or the coarse segmentation tolerance of the RDP algorithm is adjusted, or even the arc fitting module is directly disabled, outputting only line segment polygons. For extremely complex local regions, the original discrete contour can be retained as a manually verified object, or a simple outer rectangle can be used as a low-priority fallback detection window in the lowest-level logic to ensure the continuity of the automatic detection process.

[0106] In some embodiments, the system can also perform structural identification and parameter configuration of multi-shaped pads or solder objects based on the geometric topology of the refined target geometric primitive sequence. Specifically, the topological features of the closed primitive sequence are analyzed: when the target consists of four main line segments and adjacent line segments are orthogonal, the system identifies it as a rectangular pad; when the target consists of two parallel line segments and two semicircular arcs connected end to end, it is identified as an oblong pad; when the target consists of a mixture of multiple straight edges and rounded arcs, it is identified as a rounded rectangular pad; if the target consists of a single closed arc or multiple seamlessly connected arcs, it is determined to be a circular pad; and targets composed of a mixture of multiple lines and irregular arcs are classified as irregular pads. After completing the identification of the specific shape, the system will automatically select and match the corresponding SPI measurement parameters, tolerance range, and defect judgment rules, thereby significantly improving the targeting and accuracy of defect detection.

[0107] In one embodiment of 3D solder paste inspection, the method further includes constructing a 3D spatial physical relationship between bare board pads and printed solder. Specifically, the segmentation results of a 2D image are fused with a 3D height map. Regions exhibiting height protrusions are preferentially identified and extracted as solder objects, while regions in the low-height range of the bare board, combined with specific reflection intensity or color characteristics, are independently identified as pad objects. Subsequently, the aforementioned structured segmentation and geometric refinement processes are independently performed on the pad contours and solder contours, generating their respective high-precision closed primitive sequences. Based on this, the system accurately calculates advanced inspection indicators such as the center physical offset, area ratio, coverage, boundary deviation value, and bridging short-circuit risk between the corresponding pads and solder objects through spatial coordinate mapping, achieving quantitative analysis of the assembly relationship of the inspected objects without design data.

[0108] Through steps 101 to 107 above, this embodiment first acquires a two-dimensional image and a three-dimensional height map of the target object, and uses the height information to correct the initial mask generated by the two-dimensional image segmentation, eliminating misjudgments caused by lighting and background interference in the two-dimensional image, thus ensuring the accuracy of mask extraction. Secondly, it performs geometric fitting on the edge point sequence of the target mask, converting it into a combination of line segments and arc segments, achieving structured noise reduction of the contour edges. Finally, based on industrial prior geometric rules, it performs position alignment and intersection point reconstruction on the fitted primitive sequence, effectively correcting local edge deformation and corner distortion caused by discretized imaging. Overall, this embodiment, through step-by-step processing from the pixel domain to the geometric domain, can improve the accuracy of the extracted contour data, thereby improving the accuracy of SPI detection.

[0109] Please see Figure 8 This application also provides a solder contour extraction device, which can implement the above-mentioned solder contour extraction method, including: The acquisition module is used to acquire the two-dimensional image and three-dimensional height map of the target object. The image segmentation module is used to segment two-dimensional images to obtain an initial mask; The height correction module is used to correct the initial mask based on the height information in the 3D height map to obtain the target mask; The contour extraction module is used to extract the edges of the target mask and obtain a sequence of closed contour points; The geometry fitting module is used to perform geometry fitting on a closed contour point sequence, converting the closed contour point sequence into an initial geometric primitive sequence containing line segments and arc segments; The primitive reconstruction module is used to perform primitive position alignment and intersection reconstruction on the initial geometric primitive sequence based on preset geometric rules to obtain the target geometric primitive sequence; The generation module is used to generate solder contour data of the target detection object based on the target geometric primitive sequence.

[0110] In contrast to the problems in the background art, the solder contour extraction method provided in this application firstly acquires a two-dimensional image and a three-dimensional height map of the target object. Based on the height information in the three-dimensional height map, the initial mask obtained from the two-dimensional image segmentation is corrected. By combining multi-dimensional data, misjudged areas that may occur in the single two-dimensional image segmentation are eliminated, resulting in a more accurate target mask. Secondly, the edges of the target mask are extracted to obtain a closed contour point sequence, and the closed contour point sequence is geometrically fitted to convert it into an initial geometric primitive sequence containing line segments and arc segments. This transforms discrete pixels into structured geometric features, reducing edge interference caused by local pixel fluctuations. Finally, based on preset geometric rules, primitive position alignment and intersection reconstruction are performed on the initial geometric primitive sequence to obtain a target geometric primitive sequence after correcting positional deviations. The above steps work synergistically. Through the gradual processing of height information correction, geometric fitting, and rule reconstruction, the impact of image noise and discretization errors is reduced, improving the accuracy of the extracted contour data and thus enhancing the accuracy of SPI detection.

[0111] This application also provides an electronic device, including: At least one memory; At least one processor; At least one program; The program is stored in a memory, and the processor executes the at least one program to implement the solder contour extraction method described above. The electronic device can be any smart terminal, including mobile phones, tablets, personal digital assistants (PDAs), and in-vehicle computers.

[0112] Please see Figure 9 , Figure 9 The hardware structure of an electronic device according to another embodiment is illustrated. The electronic device includes: The processor 901 can be implemented using a general-purpose CPU (Central Processing Unit), microprocessor, application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of this application. The memory 902 can be implemented in the form of ROM (Read-Only Memory), static storage device, dynamic storage device, or RAM (Random Access Memory). The memory 902 can store the operating system and other application programs. When the technical solutions provided in the embodiments of this specification are implemented through software or firmware, the relevant program code is stored in the memory 902 and is called and executed by the processor 901 using the solder contour extraction method of the embodiments of this application. The 903 input / output interface is used to implement information input and output. The communication interface 904 is used to enable communication and interaction between this device and other devices. Communication can be achieved through wired means (such as USB, Ethernet cable, etc.) or wireless means (such as mobile network, WIFI, Bluetooth, etc.). Bus 905 transmits information between various components of the device (e.g., processor 901, memory 902, input / output interface 903, and communication interface 904); The processor 901, memory 902, input / output interface 903, and communication interface 904 are connected to each other within the device via bus 905.

[0113] This application embodiment also provides a storage medium, which is a computer-readable storage medium, storing a computer program that, when executed by a processor, implements the above-described solder contour extraction method.

[0114] Memory, as a non-transitory computer-readable storage medium, can be used to store non-transitory software programs and non-transitory computer-executable programs. Furthermore, memory may include high-speed random access memory, and may also include non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid-state storage device. In some embodiments, memory may optionally include memory remotely located relative to the processor, and these remote memories can be connected to the processor via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.

[0115] The embodiments described in this application are for the purpose of more clearly illustrating the technical solutions of this application, and do not constitute a limitation on the technical solutions provided in this application. As those skilled in the art will know, with the evolution of technology and the emergence of new application scenarios, the technical solutions provided in this application are also applicable to similar technical problems.

[0116] Those skilled in the art will understand that the technical solutions shown in the figures do not constitute a limitation on the embodiments of this application, and may include more or fewer steps than shown, or combine certain steps, or different steps.

[0117] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.

[0118] Those skilled in the art will understand that all or some of the steps in the methods disclosed above, as well as the functional modules / units in the systems and devices, can be implemented as software, firmware, hardware, or suitable combinations thereof.

[0119] The terms “first,” “second,” “third,” “fourth,” etc. (if present) in the specification and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms “comprising” and “having,” and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0120] It should be understood that in this application, "at least one (item)" means one or more, and "more than" means two or more. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, "A and / or B" can represent three cases: only A exists, only B exists, and both A and B exist simultaneously, where A and B can be singular or plural. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. "At least one (item) of the following" or similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one of a, b, or c. One (a) can represent: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, and c can be a single or multiple.

[0121] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of the units described above is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. The coupling or direct coupling or communication connection between the shown or discussed units may be through some interfaces, or the indirect coupling or communication connection between the apparatus or units may be electrical, mechanical, or other forms.

[0122] The units described above as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0123] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0124] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes multiple instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned storage medium includes various media capable of storing programs, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0125] The preferred embodiments of the present application have been described above with reference to the accompanying drawings, but this does not limit the scope of the claims of the present application. Any modifications, equivalent substitutions, and improvements made by those skilled in the art without departing from the scope and substance of the embodiments of the present application shall be within the scope of the claims of the present application.

Claims

1. A method for extracting solder contours, characterized in that, include: Acquire the 2D image and 3D height map of the target object; The two-dimensional image is segmented to obtain an initial mask; The initial mask is corrected based on the height information in the three-dimensional height map to obtain the target mask; Extract the edges of the target mask to obtain a closed contour point sequence; Geometric fitting is performed on the closed contour point sequence to convert it into an initial geometric primitive sequence containing line segments and arc segments. This includes: segmenting the closed contour point sequence using a polyline segmentation algorithm to generate an initial line segment sequence composed of multiple sequentially connected initial line segments; traversing the initial line segment sequence and sequentially extracting local line segment combinations composed of at least two connected initial line segments, and obtaining a subset of target contour points corresponding to the local line segment combinations; calculating a first fitting residual from the target contour point subset to the local line segment combinations, and a second fitting residual generated when fitting the target contour point subset to candidate arc segments; if the second fitting residual is less than the first fitting residual, and the candidate arc segments are... If an arc segment satisfies a preset arc quantization constraint, the local line segment combination in the initial line segment sequence is replaced with the candidate arc segment to obtain the initial geometric primitive sequence; wherein, the preset arc quantization constraint includes at least one of the following: the arc height of the candidate arc segment is greater than or equal to a preset minimum arc height threshold; the ratio of the arc height to the chord length of the candidate arc segment is greater than or equal to a preset ratio threshold; when the local line segment combination is located between two parallel initial line segments, the absolute value of the difference between the ideal radius of the candidate arc segment and half the distance between the two parallel initial line segments is less than or equal to a first tolerance threshold, and the absolute value of the difference between the arc height of the candidate arc segment and the ideal radius is less than or equal to a second tolerance threshold; Based on preset geometric rules, the initial geometric primitive sequence is aligned and its intersection points are reconstructed to obtain the target geometric primitive sequence. Based on the target geometric primitive sequence, solder contour data of the target detection object is generated.

2. The method according to claim 1, characterized in that, The step of segmenting the two-dimensional image to obtain an initial mask includes: The two-dimensional image is input into a pre-trained deep learning segmentation model, which outputs a preliminary prediction mask. Extract the color features of the area covered by the preliminary predicted mask; Within the entire area of ​​the two-dimensional image, color similarity is calculated based on the color features to extract a color-assisted segmentation mask; The preliminary prediction mask is fused with the color-assisted segmentation mask to generate the initial mask.

3. The method according to claim 1, characterized in that, The step of correcting the initial mask based on the height information in the three-dimensional height map to obtain the target mask includes: Obtain the height value of each pixel point contained in the initial mask in the three-dimensional height map. If the height value is lower than a preset height threshold, remove the corresponding pixel point from the initial mask to obtain the first modified mask. For each pixel within the first correction mask, a local window centered on that pixel is constructed, and the height gradient value within each local window is calculated. If the height gradient value of the local window meets the preset gradient anomaly threshold condition, the corresponding pixel is removed from the first correction mask to obtain the target mask.

4. The method according to claim 1, characterized in that, The step of extracting the edges of the target mask to obtain a closed contour point sequence includes: Perform connected component analysis on the target mask to determine at least one candidate target region and remove noisy connected components with an area smaller than a preset area threshold; The selected candidate target regions are processed by internal hole filling and morphological opening and closing operations, and the outer contour of each candidate target region after processing is extracted to obtain an initial contour point sequence. The initial contour point sequence is resampled and smoothed to obtain the closed contour point sequence.

5. The method according to claim 1, characterized in that, The initial geometric primitive sequence is aligned and its intersections are reconstructed based on preset geometric rules to obtain the target geometric primitive sequence, including: The orientation angles of each line segment in the initial geometric primitive sequence are extracted and clustered to construct multiple reference baseline support lines. Project the endpoints of each line segment onto the reference datum support line that matches the direction, so that each line segment snaps onto the corresponding reference datum support line, thus completing the alignment of the primitive position. Calculate the intersection point of two adjacent reference datum support lines, and use the intersection point as a common endpoint to connect the line segments attached to the two adjacent reference datum support lines to obtain the target geometric primitive sequence.

6. The method according to claim 5, characterized in that, The step of performing primitive position alignment and intersection reconstruction on the initial geometric primitive sequence based on preset geometric rules to obtain the target geometric primitive sequence further includes: Identify the target arc segment located between two parallel reference datum support lines in the initial geometric primitive sequence; Based on the positions of the two parallel reference datum support lines, the center and radius parameters of the target arc segment are corrected so that the two endpoints of the corrected target arc segment are respectively connected to the two parallel reference datum support lines. Identify the offset short line segment located between the two intersecting reference support lines as a corner artifact, delete the corner artifact, and extend the two intersecting reference support lines to the intersection.

7. A solder contour extraction device, characterized in that, include: The acquisition module is used to acquire the two-dimensional image and three-dimensional height map of the target object. An image segmentation module is used to segment the two-dimensional image to obtain an initial mask; The height correction module is used to correct the initial mask based on the height information in the three-dimensional height map to obtain the target mask; The contour extraction module is used to extract the edges of the target mask to obtain a sequence of closed contour points. The geometric fitting module is used to perform geometric fitting on the closed contour point sequence, converting the closed contour point sequence into an initial geometric primitive sequence containing line segments and arc segments. This includes: segmenting the closed contour point sequence using a polyline segmentation algorithm to generate an initial line segment sequence composed of multiple sequentially connected initial line segments; traversing the initial line segment sequence, sequentially extracting local line segment combinations composed of at least two connected initial line segments, and obtaining a subset of target contour points corresponding to the local line segment combinations; calculating a first fitting residual from the target contour point subset to the local line segment combinations, and a second fitting residual generated when fitting the target contour point subset as candidate arc segments; if the second fitting residual is less than the first fitting residual... If the candidate arc segment satisfies the preset arc quantization constraint, then the local line segment combination in the initial line segment sequence is replaced with the candidate arc segment to obtain the initial geometric primitive sequence; wherein, the preset arc quantization constraint includes at least one of the following: the arc height of the candidate arc segment is greater than or equal to a preset minimum arc height threshold; the ratio of the arc height to the chord length of the candidate arc segment is greater than or equal to a preset ratio threshold; when the local line segment combination is located between two parallel initial line segments, the absolute value of the difference between the ideal radius of the candidate arc segment and half the distance between the two parallel initial line segments is less than or equal to a first tolerance threshold, and the absolute value of the difference between the arc height of the candidate arc segment and the ideal radius is less than or equal to a second tolerance threshold; The primitive reconstruction module is used to perform primitive position alignment and intersection reconstruction on the initial geometric primitive sequence based on preset geometric rules to obtain the target geometric primitive sequence; The generation module is used to generate solder contour data of the target detection object based on the target geometric primitive sequence.

8. An electronic device, characterized in that, include: The device includes a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the solder contour extraction method as described in any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that, The storage medium stores a program that is executed by a processor to implement the solder contour extraction method as described in any one of claims 1 to 6.

Images