Method and apparatus for extracting a gummed colloid

Abstract

This application relates to the field of machine vision technology for adhesive coating, and discloses a method and apparatus for extracting adhesive coatings. This method achieves high-precision alignment of the 3D point clouds of the template workpiece and the workpiece to be measured through a two-step registration strategy combining coarse and fine registration. Based on precise registration, the distance between measurement point regions is calculated and compared with a threshold, accurately distinguishing the adhesive point cloud from the workpiece's main point cloud. This effectively avoids the misjudgment and omission problems that easily occur in traditional 2D vision, significantly improving the accuracy and extraction rate of adhesive extraction. Simultaneously, the division and independent processing mode of multiple measurement point regions can adapt to workpieces with different adhesive coating layouts, achieving effective extraction regardless of whether the adhesive coating path is regular or irregular, greatly expanding the applicability of the method.

Description

Technical Field

[0001] This application relates to the field of machine vision technology for adhesive coating, and discloses a method and apparatus for extracting adhesive coatings. Background Technology

[0002] In adhesive coating processes, the adhesive is a continuous or intermittent strip of adhesive formed by coating equipment such as robots and dispensing guns. Its core function is to achieve sealing, bonding, vibration damping, or filling between components, ensuring that the structure has the required airtightness, connection strength, sound and heat insulation performance, while also meeting specific functional requirements such as aging resistance and corrosion resistance. However, in actual adhesive coating processes, due to the complex morphology of the adhesive, environmental interference, or fluctuations in process parameters, the actual contour, size, and position of the adhesive are often difficult to accurately extract and measure. Inaccurate adhesive extraction and measurement can directly lead to uncontrolled adhesive coating quality, such as deviations in adhesive width, height, or length from design values, causing sealing leaks, insufficient bonding strength, appearance defects, or functional failures, thereby affecting the overall reliability and lifespan of the product.

[0003] The current mainstream method for colloid extraction is a 2D image detection method based on deep learning. This involves collecting and labeling a large amount of colloid coating data, training a neural network model, and then deploying the model to segment the colloid coating area of ​​the input image. However, this deep learning approach to colloid extraction from 2D images has significant limitations: its performance is highly dependent on the quality and coverage of the training data. In real-world production environments, factors such as colloid color variations, background interference, lighting fluctuations, and irregular shapes can easily lead to insufficient model generalization ability, resulting in false or missed extractions, and inaccurate edge extraction. Furthermore, 2D images cannot provide crucial 3D dimensional information such as colloid height and cross-sectional area. In addition, the model has poor interpretability, making it difficult to quickly pinpoint the root cause of detection errors, which is detrimental to process debugging and optimization.

[0004] It is evident that existing technologies cannot accurately and quickly achieve colloid extraction. Summary of the Invention

[0005] Based on this, this application provides a method and apparatus for extracting coated colloids, in order to solve the problem that it is difficult to accurately and quickly extract colloids in the prior art.

[0006] In a first aspect, this application provides a method for extracting a coated colloid, comprising:

[0007] Obtain the 3D point cloud of the template workpiece and the 3D point cloud of the workpiece to be tested; Local features of the 3D point cloud and the point cloud of the workpiece under test are extracted separately, and preliminary alignment processing is performed to obtain coarse registration results; then fine registration is performed based on the coarse registration results to obtain the finely registered point cloud of the workpiece under test. The template sub-point cloud and the sub-point cloud to be measured are determined based on multiple predetermined measurement point areas, the three-dimensional point cloud of the template workpiece, and the finely registered point cloud to be measured. For each measurement point area, calculate the distance from each point in the sub-point cloud to the surface of the corresponding area of ​​the template sub-point cloud, delete the points in the sub-point cloud that are less than the set distance threshold, and finally retain the points as the target colloidal points.

[0008] In one embodiment, local features of the 3D point cloud and local features of the point cloud of the workpiece to be measured are extracted respectively, and preliminary alignment processing is performed to obtain a coarse registration result, including: The Fast Point Feature Histogram (FPFH) is used to extract local features from the 3D point cloud of the template workpiece and the 3D point cloud of the workpiece under test, respectively, and feature matching is performed to obtain the initial transformation matrix. The 3D point cloud of the workpiece to be tested is initially aligned to the 3D point cloud of the template workpiece based on the initial transformation matrix, satisfying the following relationship: ; In the formula, For coarse registration results, Let be the initial transformation matrix. This is the three-dimensional point cloud of the workpiece to be tested.

[0009] In one embodiment, local features of the 3D point cloud of the template workpiece and the 3D point cloud of the workpiece under test are extracted using Fast Point Feature Histogram (FPFH) for feature matching, including: For points in the 3D point cloud of the template workpiece and its FPFH characteristics Find the point with the closest feature Euclidean distance in the 3D point cloud of the workpiece to be tested. It satisfies the following relationship: ; In the formula, It is the 3D point cloud of the template workpiece. A 33-dimensional feature vector of each point, It is the 3D point cloud of the template workpiece. A 33-dimensional feature vector of each point; The obtained point pairs ( , This forms an initial feature matching point pair, and yields multiple sets of feature matching point pairs for the 3D point cloud of the template workpiece and the 3D point cloud of the workpiece to be tested. The initial transformation matrix between matched feature matching point pairs is calculated using the sample consensus algorithm, as follows: ; in, Let N be the initial transformation matrix, and N be the total number of corresponding point pairs. It is a point in the 3D point cloud of the workpiece to be tested. It is a point in the 3D point cloud of the template workpiece. It is the error function of coarse registration.

[0010] In one embodiment, fine registration is performed based on the coarse registration result to obtain the finely registered point cloud to be measured, including: Based on minimizing the distance between each point in the 3D point cloud of the workpiece under test and the tangent plane of the corresponding point in the 3D point cloud of the template workpiece, the fine transformation matrix is ​​calculated as follows: ; in, For fine transformation matrix, for The normal vector at point N is the total number of corresponding point pairs. It is a point in the point cloud to be tested. It is a point in the template point cloud; Based on the coarse registration results and the fine transformation matrix, the finely registered point cloud is calculated as follows: ; in, This is the result of coarse registration.

[0011] In one embodiment, determining the template sub-point cloud and the sub-point cloud to be measured based on a predetermined plurality of measurement point areas, the three-dimensional point cloud of the template workpiece, and the finely registered point cloud to be measured includes: According to the testing requirements, multiple measurement point areas are predefined on the three-dimensional point cloud of the template workpiece. The point clouds corresponding to each measurement point area are extracted from the three-dimensional point cloud of the template workpiece and the finely registered point cloud to be tested, respectively, to obtain the template sub-point cloud and the sub-point cloud to be tested.

[0012] In one embodiment, for each measurement point region, the distance from each point in the sub-point cloud to the surface of the corresponding region of the template sub-point cloud is calculated, and the colloidal point cloud of the region is determined based on the distance and a distance threshold, including: Local fine registration is performed in each measurement point area to obtain the local fine registration sub-point cloud of the measurement point; For each point q in the sub-point cloud to be tested after local fine registration, find the nearest point of q in the template sub-point cloud, and calculate the Euclidean distance between the nearest point in the template sub-point cloud and point q. Set a distance threshold and retain all points whose Euclidean distance is greater than the distance threshold to obtain the colloidal point cloud of the measurement area.

[0013] In one embodiment, local fine registration processing is performed in each measurement point region, including: For each measurement point region, the template sub-point cloud and the sub-point cloud to be measured are processed for local fine registration using registration algorithms such as point-to-surface, point-to-point, surface-to-surface, or Gaussian mixture model-based registration.

[0014] In one embodiment, all points whose Euclidean distance is greater than a distance threshold are retained to obtain the colloidal point cloud of the measurement point region, satisfying the following relationship: ; In the formula, For points in the sub-point cloud to be tested after local fine registration, This is the sub-point cloud to be tested after local fine registration. Distance threshold for The nearest point in the template sub-point cloud.

[0015] Secondly, this application provides an extraction device for coated colloids, comprising: The first module is used to acquire the 3D point cloud of the template workpiece and the 3D point cloud of the workpiece to be tested. The second module is used to extract local features from the 3D point cloud and the point cloud of the workpiece to be measured, respectively, and perform preliminary alignment processing to obtain coarse registration results; and perform fine registration based on the coarse registration results to obtain finely registered point cloud of the workpiece to be measured. The third module is used to determine the template sub-point cloud and the sub-point cloud to be measured based on multiple predetermined measurement point areas, the three-dimensional point cloud of the template workpiece, and the finely registered point cloud to be measured. The fourth module is used to calculate the distance from each point in the sub-point cloud to the surface of the corresponding area of ​​the template sub-point cloud for each measurement point area, delete the points in the sub-point cloud that are less than the set distance threshold, and finally retain the points as the target colloidal points.

[0016] Beneficial effects: The aforementioned method for extracting adhesive colloids achieves high-precision alignment of the 3D point clouds of the template workpiece and the workpiece under test through a two-step registration strategy combining coarse and fine registration. Based on precise registration, distances are calculated for each measurement point region and compared with a threshold, accurately distinguishing the colloid point cloud from the workpiece's main point cloud. This effectively avoids the misjudgment and omission problems common in traditional 2D vision, significantly improving the accuracy and extraction rate of colloid extraction. Furthermore, the division and independent processing mode of multiple measurement point regions can adapt to workpieces with different adhesive coating layouts, achieving effective extraction regardless of whether the coating path is regular or irregular, greatly expanding the method's applicability. Attached Figure Description

[0017] Figure 1 This is a schematic diagram of a method for extracting a coated colloid in one embodiment. Detailed Implementation

[0018] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0019] It should be understood that this application can be applied to the adhesive coating process; it is only an example and not a limitation.

[0020] Please see Figure 1 The method for extracting the adhesive colloid provided in this application includes: Obtain the 3D point cloud of the template workpiece and the 3D point cloud of the workpiece to be tested.

[0021] In this step, point cloud acquisition can be achieved using a 3D camera; this is merely an example and not a limitation. After acquiring the point cloud data, preprocessing such as denoising and downsampling can be performed. Furthermore, the template workpiece is in a glue-free state, while the workpiece under test is in a glued state.

[0022] Local features of the 3D point cloud and the point cloud of the workpiece under test are extracted separately, and preliminary alignment processing is performed to obtain coarse registration results; then fine registration is performed based on the coarse registration results to obtain finely registered point cloud of the workpiece under test.

[0023] In this step, in addition to Fast Point Feature Histograms (FPFH), other feature description methods such as Signature of Histograms of Orientations (SHOT) and 3D Shape Context (3DSC) can be used to obtain local features of the point cloud, or feature extraction networks based on deep learning can be used to obtain local features of the point cloud.

[0024] The template sub-point cloud and the sub-point cloud to be measured are determined based on multiple predetermined measurement point areas, the three-dimensional point cloud of the template workpiece, and the finely registered point cloud to be measured.

[0025] When multiple measurement point areas are predetermined, in addition to manually selecting them on the template point cloud, they can also be achieved through CAD model mapping, automatic recognition and segmentation of three-dimensional features, or automatic region generation methods based on prior dimensions.

[0026] For each measurement point area, calculate the distance from each point in the sub-point cloud to the surface of the corresponding area of ​​the template sub-point cloud, delete the points in the sub-point cloud that are less than the set distance threshold, and finally retain the points as the target colloidal points.

[0027] The aforementioned method for extracting adhesive colloids achieves high-precision alignment of the 3D point clouds of the template workpiece and the workpiece under test through a two-step registration strategy combining coarse and fine registration. Based on precise registration, distances are calculated for each measurement point region and compared with a threshold, accurately distinguishing the colloid point cloud from the workpiece's main point cloud. This effectively avoids the misjudgment and omission problems common in traditional 2D vision, significantly improving the accuracy and extraction rate of colloid extraction. Furthermore, the division and independent processing mode of multiple measurement point regions can adapt to workpieces with different adhesive coating layouts, achieving effective extraction regardless of whether the coating path is regular or irregular, greatly expanding the method's applicability.

[0028] The extraction method of the above-mentioned coated colloid is described below with a complete example: 1. Obtain the 3D point cloud of the template workpiece (without glue). 3D point cloud of the workpiece under test (with adhesive) Both are preprocessed, including denoising and downsampling.

[0029] The Fast Point Feature Histogram (FPFH) is used to extract local features from the 3D point cloud of the template workpiece and the 3D point cloud of the workpiece under test, respectively, and feature matching is performed to obtain the initial transformation matrix. The 3D point cloud of the workpiece to be tested is initially aligned to the 3D point cloud of the template workpiece based on the initial transformation matrix, satisfying the following relationship:

[0030] Specifically, Fast Point Feature Histogram (FPFH) is used to extract local features from the 3D point cloud of the template workpiece and the 3D point cloud of the workpiece under test, respectively, for feature matching, including: For points in the 3D point cloud of the template workpiece and its FPFH characteristics Find the point with the closest feature Euclidean distance in the 3D point cloud of the workpiece to be tested. It satisfies the following relationship: ; In the formula, It is the 3D point cloud of the template workpiece. A 33-dimensional feature vector of each point, It is the 3D point cloud of the template workpiece. A 33-dimensional feature vector of each point; The obtained point pairs ( , This forms an initial feature matching point pair, and yields multiple sets of feature matching point pairs for the 3D point cloud of the template workpiece and the 3D point cloud of the workpiece to be tested. The initial transformation matrix between matched feature matching point pairs is calculated using the sample consensus algorithm, as follows: ; in, Let N be the initial transformation matrix, and N be the total number of corresponding point pairs. It is a point in the 3D point cloud of the workpiece to be tested. It is a point in the 3D point cloud of the template workpiece. It is the error function of coarse registration.

[0031] Based on the coarse registration, the point-to-surface ICP algorithm is used to further optimize the registration accuracy. This algorithm calculates the error function of the fine transformation matrix based on minimizing the distance from each point in the 3D point cloud of the workpiece to the tangent plane of the corresponding point in the 3D point cloud of the template workpiece: ; in, For fine transformation matrix, for The normal vector at point N, where N is the total number of corresponding point pairs. It is a point in the point cloud to be tested. A point in the template point cloud. The final, precisely registered point cloud is obtained:

[0032] In template point cloud Based on the detection needs, multiple measurement point areas (ROIs) are predefined. .Will and Corresponding to each Extract the point clouds separately to obtain template sub-point clouds. and the sub-point cloud to be tested .

[0033] For each measuring point area Perform the following operations: Local fine registration: for and Point-to-surface ICP is used again to improve local alignment accuracy, resulting in... .

[0034] Closest point distance calculation: For For each point q, find its position in... Calculate the Euclidean distance between the nearest point p in the array. .

[0035] Colloidal point extraction: Set distance threshold Retain all that satisfy d> The points constitute the colloidal point cloud of this region. :

[0036] in, For points in the sub-point cloud to be tested after local fine registration, This is the sub-point cloud to be tested after local fine registration. Distance threshold for The nearest point in the template sub-point cloud.

[0037] In this step, an adjustable distance threshold is set. Colloidal point screening is performed by determining whether the distance from each point in the test point cloud to the corresponding surface of the template point cloud is greater than [value missing]. It can achieve the segmentation of complete colloids.

[0038] In this implementation, in addition to using the brute-force search method, the nearest point search between point clouds can typically employ accelerated search methods based on spatial data structures such as KD-Tree and Octree to improve computational efficiency. This is merely an example and not a limitation.

[0039] In summary, the method for extracting adhesive colloids provided in this application achieves high-precision alignment by performing coarse and fine registration between the point cloud of the workpiece to be tested and the point cloud of the template workpiece. Within multiple predefined detection areas, the distance from each point in the point cloud to the corresponding area surface of the template point cloud is calculated, and a distance threshold is set to extract points exceeding the standard surface, thereby accurately segmenting the three-dimensional region of the colloid. This allows for the quantitative evaluation of the colloid's width, height, volume, and continuity. Compared to manual visual or contact measurement methods, this method achieves fully automated, non-contact detection, significantly improving detection efficiency and objectivity. Furthermore, compared to adhesive detection methods based on two-dimensional images, this method directly obtains the true spatial dimensions of the colloid from the three-dimensional point cloud, unaffected by lighting conditions, colloid color, or workpiece surface reflection, resulting in more accurate and reliable measurement results. Moreover, through template registration and regional analysis strategies, even if the workpiece shifts, rotates, or switches between multiple product models within the detection field, this method can still stably and accurately complete colloid extraction and evaluation, demonstrating stronger adaptability and robustness.

[0040] This application also provides an extraction device for coated colloids, comprising: The first module is used to acquire the 3D point cloud of the template workpiece and the 3D point cloud of the workpiece to be tested. The second module is used to extract local features from the 3D point cloud and the point cloud of the workpiece to be measured, respectively, and perform preliminary alignment processing to obtain coarse registration results; and perform fine registration based on the coarse registration results to obtain finely registered point cloud of the workpiece to be measured. The third module is used to determine the template sub-point cloud and the sub-point cloud to be measured based on multiple predetermined measurement point areas, the three-dimensional point cloud of the template workpiece, and the finely registered point cloud to be measured. The fourth module is used to calculate the distance from each point in the sub-point cloud to the surface of the corresponding area of ​​the template sub-point cloud for each measurement point area, delete the points in the sub-point cloud that are less than the set distance threshold, and finally retain the points as the target colloidal points.

[0041] The device for extracting coated colloids can realize all the embodiments of the above-mentioned method for extracting coated colloids and achieve the same beneficial effects, which will not be elaborated here.

[0042] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0043] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of the invention. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A method for extracting a coated colloid, characterized in that, include: Obtain the 3D point cloud of the template workpiece and the 3D point cloud of the workpiece to be tested; Local features of the 3D point cloud and the point cloud of the workpiece under test are extracted separately, and preliminary alignment processing is performed to obtain coarse registration results; then fine registration is performed based on the coarse registration results to obtain the finely registered point cloud of the workpiece under test. The template sub-point cloud and the sub-point cloud to be measured are determined based on multiple predetermined measurement point areas, the three-dimensional point cloud of the template workpiece, and the finely registered point cloud to be measured. For each measurement point area, calculate the distance from each point in the sub-point cloud to the surface of the corresponding area of ​​the template sub-point cloud, delete the points in the sub-point cloud that are less than the set distance threshold, and finally retain the points as the target colloidal points.

2. The method for extracting the coated colloid according to claim 1, characterized in that, The process of extracting local features from the 3D point cloud and the point cloud of the workpiece under test, and performing preliminary alignment processing to obtain coarse registration results, includes: The Fast Point Feature Histogram (FPFH) is used to extract local features from the 3D point cloud of the template workpiece and the 3D point cloud of the workpiece under test, respectively, and feature matching is performed to obtain the initial transformation matrix. The 3D point cloud of the workpiece to be tested is initially aligned to the 3D point cloud of the template workpiece based on the initial transformation matrix, satisfying the following relationship: ； In the formula, For coarse registration results, Let be the initial transformation matrix. This is the three-dimensional point cloud of the workpiece to be tested.

3. The method for extracting the coated colloid according to claim 1, characterized in that, Local features of the 3D point clouds of the template workpiece and the workpiece under test are extracted using Fast Point Feature Histogram (FPFH) for feature matching, including: For points in the 3D point cloud of the template workpiece Using its FPFH features, the point with the closest feature Euclidean distance is found in the 3D point cloud of the workpiece under test. It satisfies the following relationship: ； In the formula, It is the 3D point cloud of the template workpiece. A 33-dimensional feature vector of each point, It is the 3D point cloud of the template workpiece. A 33-dimensional feature vector of each point; The obtained point pairs ( , This forms an initial feature matching point pair, and yields multiple sets of feature matching point pairs for the 3D point cloud of the template workpiece and the 3D point cloud of the workpiece to be tested. The initial transformation matrix between matched feature matching point pairs is calculated using the sample consensus algorithm, as follows: ； in, Let N be the initial transformation matrix, and N be the total number of corresponding point pairs. It is a point in the 3D point cloud of the workpiece to be tested. It is a point in the 3D point cloud of the template workpiece. It is the error function of coarse registration.

4. The method for extracting the coated colloid according to claim 1, characterized in that, The process of performing fine registration based on the coarse registration result to obtain the finely registered point cloud to be measured includes: Based on minimizing the distance from each point in the 3D point cloud of the workpiece under test to the tangent plane of the corresponding point in the 3D point cloud of the template workpiece, the error function of the fine transformation matrix is ​​calculated as follows: ； in, For fine transformation matrix, for The normal vector at point N is the total number of corresponding point pairs. It is a point in the point cloud to be tested. It is a point in the template point cloud; Based on the coarse registration results and the fine transformation matrix, the finely registered point cloud is calculated as follows: ； in, This is the result of coarse registration.

5. The method for extracting the coated colloid according to claim 1, characterized in that, The process of determining the template sub-point cloud and the sub-point cloud to be measured based on multiple predetermined measuring point areas, the three-dimensional point cloud of the template workpiece, and the finely registered point cloud to be measured includes: According to the testing requirements, multiple measurement point areas are predefined on the three-dimensional point cloud of the template workpiece. The point clouds corresponding to each measurement point area are extracted from the three-dimensional point cloud of the template workpiece and the finely registered point cloud to be tested, respectively, to obtain the template sub-point cloud and the sub-point cloud to be tested.

6. The method for extracting the coated colloid according to claim 1, characterized in that, For each measurement point region, the distance from each point in the sub-point cloud to the surface of the corresponding region of the template sub-point cloud is calculated, and the colloidal point cloud of that region is determined based on this distance and a distance threshold, including: Local fine registration is performed in each measurement point area to obtain the local fine registration sub-point cloud of the measurement point; For each point q in the sub-point cloud to be tested after local fine registration, find the nearest point of q in the template sub-point cloud, and calculate the Euclidean distance between the nearest point in the template sub-point cloud and point q. Set a distance threshold and retain all points whose Euclidean distance is greater than the distance threshold to obtain the colloidal point cloud of the measurement area.

7. The method for extracting the coated colloid according to claim 6, characterized in that, The local fine registration process in each measurement point area includes: For each measurement point region, the template sub-point cloud and the sub-point cloud to be measured are processed for local fine registration using registration algorithms such as point-to-surface, point-to-point, surface-to-surface, or Gaussian mixture model-based registration.

8. The method for extracting the coated colloid according to claim 6, characterized in that, The colloidal point cloud of the measurement point region is obtained by retaining all points whose Euclidean distance is greater than the distance threshold, satisfying the following relationship: ； In the formula, For points in the sub-point cloud to be tested after local fine registration, This is the sub-point cloud to be tested after local fine registration. Distance threshold for The nearest point in the template sub-point cloud.

9. An extraction device for coated colloids, characterized in that, include: The first module is used to acquire the 3D point cloud of the template workpiece and the 3D point cloud of the workpiece to be tested; The second module is used to extract local features from the 3D point cloud and the point cloud of the workpiece to be measured, respectively, and perform preliminary alignment processing to obtain coarse registration results; and perform fine registration based on the coarse registration results to obtain finely registered point cloud of the workpiece to be measured. The third module is used to determine the template sub-point cloud and the sub-point cloud to be measured based on multiple predetermined measurement point areas, the three-dimensional point cloud of the template workpiece, and the finely registered point cloud to be measured. The fourth module is used to calculate the distance from each point in the sub-point cloud to the surface of the corresponding area of ​​the template sub-point cloud for each measurement point area, delete the points in the sub-point cloud that are less than the set distance threshold, and finally retain the points as the target colloidal points.

Images