Skeleton point acquisition method of adhesive tape image
Through automated labeling and teaching the method of skeleton point position correction, the accuracy and robustness of the acquisition of the "corner" glue strip image skeleton point in glue coating detection are solved, and efficient and accurate positioning of the glue strip area is achieved.
Patent Information
- Application Number
- CN202411585775.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-11-08
- Publication Date
- 2025-02-14
AI Technical Summary
The prior art is difficult to accurately obtain the skeleton points of the "corner" glue strip image in glue coating detection, especially in the case of signal delay and unstable image quality, resulting in low positioning accuracy and poor robustness.
Through the method of automatically labeling skeleton points, the position correction is performed using the teaching skeleton points, which is suitable for actual measuring tape, and the automatic acquisition of skeleton points is realized and the dependence on the quality of the drawing is reduced.
Improves the accuracy and robustness of the "corner" rubber strip image, reduces manual participation, short time and high accuracy.
Smart Images
Figure CN119444790A_ABST
Abstract
Description
Technical Field
[0001] The invention relates to the field of glue coating detection, and in particular to a method for acquiring skeleton points of a glue strip image. Background Art
[0002] In the visual gluing inspection scenario, batches of workpieces to be coated are placed under the glue gun in sequence. When a single workpiece is in place, the robot drives the glue gun to glue the workpiece according to the set gluing trajectory. The camera collects the image of the coated glue strip. After the workpiece is glued, the robot returns to the starting position and waits for the next gluing. The collected glue strip image is used to detect the glue width of the glue strip, whether there is glue breakage, etc. When processing the glue strip image, it is necessary to first obtain the skeleton points on the glue strip to locate the glue strip area. In the prior art, there are the following two skeleton point acquisition methods:
[0003] Method 1: In the rubber strip image, directly use the existing skeleton point extraction method (such as the skeleton point refinement method) to extract the skeleton points on the rubber strip. This method has the following disadvantages: In the glue coating scene, the coated workpiece is a metal sheet metal part with reflective properties, such as the sheet metal part on the car body; in addition, in some scenes, the color of the coated rubber strip is very close to the color of the sheet metal part, for example, black glue is coated on a black sheet metal part. These phenomena will make it difficult to accurately capture the edge of the coated rubber strip. When it is presented on the image, the edge will be blurred, and the accurate edge of the rubber strip cannot be extracted. For this reason, this method relies on the quality of image acquisition. The skeleton points of the extracted rubber strip area have poor stability and are difficult to control in terms of accuracy. In addition, if there is a break in the measured rubber strip, the skeleton points cannot be extracted at the break, and the break area cannot be located.
[0004] Method 2: First use the teaching process: select a normal rubber strip image with no broken glue and uniform rubber strips. In this image, manually calibrate the edge of the rubber strip, mark multiple skeleton points in the center of the rubber strip area, and store them as teaching skeleton points. In the rubber strip images of other workpieces collected subsequently, directly call the teaching skeleton points as the skeleton points of the current rubber strip to locate the rubber strip area in the current image. Compared with the method in method 1, this method is no longer subject to the constraints of image acquisition quality. It is widely used for positioning straight rubber strips (the glue coating trajectory is a straight line), but for "corner" rubber strip images (including straight rubber strips connected to curved rubber strips), because the glue coating trajectory includes at least one turning trajectory and a straight trajectory connected to the turning trajectory, at this time, the application of method 2 has the following problems:
[0005] During the gluing process, the signal transmission jitters during robot communication, which causes signal delay. Therefore, when there is a turning track in the gluing track, there are differences between the images of the glue strips of different workpieces captured by the camera:
[0006] Example 1: "L" shaped rubber strip:
[0007] The glue gun turns from left to right and then applies glue from top to bottom. Figure 1a Indicates the first collected image of the adhesive strip, with the skeleton points (large dots in the picture) marked on the image; Figure 1b Indicates the second rubber strip image captured. The delay time t during this communication becomes shorter, the camera captures images faster, and more rubber strip areas are captured. Figure 1c Indicates the third rubber strip image captured. The delay time t during this communication becomes longer, the camera captures images more slowly, and fewer rubber strip areas are captured.
[0008] Example 2: "U" shaped rubber strip:
[0009] The glue gun applies glue from left to right, then from top to bottom after turning, and then from right to left after turning. Figure 2a Indicates the first collected image of the adhesive strip, with the skeleton points (large dots in the picture) marked on the image; Figure 2b Indicates the second rubber strip image captured. The delay time t during this communication becomes shorter, the camera captures images faster, and more rubber strip areas are captured. Figure 2c Indicates the third rubber strip image captured. The delay time t during this communication becomes longer, the camera captures images more slowly, and fewer rubber strip areas are captured.
[0010] By comparing the two examples, it can be clearly found that for the corner-type rubber strip image, there is a position deviation between the taught skeleton points and the measured rubber strip. If the taught skeleton points are used directly, the position of the rubber strip will also be inaccurate. Summary of the invention
[0011] In order to solve the above technical problems, the present invention provides a method for obtaining skeleton points of a rubber strip image. The method corrects the position of the teaching skeleton points so that it can be more suitable for the measured rubber strip and is closer to the position of the measured rubber strip. The method does not require manual participation and realizes the automatic marking of skeleton points, which has the characteristics of short time consumption and high accuracy. It is suitable for effectively locating the rubber strip area of the "corner" rubber strip image during the automobile glue coating inspection process.
[0012] The technical solution is as follows:
[0013] A method for acquiring skeleton points of a glue strip image. In a glue coating scene, batches of workpieces to be coated are placed under a glue gun in sequence. When a single workpiece is in place, the glue gun coats the workpiece with glue according to a set glue coating trajectory, and a camera captures the coated glue strip image.
[0014] The gluing track includes at least one turning track and a straight track connected to the turning track; the turning track is in an arc shape or a quasi-arc shape;
[0015] From the acquired rubber strip images, select a rubber strip image with no broken glue and uniform rubber strips, manually mark multiple skeleton points on the rubber strip in the image, mark these skeleton points as teaching skeleton points, use the teaching skeleton points to fit lines to obtain teaching lines, and store the teaching lines and the pixel coordinates of each teaching skeleton point in the image;
[0016] During detection, the skeleton points of the newly acquired rubber strip image are automatically annotated through the following steps:
[0017] 1) extracting multiple initial skeleton points on the rubber strip from the newly acquired rubber strip image;
[0018] 2) Calculate the distance between each initial skeleton point and the teaching line respectively. If the distance value is less than a threshold, directly mark each teaching skeleton point in the image; otherwise, move each initial skeleton point synchronously toward the teaching line so that the initial skeleton point approaches the teaching skeleton point, obtain the direction and distance of movement, and then move each teaching skeleton point in the opposite direction by the same distance. Mark each teaching skeleton point after movement in the image to complete the automatic acquisition of skeleton points.
[0019] Further, in step 2), if the distance value is not less than the threshold, the following processing is performed:
[0020] ① Fitting a line using the initial skeleton points, recorded as the initial line; calculating the intersection between the initial line and the teaching line, recorded as point A;
[0021] ② Move point A along the teaching line by △s pixels, and the point after the movement is recorded as point B; calculate and store the vector from point A to point B according to the coordinates of point B and point A;
[0022] Move each initial skeleton point according to the direction and modulus of the most recently stored vector;
[0023] ③ Calculate the distance between each initial skeleton point after the move and the teaching line. If the distance value is less than the threshold, directly proceed to step ④; otherwise, mark point B as the new point A, clear the original point A and point B, record each initial skeleton point after the move as the new initial skeleton point, clear the original initial skeleton point, and then use the new point A and the new initial skeleton point to jump to step ②;
[0024] ④ Retrieve the vectors stored in each step ②, and then move each taught skeleton point in the opposite direction of each vector in order from back to front;
[0025] Mark each taught skeleton point after the last movement in the current rubber strip image to complete the automatic acquisition of the skeleton points.
[0026] Furthermore, in step ②, the coordinates of point B are obtained using the following method:
[0027] Based on the curve equation of the teaching line obtained by fitting the teaching skeleton points, the tangent and normal at point A are obtained, the normal is moved by △s pixels along the direction of the tangent, and the coordinates of the intersection of the moved normal and the teaching line are marked as the coordinates of point B.
[0028] Preferably, in step ③, if the distance calculated after the movement is larger than the distance calculated before the movement, it means that point A is moving in the wrong direction. When step ② is performed again, point A is moved in the opposite direction to the previous direction.
[0029] Preferably, in step ②, the value of Δs is 5 pixels to 10 pixels.
[0030] Preferably, in step 2), the distance includes a distance mean and a maximum distance value;
[0031] The threshold corresponding to the mean distance is: half of the theoretical adhesive width; the threshold corresponding to the maximum distance is: the theoretical adhesive width;
[0032] Determine whether the mean distance and the maximum distance are both less than their corresponding thresholds.
[0033] This method directly retrieves the teaching skeleton points and corrects the positions to effectively locate the measured rubber strip area; it also realizes the automatic labeling of skeleton points. Compared with the existing methods, this method reduces the requirements for image quality and can still effectively locate rubber strip images with high noise, blurred edges, and poor foreground-background contrast, especially for "corner" rubber strip images, with higher positioning accuracy and stronger robustness. BRIEF DESCRIPTION OF THE DRAWINGS
[0034] Figures 1a to 1c The schematic diagrams of the positional relationship between the "L"-shaped rubber strip area and the teaching skeleton points in different images are respectively;
[0035] Figure 2a to 2c The schematic diagrams of the positional relationship between the "U"-shaped rubber strip area and the teaching skeleton points in different images are respectively;
[0036] Figure 3 It is a schematic diagram of the process of point A moving along the teaching line in a specific implementation manner. DETAILED DESCRIPTION
[0037] The technical solution of the present invention is described in detail below in conjunction with the accompanying drawings and embodiments.
[0038] A method for acquiring skeleton points of a glue strip image. In a glue coating scene, batches of workpieces to be coated are placed under a glue gun in sequence. When a single workpiece is in place, the glue gun coats the workpiece with glue according to a set glue coating trajectory, and a camera captures the coated glue strip image.
[0039] In this embodiment, the gluing track includes at least one turning track and a straight track connected to the turning track, and the turning track is arc-shaped or quasi-arc-shaped;
[0040] After the above glue application track, the glue strip image obtained is the "corner" glue strip image, which includes: straight glue strips connected to the curved glue strips respectively, and the angle between the straight glue strips is the angle of the glue gun turning (such as Figure 1a to Figure 1c "L" shaped rubber strip, Figure 2a to Figure 2c "U" shaped strip);
[0041] From the acquired rubber strip images, select a rubber strip image with no broken glue and uniform rubber strips, manually mark multiple skeleton points on the rubber strip in the image, mark these skeleton points as teaching skeleton points, use the teaching skeleton points to fit lines to obtain teaching lines, and store the teaching lines and the pixel coordinates of each teaching skeleton point in the image;
[0042] Preferably, the interval between adjacent teaching skeleton points is 5 to 30 pixels.
[0043] During detection, for a newly acquired rubber strip image, automatically annotate the skeleton points of the rubber strip through the following steps:
[0044] 1) extracting multiple initial skeleton points on the rubber strip from the newly acquired rubber strip image;
[0045] Specifically, the method for extracting initial skeleton points is an existing method, such as a skeleton point refinement method.
[0046] 2) Calculate the distance between each initial skeleton point and the teaching line respectively. If the distance value is less than the threshold, directly mark each teaching skeleton point in the image; otherwise, move each initial skeleton point synchronously to the teaching line so that the initial skeleton point approaches the teaching skeleton point (coincides with or is close to the teaching skeleton point), obtain the direction and distance of movement, and then move each teaching skeleton point in the opposite direction by the same distance, mark each teaching skeleton point after movement in the image, and complete the automatic acquisition of skeleton points.
[0047] Specifically, in step 2), if the distance value is not less than the threshold, the following processing is performed:
[0048] ① Use the initial skeleton points to fit the line, recorded as the initial line; Figure 3 As shown, the initial line L is calculated 测 With teaching line L 示 The intersection point between them is recorded as point A, and the teaching line is a line fitted by using the teaching skeleton points;
[0049] ② Move point A along the teaching line by △s pixels, and the point after the movement is recorded as point B; calculate and store the vector from point A to point B according to the coordinates of point B and point A;
[0050] In specific implementation, the value of △s is 5 pixels to 10 pixels;
[0051] Move each initial skeleton point according to the direction and modulus of the latest stored vector;
[0052] ③ Calculate the distance between each initial skeleton point after the move and the teaching line. If the distance value is less than the threshold, directly proceed to step ④; otherwise, mark point B as the new point A, clear the original point A and point B, record each initial skeleton point after the move as the new initial skeleton point, clear the original initial skeleton point, and then use the new point A and the new initial skeleton point to jump to step ②;
[0053] ④ Retrieve the vectors stored in each step ②, and then move each taught skeleton point in the opposite direction of each vector in order from back to front;
[0054] Mark each taught skeleton point after the last movement in the current rubber strip image to complete the automatic acquisition of the skeleton points.
[0055] like Figure 3 As shown, step ② is performed three times, and the stored vectors are (dx1, dy1), (dx2, dy2), and (dx3, dy3), respectively. In step ④, each taught skeleton point is moved in the opposite direction of (dx3, dy3), the opposite direction of (dx2, dy2), and the opposite direction of (dx1, dy1), and each taught skeleton point after the last movement is marked in the current corner strip image to complete the automatic acquisition of the skeleton points.
[0056] More specifically, in step ②, the coordinates of point B are obtained using the following method:
[0057] Based on the curve equation of the teaching line obtained by fitting the teaching skeleton points, the tangent and normal at point A are obtained, the normal is moved by △s pixels along the direction of the tangent, and the coordinates of the intersection of the moved normal and the teaching line are marked as the coordinates of point B.
[0058] In particular, in step ③, if the distance calculated after the movement is larger than the distance calculated before the movement, it means that point A is moving in the wrong direction. When step ② is performed again, point A is moved in the opposite direction to the previous direction.
[0059] In specific implementation, in order to increase the binding force, the distance value includes the distance mean and the maximum distance value;
[0060] Calculate the average and maximum distances between each initial skeleton point and the teaching line;
[0061] The threshold corresponding to the mean distance is: half of the theoretical adhesive width; the threshold corresponding to the maximum distance is: the theoretical adhesive width;
[0062] It is determined whether the mean distance and the maximum distance are both less than their corresponding thresholds. If both are less than, it means that the initial skeleton point coincides with or is close to the teaching skeleton point.
[0063] For convenience in explanation and precise definition of the appended claims, the terms “upper,” “lower,” “inner,” and “outer” are used to describe features of the exemplary embodiments with reference to the positions of such features as shown in the drawings.
[0064] The foregoing description of the specific exemplary embodiments of the present invention is presented for the purpose of illustration and description. The foregoing description is not intended to be exhaustive, nor is it intended to limit the present invention to the precise form disclosed, and it is apparent that many changes and variations are possible in light of the above teachings. The exemplary embodiments are selected and described to explain the specific principles of the present invention and its practical application, so that other technicians in the field can realize and utilize the various exemplary embodiments of the present invention and its different selected forms and modified forms. The scope of the present invention is intended to be limited by the appended claims and their equivalents.
Claims
1. A method for acquiring skeleton points of a glue strip image. In a glue coating scene, batches of workpieces to be coated are placed under a glue gun in sequence. When a single workpiece is in place, the glue gun coats the workpiece with glue according to a set glue coating trajectory, and a camera captures the coated glue strip image. It is characterized in that The gluing track includes at least one turning track and a straight track connected to the turning track; the turning track is in an arc shape or a quasi-arc shape; From the acquired rubber strip images, select a rubber strip image with no broken glue and uniform rubber strips, manually mark multiple skeleton points on the rubber strip in the image, mark these skeleton points as teaching skeleton points, use the teaching skeleton points to fit lines to obtain teaching lines, and store the teaching lines and the pixel coordinates of each teaching skeleton point in the image; During detection, the skeleton points of the newly acquired rubber strip image are automatically annotated through the following steps: 1) extracting multiple initial skeleton points on the rubber strip from the newly acquired rubber strip image; 2) Calculate the distance between each initial skeleton point and the teaching line respectively. If the distance value is less than a threshold, directly mark each teaching skeleton point in the image; otherwise, move each initial skeleton point synchronously toward the teaching line so that the initial skeleton point approaches the teaching skeleton point, obtain the direction and distance of movement, and then move each teaching skeleton point in the opposite direction by the same distance. Mark each teaching skeleton point after movement in the image to complete the automatic acquisition of skeleton points.
2. The method for acquiring skeleton points of a rubber strip image as claimed in claim 1, characterized in that: In step 2), if the distance value is not less than the threshold, the following processing is performed: ① Fitting a line using the initial skeleton points, recorded as the initial line; calculating the intersection between the initial line and the teaching line, recorded as point A; ② Move point A along the teaching line by △s pixels, and the point after the movement is recorded as point B; calculate and store the vector from point A to point B according to the coordinates of point B and point A; Move each initial skeleton point according to the direction and modulus of the most recently stored vector; ③ Calculate the distance between each initial skeleton point after the move and the teaching line. If the distance value is less than the threshold, directly proceed to step ④; otherwise, mark point B as the new point A, clear the original point A and point B, record each initial skeleton point after the move as the new initial skeleton point, clear the original initial skeleton point, and then use the new point A and the new initial skeleton point to jump to step ②; ④ Retrieve the vectors stored in step ② each time, and then move each taught skeleton point in the opposite direction of each vector in order from back to front; Mark each taught skeleton point after the last movement in the current rubber strip image to complete the automatic acquisition of the skeleton points.
3. The method for acquiring skeleton points of a rubber strip image as claimed in claim 2, characterized in that: In step ②, the coordinates of point B are obtained using the following method: Based on the curve equation of the teaching line obtained by fitting the teaching skeleton points, the tangent and normal at point A are obtained, the normal is moved by △s pixels along the direction of the tangent, and the coordinates of the intersection of the moved normal and the teaching line are marked as the coordinates of point B.
4. The method for acquiring skeleton points of a rubber strip image as claimed in claim 2, characterized in that: In step ③, if the distance calculated after the movement is larger than the distance calculated before the movement, it means that point A is moving in the wrong direction. When performing step ② again, point A is moved in the opposite direction to the previous direction.
5. The method for acquiring skeleton points of a rubber strip image as claimed in claim 2, characterized in that: In step ②, the value of △s is 5 pixels to 10 pixels.
6. The method for acquiring skeleton points of a rubber strip image according to claim 1, characterized in that: In step 2), the distance includes the mean distance and the maximum distance value; The threshold corresponding to the mean distance is: half of the theoretical adhesive width; the threshold corresponding to the maximum distance is: the theoretical adhesive width; Determine whether the mean distance and the maximum distance are both less than their corresponding thresholds.