A sub-pixel precision method for multi-type workpiece recognition and positioning

Abstract

The present invention discloses a method for identifying and locating multiple types of workpieces with sub-pixel precision, comprising the following steps: S1. collecting a preset number of workpiece images to construct a training data set and training a Yolov5 network; S2. using the trained Yolov5 network to identify and roughly locate multiple types of workpieces; S3. extracting the pixel-precision outermost contours of the multiple types of workpieces; S4. calculating and fitting the sub-pixel precision outermost contours of the multiple types of workpieces based on the pixel-precision outermost contours in S3; S5. extracting the minimum circumscribed rectangle of the multiple types of workpieces based on the sub-pixel precision outermost contours in S4; S6. calculating the positions and angles of the multiple types of workpieces based on the minimum circumscribed rectangle in S5 to achieve precise positioning of the multiple types of workpieces. The present invention provides a practical solution for users to perform personalized laser marking of multiple types of workpieces with a variety of different patterns to meet certain high-precision marking requirements for laser marking applications. Theoretically, as long as the Yolov5 network is trained sufficiently, the success rate of identification and positioning of this method can reach 100%.

Description

Technical Field

[0001] The present invention belongs to the field of industrial vision, and in particular relates to a method for identifying and positioning multiple types of workpieces with sub-pixel accuracy. Background Art

[0002] Laser marking uses a high-energy-density laser to locally illuminate a workpiece, vaporizing or changing the color of the surface material, leaving a permanent mark. Laser marking technology is currently widely used in various industries, such as intelligent manufacturing and production processing, where contactless, permanent marking is required.

[0003] Automatic laser marking relies heavily on the identification and precise positioning of workpieces. Currently, most workpiece identification and positioning methods for laser marking focus solely on locating workpieces of the same type, rarely addressing multiple types. This directly restricts the laser marking industry's transition toward personalized, flexible marking. Furthermore, most of these identification and positioning methods only achieve pixel-level accuracy, which is unsatisfactory for certain high-precision marking applications.

[0004] To this end, the present invention proposes and implements a sub-pixel precision multi-type workpiece recognition and positioning method that can simultaneously identify multiple types of workpieces and position each workpiece with sub-pixel precision. Summary of the Invention

[0005] Purpose of the invention: In order to solve the above problems, accelerate the transformation of the laser marking industry towards personalized and flexible marking, and meet the processing requirements of certain high-precision marking, the present invention proposes a multi-type workpiece identification and positioning method with sub-pixel accuracy.

[0006] Technical solution: The present invention solves the technical problem and proposes a sub-pixel precision multi-type workpiece identification and positioning method, which includes the following steps:

[0007] (1) Collecting workpiece images, marking the types of all workpieces in the image, and marking the rectangular areas where all workpieces are located to form a training data set. The training data set contains workpiece images and marking files. The marking files record the rectangular areas where all workpieces are located and the types of workpieces in the workpiece images.

[0008] (2) Train the Yolov5 network based on the training data set. The input is the workpiece image in the training set. The output is the type recognition results of all workpieces in the image and the rectangular areas where all workpieces are located. The trained Yolov5 network is able to recognize the types of workpieces and the rectangular areas where the workpieces are located.

[0009] (3) Input a new image containing multiple types of artifacts, use the Yolov5 network to identify and roughly locate all artifacts in the new image, and roughly locate the rectangular areas corresponding to all artifacts;

[0010] (4) extracting the pixel-accurate outermost contours of all workpieces from the new image containing multiple types of workpieces in (3), and recording it as the first outer contour;

[0011] (5) Calculate and fit the outermost contour with sub-pixel accuracy for the first outer contour in (4), which is recorded as the second outer contour;

[0012] (6) extracting the minimum circumscribed rectangle corresponding to each workpiece based on the second outer contour in (5);

[0013] (7) Based on the minimum bounding rectangle in (6), the sub-pixel precision position and angle of all workpieces are calculated to complete the precise positioning of multiple types of workpieces, and the position and angle of all workpieces are precisely positioned;

[0014] (8) Match the fine positioning result in (7) with the coarse positioning result in (3) to obtain the final identification and positioning result of each workpiece.

[0015] Furthermore, the specific implementation method of step (1) is as follows: in an actual laser marking environment, a preset number of workpiece images are collected, and the workpieces in some or all of the images collected in the actual laser marking environment are rotated and translated to generate more workpiece images, and then all the workpieces in the image are marked by type, and the rectangular areas where all the workpieces are located are marked to form a training data set. The training data set includes workpiece images and marking files, and the marking files record the rectangular areas and workpiece types where all the workpieces in the workpiece images are located.

[0016] Furthermore, the specific implementation method of step (3) is as follows: input a new workpiece image collected in real time, use the Yolov5 network trained in (1) to perform type recognition and coarse positioning of all workpieces in the image, where the type recognition result includes the type ID and score value Score of each workpiece, and the coarse positioning result includes the initial rectangular area of ​​each workpiece and the center of the initial rectangular area.

[0017] Furthermore, the specific implementation method of step (4) is as follows: first, the new image collected in (3) is subjected to Gaussian filtering, and then gamma contrast enhancement is performed, Canny edge detection is performed, and the pixel-accurate outermost contour of all workpieces in the new image is extracted, which is recorded as the first outer contour.

[0018] Furthermore, in step (5), the second outer contour is calculated and fitted based on the first outer contour in (4), and the specific method is as follows:

[0019] The first outer contour point set of each workpiece obtained from step (4) is S1 = {P'1, P'2, ..., P' m}, where m is the number of points of the first outer contour of each workpiece, and all contour points in S1 are calculated and fitted to sub-pixel positions;

[0020] Let pixel-accurate contour point P' i The coordinates are (x, y), where i = 1 to m, and the gradients of the image in the X and Y directions corresponding to the pixel-precision contour point (x, y) are G x (x,y),G y (x,y), and then calculate the gradient angle GAngle(x,y) using the following formula:

[0021]

[0022] A scanning window of size N×N is constructed with the pixel-precision contour point coordinates (x, y) as the center. N is a constant and an odd number, which is the size of the scanning window. Let the set of all pixel-precision contour points in the window be S={P1,P2,...,P n}, perform Gaussian weighted summing and averaging of the gradients of all contour points in S in the X and Y directions of the image, and obtain the Gaussian weighted average gradient values ​​Gmean of the pixel-precision contour point coordinates (x, y) in the X and Y directions of the scanning window respectively x (x,y),Gmean y (x,y):

[0023]

[0024] Where n is the total number of pixel-accurate contour points in the N×N scanning window, G x (i,j) and G y (i, j) are the gradients of the image in the X and Y directions corresponding to the pixel-accurate contour point (i, j) in the scanning window, respectively. x (i,j)=w y (i, j) is the Gaussian weight of the pixel-accurate contour point (i, j) within the scan window relative to the window center (x, y), and is calculated as follows:

[0025]

[0026] For the pixel-accurate contour point (x, y), the correction values ​​in the X and Y directions of the image are Δx and Δy respectively, where:

[0027]

[0028] Calculate the sub-pixel precision contour point (x', y'):

[0029]

[0030] The sub-pixel precision contour point coordinates (x', y') obtained by correcting the pixel precision contour point coordinates (x, y) constitute the second outer contour point set S2 = {P"1, P"2, ..., P" m}, where P" i The coordinates are (x', y'), i = 1 ~ m.

[0031] Furthermore, the specific implementation method of step (7) is as follows: Assume that the coordinates of the four endpoints of the minimum bounding rectangle are (x1, y1), (x2, y2), (x3, y3), (x4, y4), and calculate the sub-pixel precision position P of each workpiece. c ,P c The coordinates (x c ,y c ) is obtained by the following formula:

[0032]

[0033] The clockwise angle r between the longest side of the minimum circumscribed rectangle and the X direction of the image is calculated as the sub-pixel precision angle of the workpiece, thus completing the precise positioning of multiple types of workpieces.

[0034] Furthermore, the specific implementation method of step (8) is as follows: traverse the center P1 of each workpiece in the fine positioning result (7), traverse the center P2 of each workpiece in the coarse positioning result (3), calculate the Euclidean distance d between P1 and P2, if d is less than the set threshold T, it is considered that the results of fine positioning and coarse positioning are successfully matched, and the position and angle of the workpiece in the fine positioning result are compared with the workpiece type ID of the workpiece in the coarse positioning result and the score value of the workpiece when it belongs to this workpiece type, that is, the workpiece type ID, score value, position, and angle corresponding to each workpiece are stored in the result matrix to complete the recognition and positioning of multiple types of workpieces with sub-pixel accuracy.

[0035] Beneficial effects: Compared with the prior art, the technical solution of the present invention has the following beneficial effects:

[0036] The method of the present invention can simultaneously identify and locate multiple types of workpieces, which can promote the laser marking industry to develop in the direction of personalized and flexible marking of different patterns on multiple types of workpieces; the sub-pixel precision positioning effect can meet the needs of certain high-precision marking; in the present invention, workpiece identification and workpiece positioning are independent of each other and can be executed synchronously, which greatly improves the recognition and positioning speed of multiple types of workpieces. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] The present invention will be further described below with reference to the accompanying drawings and examples.

[0038] Figure 1 It is a flow chart of the system of the present invention;

[0039] Figure 2 This is a rendering of Example 1 of the present invention.

[0040] The white outline is the outermost outline of the workpiece, ID indicates the type of each workpiece, Score indicates the probability of each workpiece belonging to the type ID, and (x, y, r) respectively represent the center (x, y) value and rotation angle r of each workpiece. DETAILED DESCRIPTION

[0041] The present invention will be described in further detail below with reference to the accompanying drawings and specific embodiments. The specific embodiments described are only intended to illustrate the present invention and are not intended to limit the present invention.

[0042] The present invention proposes a method for identifying and locating multiple types of workpieces with sub-pixel accuracy, which comprises the following steps:

[0043] (1) Collecting workpiece images, marking the types of all workpieces in the image, and marking the rectangular areas where all workpieces are located to form a training data set. The training data set contains workpiece images and marking files. The marking files record the rectangular areas where all workpieces are located and the types of workpieces in the workpiece images.

[0044] (2) Train the Yolov5 network based on the training data set. The input is the workpiece image in the training set. The output is the type recognition results of all workpieces in the image and the rectangular areas where all workpieces are located. The trained Yolov5 network is able to recognize the types of workpieces and the rectangular areas where the workpieces are located.

[0045] (3) Input a new image containing multiple types of artifacts, use the Yolov5 network to identify and roughly locate all artifacts in the new image, and roughly locate the rectangular areas corresponding to all artifacts;

[0046] (4) extracting the pixel-accurate outermost contours of all workpieces from the new image containing multiple types of workpieces in (3), and recording it as the first outer contour;

[0047] (5) Calculate and fit the outermost contour with sub-pixel accuracy for the first outer contour in (4), which is recorded as the second outer contour;

[0048] (6) extracting the minimum circumscribed rectangle corresponding to each workpiece based on the second outer contour in (5);

[0049] (7) Based on the minimum bounding rectangle in (6), the sub-pixel precision position and angle of all workpieces are calculated to complete the precise positioning of multiple types of workpieces, and the position and angle of all workpieces are precisely positioned;

[0050] (8) Match the fine positioning result in (7) with the coarse positioning result in (3) to obtain the final identification and positioning result of each workpiece.

[0051] Furthermore, the specific implementation method of step (1) is as follows: in an actual laser marking environment, a preset number of workpiece images are collected, and the workpieces in some or all of the images collected in the actual laser marking environment are rotated and translated to generate more workpiece images, and then all the workpieces in the image are marked by type, and the rectangular areas where all the workpieces are located are marked to form a training data set. The training data set includes workpiece images and marking files, and the marking files record the rectangular areas and workpiece types where all the workpieces in the workpiece images are located.

[0052] Furthermore, the specific implementation method of step (3) is as follows: input a new workpiece image collected in real time, use the Yolov5 network trained in (1) to perform type recognition and coarse positioning of all workpieces in the image, where the type recognition result includes the type ID and score value Score of each workpiece, and the coarse positioning result includes the initial rectangular area of ​​each workpiece and the center of the initial rectangular area.

[0053] Furthermore, the specific implementation method of step (4) is as follows: first, the new image collected in (3) is subjected to Gaussian filtering, and then gamma contrast enhancement is performed, Canny edge detection is performed, and the pixel-accurate outermost contour of all workpieces in the new image is extracted, which is recorded as the first outer contour.

[0054] Furthermore, in step (5), the second outer contour is calculated and fitted based on the first outer contour in (4), and the specific method is as follows:

[0055] The first outer contour point set of each workpiece obtained from step (4) is S1 = {P'1, P'2, ..., P' m}, where m is the number of points of the first outer contour of each workpiece, and all contour points in S1 are calculated and fitted to sub-pixel positions;

[0056] Let pixel-accurate contour point P' i The coordinates are (x, y), where i = 1 to m, and the gradients of the image in the X and Y directions corresponding to the pixel-precision contour point (x, y) are G x (x,y),G y (x,y), and then calculate the gradient angle GAngle(x,y) using the following formula:

[0057]

[0058] A scanning window of size N×N is constructed with the pixel-precision contour point coordinates (x, y) as the center. N is a constant and an odd number, which is the size of the scanning window. Let the set of all pixel-precision contour points in the window be S={P1,P2,...,P n}, perform Gaussian weighted summing and averaging of the gradients of all contour points in S in the X and Y directions of the image, and obtain the Gaussian weighted average gradient values ​​Gmean of the pixel-precision contour point coordinates (x, y) in the X and Y directions of the scanning window respectively x (x,y),Gmean y (x,y):

[0059]

[0060] Where n is the total number of pixel-accurate contour points in the N×N scanning window, G x (i,j) and G y (i, j) are the gradients of the image in the X and Y directions corresponding to the pixel-accurate contour point (i, j) in the scanning window, respectively. x (i,j)=w y (i, j) is the Gaussian weight of the pixel-accurate contour point (i, j) within the scan window relative to the window center (x, y), and is calculated as follows:

[0061]

[0062] For the pixel-accurate contour point (x, y), the correction values ​​in the X and Y directions of the image are Δx and Δy respectively, where:

[0063]

[0064] Calculate the sub-pixel precision contour point (x', y'):

[0065]

[0066] The sub-pixel precision contour point coordinates (x', y') obtained by correcting the pixel precision contour point coordinates (x, y) constitute the second outer contour point set S2 = {P"1, P"2, ..., P" m}, where P" i The coordinates are (x', y'), i = 1 ~ m.

[0067] Furthermore, the specific implementation method of step (7) is as follows: Assume that the coordinates of the four endpoints of the minimum bounding rectangle are (x1, y1), (x2, y2), (x3, y3), (x4, y4), and calculate the sub-pixel precision position P of each workpiece. c ,P c The coordinates (x c ,yc ) is obtained by the following formula:

[0068]

[0069] The clockwise angle r between the longest side of the minimum circumscribed rectangle and the X direction of the image is calculated as the sub-pixel precision angle of the workpiece, thus completing the precise positioning of multiple types of workpieces.

[0070] Furthermore, the specific implementation method of step (8) is as follows: traverse the center P1 of each workpiece in the fine positioning result (7), traverse the center P2 of each workpiece in the coarse positioning result (3), calculate the Euclidean distance d between P1 and P2, if d is less than the set threshold T, it is considered that the results of fine positioning and coarse positioning are successfully matched, and the position and angle of the workpiece in the fine positioning result are compared with the workpiece type ID of the workpiece in the coarse positioning result and the score value of the workpiece when it belongs to this workpiece type, that is, the workpiece type ID, score value, position, and angle corresponding to each workpiece are stored in the result matrix to complete the recognition and positioning of multiple types of workpieces with sub-pixel accuracy.

[0071] Example 1

[0072] like Figure 2 As shown in the figure, there are four types of workpieces in the predicted image. In the coarse positioning link, the method of the present invention identifies all four types of workpieces through the Yolov5 network, and displays the type ID corresponding to each workpiece and the probability value of the workpiece to which this type ID belongs, that is, the score value, above the workpiece. In the fine positioning link, the method of the present invention first extracts the pixel-precision outermost contour of each workpiece, and then calculates and fits the higher-precision sub-pixel precision outermost contour. Finally, the workpiece is accurately positioned according to the sub-pixel precision outermost contour, and the final workpiece positioning result, that is, the center coordinate (x, y) value and rotation angle r of each workpiece are displayed above the workpiece. The coarse positioning and fine positioning are combined to complete the accurate identification and positioning of the four types of workpieces in the predicted image.

[0073] Figure 2 This demonstrates that the method can accurately identify and locate multiple types of workpieces in the predicted image, demonstrating its feasibility and effectiveness. In actual laser marking applications, the method can identify and locate multiple types of workpieces, enabling the precise laser marking of specific types of workpieces at specific locations.

Claims

1. A multi-type workpiece recognition and positioning method with sub-pixel accuracy, characterized in that: The method comprises the following steps: (1) Collecting workpiece images, marking the types of all workpieces in the image, and marking the rectangular areas where all workpieces are located to form a training data set. The training data set contains workpiece images and marking files. The marking files record the rectangular areas where all workpieces are located and the types of workpieces in the workpiece images. (2) Train the Yolov5 network based on the training data set. The input is the workpiece image in the training set. The output is the type recognition results of all workpieces in the image and the rectangular areas where all workpieces are located. The trained Yolov5 network is able to recognize the types of workpieces and the rectangular areas where the workpieces are located. (3) Input a new image containing multiple types of artifacts, use the Yolov5 network to identify and roughly locate all artifacts in the new image, and roughly locate the rectangular areas corresponding to all artifacts; (4) extracting the pixel-accurate outermost contours of all workpieces from the new image containing multiple types of workpieces in (3), and recording it as the first outer contour; (5) Calculate and fit the outermost contour with sub-pixel accuracy for the first outer contour in (4), which is recorded as the second outer contour; (6) extracting the minimum circumscribed rectangle corresponding to each workpiece based on the second outer contour in (5); (7) Based on the minimum bounding rectangle in (6), the sub-pixel precision position and angle of all workpieces are calculated to complete the precise positioning of multiple types of workpieces, and the position and angle of all workpieces are precisely positioned; (8) Matching the fine positioning result in (7) with the coarse positioning result in (3) to obtain the final identification and positioning result of each workpiece; In step (5), the second outer contour is calculated and fitted based on the first outer contour in (4). The specific method is as follows: The first outer contour point set of each workpiece obtained from step (4) is S1 = {P1′, P′2, ..., P′ m }, where m is the number of points of the first outer contour of each workpiece, and all contour points in S1 are calculated and fitted to sub-pixel positions; Let pixel-accurate contour point P i The coordinates of ′ are (x, y), where i = 1 to m, and the gradients of the image in the X and Y directions corresponding to the pixel-precision contour point (x, y) are G x (x,y),G y (x,y), and then calculate the gradient angle GAngle(x,y) using the following formula: A scanning window of size N×N is constructed with the pixel-precision contour point coordinates (x, y) as the center. N is a constant and an odd number, which is the size of the scanning window. Let the set of all pixel-precision contour points in the window be S={P1,P2,...,P n }, perform Gaussian weighted summing and averaging of the gradients of all contour points in S in the X and Y directions of the image, and obtain the Gaussian weighted average gradient values ​​Gmean of the pixel-precision contour point coordinates (x, y) in the X and Y directions of the scanning window respectively x (x,y),Gmean y (x,y): Where n is the total number of pixel-accurate contour points in the N×N scanning window, G x (i,j) and G y (i, j) are the gradients of the image in the X and Y directions corresponding to the pixel-accurate contour point (i, j) in the scanning window, respectively. x (i,j)=w y (i, j) is the Gaussian weight of the pixel-accurate contour point (i, j) within the scan window relative to the window center (x, y), and is calculated as follows: For the pixel-accurate contour point (x, y), the correction values ​​in the X and Y directions of the image are Δx and Δy respectively, where: Calculate the sub-pixel precision contour point (x′, y′): The sub-pixel precision contour point coordinates (x′, y′) obtained by correcting the pixel precision contour point coordinates (x, y) constitute the second outer contour point set S2 = {P1”, P2”, ..., P m "}, where P i The coordinates of " are (x′, y′), i = 1 ~ m.

2. The method for identifying and locating multiple types of workpieces with sub-pixel accuracy according to claim 1, characterized in that: The specific implementation method of step (1) is as follows: in an actual laser marking environment, a preset number of workpiece images are collected, and the workpieces in some or all of the images collected in the actual laser marking environment are rotated and translated to generate more workpiece images, and all workpieces in the image are marked by type, and the rectangular areas where all workpieces are located are marked to form a training data set. The training data set includes workpiece images and marking files, and the marking files record the rectangular areas and workpiece types of all workpieces in the workpiece images.

3. The method for identifying and locating multiple types of workpieces with sub-pixel accuracy according to claim 1, characterized in that: The specific implementation method of step (3) is as follows: input a new workpiece image collected in real time, use the Yolov5 network trained in (1) to perform type recognition and coarse positioning of all workpieces in the image, where the type recognition result includes the type ID and score value Score of each workpiece, and the coarse positioning result includes the initial rectangular area of ​​each workpiece and the center of the initial rectangular area.

4. The method for identifying and locating multiple types of workpieces with sub-pixel accuracy according to claim 1, characterized in that: The specific implementation method of step (4) is as follows: first, the new image collected in (3) is subjected to Gaussian filtering, and then gamma contrast enhancement is performed, Canny edge detection is performed, and the pixel-accurate outermost contour of all workpieces in the new image is extracted, which is recorded as the first outer contour.

5. The method for identifying and locating multiple types of workpieces with sub-pixel accuracy according to claim 1, characterized in that: The specific implementation method of step (7) is as follows: Assume that the coordinates of the four endpoints of the minimum bounding rectangle are (x1, y1), (x2, y2), (x3, y3), (x4, y4), and calculate the sub-pixel precision position P of each workpiece. c ,P c The coordinates (x c ,y c ) is obtained by the following formula: The clockwise angle r between the longest side of the minimum circumscribed rectangle and the X direction of the image is calculated as the sub-pixel precision angle of the workpiece, thus completing the precise positioning of multiple types of workpieces.

6. The method for identifying and locating multiple types of workpieces with sub-pixel accuracy according to claim 1, characterized in that: The specific implementation method of step (8) is as follows: traverse the center P1 of each workpiece in the fine positioning result (7), traverse the center P2 of each workpiece in the coarse positioning result (3), calculate the Euclidean distance d between P1 and P2, and if d is less than the set threshold T, it is considered that the results of fine positioning and coarse positioning are successfully matched, and the position and angle of the workpiece in the fine positioning result are compared with the workpiece type ID of the workpiece in the coarse positioning result and the score value of the workpiece when it belongs to this workpiece type, that is, the workpiece type ID, score value, position, and angle corresponding to each workpiece are stored in the result matrix to complete the recognition and positioning of multiple types of workpieces with sub-pixel accuracy.

Images