Barrel opening positioning method based on machine vision

The precise positioning of the barrel port is achieved through machine vision technology, which solves the problems of poor accuracy and low safety in traditional manual operation, improves filling efficiency and safety, and can handle various types of liquids.

CN119027493AInactive Publication Date: 2024-11-26JIANGSU JINWANG PACKING SCI TECH CO LTD
View PDF 3 Cites 0 Cited by

Patent Information

Application Number
CN202411146637.1
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-08-21
Publication Date
2024-11-26
Estimated Expiration
Not applicable · inactive patent

AI Technical Summary

Technical Problem

Traditional barrel port positioning methods rely on manual operation, have poor accuracy and low filling efficiency, especially when dealing with toxic, flammable, explosive or corrosive liquids, workers' safety is threatened.

Method used

The machine vision-based positioning method is adopted to obtain the precise position of the barrel through camera calibration, template matching, edge detection and coordinate conversion, so as to avoid workers from directly contacting materials and improve safety and filling efficiency.

Benefits of technology

Reliable positioning of the barrel opening is achieved, filling efficiency and safety is improved, and the capping of the barrel opening directly above and obliquely is achieved, which improves the capping pass rate.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119027493A_ABST
    Figure CN119027493A_ABST
Patent Text Reader

Abstract

The invention relates to the technical field of automatic visual positioning, in particular to a barrel opening positioning method based on machine vision, which comprises the following steps of: executing an online step before leaving a factory to obtain an internal and external parameter matrix of a camera, a template matching file, a relative position and a photographing position; distortion correction is carried out on the image acquired by the camera; determining an ROI of edge detection; fitting a complete circle corresponding to the barrel opening under the pixel coordinate system; converting and outputting an image and coordinates; the coordinates of the barrel opening are transmitted to the PLC; and repeating the steps until all barrels are positioned. After the online steps are completed, the coordinates of the barrel opening can be obtained through one key under the condition that no manual operation or parameter adjustment is conducted, the filling head is guided to be inserted into the barrel opening, and high-precision full-automatic filling is achieved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The invention relates to the technical field of automated visual positioning, and in particular to a barrel opening positioning method based on machine vision. Background Art

[0002] Drum mouth positioning is an important task in the process of automated production and filling. The traditional method of drum mouth positioning relies on manual operation, which has poor accuracy and low filling efficiency; especially when filling toxic, flammable, explosive or corrosive liquids, workers are easily injured.

[0003] Combined with the rapidly developing machine vision technology, the present invention realizes the reliable positioning of the barrel mouth, can obtain the precise position of the barrel mouth with one click, avoids direct contact between workers and materials, and improves safety and filling efficiency. Summary of the invention

[0004] In view of the above technical problems to be solved, a barrel mouth positioning method based on machine vision is provided.

[0005] To achieve the above object, a barrel opening positioning method based on machine vision of the present invention comprises the following steps: Step S1: Perform online steps before leaving the factory to obtain the camera's internal and external parameter matrix, template matching file, relative position, and photo position, including camera calibration, frame selection template, template matching NCC file generation, relative position calibration, and photo position determination; Step S2: According to the camera intrinsic parameter matrix obtained in step S1, the image captured by the camera is subjected to distortion correction; Step S3: determining the ROI (region of interest) for edge detection, including template matching, generating a rectangular ROI, binarization, taking the binarized edge, dilation processing, and generating a ring ROI; Step S4: fitting a complete circle corresponding to the barrel mouth in the pixel coordinate system, including edge detection, edge length measurement, noise screening, arc fitting, arc length measurement, and completing the complete circle according to the longest arc; Step S5: according to the camera internal and external parameter matrix obtained in step S1, convert and output the image and coordinates, including converting the complete circle fitted in the pixel coordinate system to the world coordinate system, and outputting the image and barrel mouth coordinates; Step S6: The barrel mouth coordinates obtained in step S5 are transmitted to the PLC, driving the filling head to move to the barrel mouth center; Step S7: Repeat steps S2 to S6 until all buckets are positioned.

[0006] In a preferred embodiment of the present invention, the online steps before delivery in step S1 include: Step S1-1: The area array camera is automatically connected and collects grayscale images of the complete barrel surface and images of calibration plates in different postures; Step S1-2: Perform camera calibration according to the calibration plate image acquired in step S1-1 to obtain the camera intrinsic parameter matrix and extrinsic parameter matrix; Step S1-3: manually select a template image based on the complete barrel surface grayscale image acquired in step S1-1, and save it as a file; Step S1-4: Generate a template matching NCC (normalized cross correlation) file according to the template image file saved in step S1-3; Step S1-5: calibrate the relative position of the camera center and the filling head in the X and Y directions; Step S1-6: For each barrel, determine the photographing position to ensure that the camera can capture the entire barrel surface at each photographing position.

[0007] In a preferred embodiment of the present invention, the step of determining the ROI for edge detection in step S3 includes: Step S3-1: For each barrel opening (assuming that each barrel has n barrel openings), perform NCC template matching within the range of the distortion-corrected complete barrel surface grayscale image obtained in step S2 according to the template matching NCC file obtained in step S1-4, and obtain n coordinates with the greatest matching degree. The expression is: In the formula, is the coordinate of the upper left corner of the sub-image in the image, is the coordinate of the pixel in the template, For template Pixel width and height, For subgraph The average pixel value of For template The average pixel value of Step S3-2: for each coordinate with the greatest matching degree obtained in step S3-1, generate a rectangular ROI with the coordinate as the upper left corner and the width and height equal to the width and height of the template image; Step S3-3: For each rectangular ROI area obtained in step S3-2, perform Otsu binarization processing and take the outer edge of the binarized image, which is expressed as: In the formula, is the Otsu operator at the point The value at is the Laplace operator, is a Gaussian function, is the input image, is the convolution operation; Step S3-4: For the outer edge of the binary image obtained in step S3-3, dilation processing is performed, and the dilated image is taken as a ring ROI, which is expressed as: In the formula, is the binary internal edge, B is the structural element, is a point in the dilated image, is the two-dimensional real number plane, For collection The point in For collection The point in .

[0008] In a preferred embodiment of the present invention, the step S4 of fitting a complete circle corresponding to the barrel opening in the pixel coordinate system comprises: Step S4-1: using Canny edge detection to extract edge contours within the annular ROI obtained in step S3-4, and obtaining a binary edge image; Step S4-2: for each continuous edge detected in step S4-1, measure its length, those with length less than the threshold are regarded as noise and filtered out, and take all continuous edges with length greater than the threshold; Step S4-3: For each continuous edge obtained in step S4-2 whose length is greater than the threshold, fit it into an arc, and the expression is: In the formula, are the coordinates of the fitted circle, is the diameter of the fitted circle, For the Coordinates of contour points ; Step S4-4: for each arc fitted in step S4-3, measure its length and select the longest arc; Step S4-5: Complete the circle according to the longest arc selected in step S4-4.

[0009] In a preferred embodiment of the present invention, the step S5 of converting and outputting the image and coordinates includes: Step S5-1: According to the camera intrinsic parameter matrix and extrinsic parameter matrix obtained in step S2, the complete circle obtained in step S4-5 is converted into a world coordinate system whose unit is millimeter and whose coordinate axis origin and positive direction are consistent with the filling head, and the two-dimensional coordinates and diameter of the complete circle in the world coordinate system are obtained; Step S5-2: Based on the n complete circles obtained in step S4-5, the n complete circle images are displayed on the complete barrel surface grayscale image acquired in step S1-1, and the complete circle diameters detected from the n barrel openings in the world coordinate system are compared to obtain the complete circle coordinates in the world coordinate system that are closer to the barrel opening design diameter.

[0010] Beneficial effects: The barrel mouth positioning method based on machine vision of the present invention realizes 360-degree compensation of the capping head, increases the positioning of the barrel mouth during the capping process, and improves the qualified rate of capping. Due to the large compensation amount, the barrel mouth directly above can be capped, and the oblique barrel mouth can also be capped. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] The present invention will be further described in detail below in conjunction with the accompanying drawings and specific embodiments.

[0012] Figure 1 It is a flowchart of an embodiment of the present invention.

[0013] Figure 2 This is a diagram showing the effect of a circular ROI in step S3 of an embodiment of the present invention.

[0014] Figure 3 This is a diagram of the final detection effect of an embodiment of the present invention. DETAILED DESCRIPTION

[0015] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0016] The present invention provides a barrel opening positioning method based on machine vision, and its operation process is as follows: Figure 1 As shown, the specific steps include: Step S1: Perform online steps before leaving the factory to obtain the camera's internal and external parameter matrix, template matching file, relative position, and photo position, including camera calibration, frame selection template, template matching NCC file generation, relative position calibration, and photo position determination; Step S1-1: the area array camera is automatically connected, the camera is manually moved until the complete barrel surface can be displayed, the grayscale image of the complete barrel surface is collected, the calibration plate is placed under the camera in different postures, and the images of the calibration plate in different postures are collected; Step S1-2: Perform camera calibration according to the calibration plate image acquired in step S1-1 to obtain the camera intrinsic parameter matrix and extrinsic parameter matrix; Step S1-3: manually select a template image based on the complete barrel surface grayscale image acquired in step S1-1, and save it as a file; Step S1-4: Generate a template matching NCC (normalized cross correlation) file according to the template image file saved in step S1-3; Step S1-5: Mark the barrel surface, align the filling head and the camera with the center of the mark, and calibrate the relative position of the camera center and the filling head in the X and Y directions; Step S1-6: For each barrel, manually move the camera to ensure that the camera can capture the entire barrel surface at each shooting position, and record the coordinates of the shooting position.

[0017] Step S2: According to the camera intrinsic parameter matrix obtained in step S1, the image captured by the camera is subjected to distortion correction; Step S3: Figure 2 As shown, the ROI (region of interest) for edge detection is determined, including template matching, generating a rectangular ROI, binarization, taking a binarized edge, dilation processing, and generating a ring ROI; Step S3-1: For each barrel opening (assuming each barrel has two barrel openings), perform NCC template matching within the range of the distortion-corrected complete barrel surface grayscale image obtained in step S2 according to the template matching NCC file obtained in step S1-4, and obtain the two coordinates with the greatest matching degree. The expression is: In the formula, is the coordinate of the upper left corner of the sub-image in the image, is the coordinate of the pixel in the template, For template Pixel width and height, For subgraph The average pixel value of For template The average pixel value of Step S3-2: for the two coordinates with the greatest matching degree obtained in step S3-1, generate a rectangular ROI with the coordinates as the upper left corner and the width and height equal to the width and height of the template image; Step S3-3: For the two rectangular ROI regions obtained in step S3-2, perform Otsu binarization processing and take the outer edge of the binarized image, which is expressed as: In the formula, is the Otsu operator at the point The value at is the Laplace operator, is a Gaussian function, is the input image, is the convolution operation; Step S3-4: For the outer edge of the binary image obtained in step S3-3, dilation processing is performed, and Figure 2 As shown, the expanded image is taken as a ring ROI, and its expression is:

[0018] In the formula, is the binary internal edge, B is the structural element, is a point in the dilated image, is the two-dimensional real number plane, For collection The point in For collection The point in .

[0019] Step S4: fitting a complete circle corresponding to the barrel mouth in the pixel coordinate system, including edge detection, edge length measurement, noise screening, arc fitting, arc length measurement, and completing the complete circle according to the longest arc; Step S4-1: using Canny edge detection to extract edge contours within the annular ROI obtained in step S3-4, and obtaining a binary edge image; Step S4-2: for each continuous edge detected in step S4-1, measure its length, those with length less than the threshold are regarded as noise and filtered out, and take all continuous edges with length greater than the threshold; Step S4-3: For each continuous edge obtained in step S4-2 whose length is greater than the threshold, fit it into an arc, and the expression is: In the formula, are the coordinates of the fitted circle, is the diameter of the fitted circle, For the Coordinates of contour points ; Step S4-4: for each arc fitted in step S4-3, measure its length and select the longest arc; Step S4-5: Complete the circle according to the longest arc selected in step S4-4.

[0020] Step S5: According to the camera internal and external parameter matrix obtained in step S1, the image and coordinates are converted and output, including converting the complete circle fitted in the pixel coordinate system to the world coordinate system, such as Figure 3 As shown, the output image and barrel mouth coordinates; Step S5-1: According to the camera intrinsic parameter matrix and extrinsic parameter matrix obtained in step S2, the complete circle obtained in step S4-5 is converted into a world coordinate system whose unit is millimeter and whose coordinate axis origin and positive direction are consistent with the filling head, and the two-dimensional coordinates and diameter of the complete circle in the world coordinate system are obtained; Step S5-2: According to the n complete circles obtained in step S4-5, Figure 3As shown, n complete circle images are displayed on the complete barrel surface grayscale image acquired in step S1-1, and the complete circle diameters detected from the n barrel openings in the world coordinate system are compared, and the complete circle coordinates in the world coordinate system that are closer to the barrel opening design diameter are taken.

[0021] Step S6: The barrel mouth coordinates obtained in step S5 are transmitted to the PLC, driving the filling head to move to the barrel mouth center; Step S7: Repeat steps S2 to S6 until all buckets are positioned.

[0022] It should be noted that, in this document, relational terms such as first and second, etc. are merely used to distinguish one entity from another entity, but do not necessarily require or imply the existence of any such actual relationship or order between these entities.

[0023] The above examples are merely illustrative of the present invention and do not constitute a limitation on the protection scope of the present invention. All designs that are the same or similar to the present invention belong to the protection scope of the present invention.

Claims

1. A barrel opening positioning method based on machine vision, characterized in that: The process includes the following steps: Step S1: before leaving the factory, perform online steps to obtain the camera internal and external parameter matrix, template matching file, relative position, and photo position, including camera calibration, frame selection template, template matching NCC file generation, relative position calibration, and photo position determination; Step S2: According to the camera intrinsic parameter matrix obtained in step S1, the image captured by the camera is subjected to distortion correction; Step S3: determining the ROI (region of interest) for edge detection, including template matching, generating a rectangular ROI, binarization, taking the binarized edge, dilation processing, and generating a ring ROI; Step S4: fitting a complete circle corresponding to the barrel mouth in the pixel coordinate system, including edge detection, edge length measurement, noise screening, arc fitting, arc length measurement, and completing the complete circle according to the longest arc; Step S5: according to the camera internal and external parameter matrix obtained in step S1, convert and output the image and coordinates, including converting the complete circle fitted in the pixel coordinate system to the world coordinate system, and outputting the image and barrel mouth coordinates; Step S6: The barrel mouth coordinates obtained in step S5 are transmitted to the PLC, driving the filling head to move to the barrel mouth center; Step S7: Repeat steps S2 to S6 until all buckets are positioned.

2. The barrel opening positioning method based on machine vision according to claim 1, characterized in that: In step S1, the online steps before delivery include: Step S1-1: The area array camera is automatically connected and collects grayscale images of the complete barrel surface and images of calibration plates in different postures; Step S1-2: Perform camera calibration according to the calibration plate image acquired in step S1-1 to obtain the camera intrinsic parameter matrix and extrinsic parameter matrix; Step S1-3: manually select a template image based on the complete barrel surface grayscale image acquired in step S1-1, and save it as a file; Step S1-4: Generate a template matching NCC (normalized cross correlation) file according to the template image file saved in step S1-3; Step S1-5: calibrate the relative position of the camera center and the filling head in the X and Y directions; Step S1-6: For each barrel, determine the photographing position to ensure that the camera can capture the entire barrel surface at each photographing position.

3. The barrel opening positioning method based on machine vision according to claim 1, characterized in that: The step of determining the ROI for edge detection in step S3 includes: Step S3-1: For each barrel opening (assuming that each barrel has n barrel openings), perform NCC template matching within the range of the distortion-corrected complete barrel surface grayscale image obtained in step S2 according to the template matching NCC file obtained in step S1-4, and obtain n coordinates with the greatest matching degree. The expression is: In the formula, is the coordinate of the upper left corner of the sub-image in the image, is the coordinate of the pixel in the template, For template Pixel width and height, For subgraph The average pixel value of For template The average pixel value of Step S3-2: for each coordinate with the greatest matching degree obtained in step S3-1, generate a rectangular ROI with the coordinate as the upper left corner and the width and height equal to the width and height of the template image; Step S3-3: For each rectangular ROI area obtained in step S3-2, perform Otsu binarization processing and take the outer edge of the binarized image, which is expressed as: In the formula, is the Otsu operator at the point The value at is the Laplace operator, is a Gaussian function, is the input image, is the convolution operation; Step S3-4: For the outer edge of the binary image obtained in step S3-3, dilation processing is performed, and the dilated image is taken as a ring ROI, which is expressed as: In the formula, is the binary internal edge, B is the structural element, is a point in the dilated image, is the two-dimensional real number plane, For collection The point in For collection The point in .

4. The barrel opening positioning method based on machine vision according to claim 1, characterized in that: The step S4 of fitting a complete circle corresponding to the barrel opening in the pixel coordinate system comprises: Step S4-1: using Canny edge detection to extract edge contours within the annular ROI obtained in step S3-4, and obtaining a binary edge image; Step S4-2: for each continuous edge detected in step S4-1, measure its length, those with length less than the threshold are regarded as noise and filtered out, and take all continuous edges with length greater than the threshold; Step S4-3: For each continuous edge obtained in step S4-2 whose length is greater than the threshold, fit it into an arc, and the expression is: In the formula, are the coordinates of the fitted circle, is the diameter of the fitted circle, For the Coordinates of contour points ; Step S4-4: for each arc fitted in step S4-3, measure its length and select the longest arc; Step S4-5: Complete the circle according to the longest arc selected in step S4-4.

5. The barrel opening positioning method based on machine vision according to claim 1, characterized in that: The step S5 of converting and outputting the image and coordinates comprises: Step S5-1: According to the camera intrinsic parameter matrix and extrinsic parameter matrix obtained in step S2, the complete circle obtained in step S4-5 is converted into a world coordinate system whose unit is millimeter and whose coordinate axis origin and positive direction are consistent with the filling head, and the two-dimensional coordinates and diameter of the complete circle in the world coordinate system are obtained; Step S5-2: Based on the n complete circles obtained in step S4-5, the n complete circle images are displayed on the complete barrel surface grayscale image acquired in step S1-1, and the complete circle diameters detected from the n barrel openings in the world coordinate system are compared to obtain the complete circle coordinates in the world coordinate system that are closer to the barrel opening design diameter.

Citation Information

Patent Citations

  • Laser vision three-dimensional vision positioning method for a fillet weld in a tank body

    CN109658456A

  • Filling barrel opening position measuring method based on machine vision

    CN117132655A

  • Calibration method for 3D structured light system, and electronic device and storage medium

    WO2022052313A1