Method, system and device for detecting aperture, pitch and hole position based on machine vision

Abstract

The present application belongs to the technical field of mechanical measurement method and mechanical measurement device, and particularly relates to a hole diameter, hole pitch and hole position detection method, system and device based on machine vision. The method of the present application moves the detection equipment according to a predetermined path through a motion assembly, collects images of hole diameter, hole pitch and hole position, and processes the images using a software system, including preprocessing using Gaussian filtering, finding the maximum inscribed circle, the fitting circle, the minimum circumscribed circle of the hole and the boundary of the measured region using various algorithms, and then calculating the hole diameter, hole pitch and hole position using the three circles, the boundary of the measured region and the center coordinates. Further, the hole quality is evaluated by grading using a deep learning-based method. The method has the characteristics of high detection efficiency, fast speed, small error, good method stability, convenient operation and online detection support, and can well meet the production needs of the mechanical manufacturing industry, and has a broad application prospect.

Description

TECHNICAL FIELD

[0001] The present application belongs to the technical field of mechanical measurement method and mechanical measurement device, and particularly relates to a hole diameter, hole pitch and hole position detection method and system based on machine vision and a device related thereto. BACKGROUND

[0002] At present, the detection methods for hole diameter, hole pitch and hole position in the field of mechanical manufacturing and related scientific research work are mostly manual detection, that is, technical personnel use micrometers or go-no-go gauges and other measuring tools to detect the hole diameter, hole pitch and hole position of a workpiece. However, manual detection has inherent defects such as low detection efficiency, large error, low precision and poor reproducibility, and most of the detection needs to be carried out under actual contact conditions, which may cause accidental damage to the workpiece, and it is difficult to meet the application requirements of high efficiency, high speed and high precision detection in the mechanical manufacturing industry. Therefore, it is urgent to develop an automatic hole diameter detection method with high detection efficiency, small error, high precision and good reproducibility to meet the actual production needs.

[0003] In addition, although there are some automatic measuring instruments for hole diameter detection, such as the IMAGE3 series of image measuring instruments of POMEAS, they use telecentric lenses and telecentric light sources to detect the hole diameter in a backlit manner. This detection method judges the edge of the object by whether there is an obstruction, and obtains related size information based on this. Therefore, this type of image measuring instrument is only suitable for detection conditions without obstruction from the back and offline detection methods. During the detection process, the detection accuracy and accuracy are easily affected by the material of the back of the measured object, and most of these measuring instruments do not support online detection, which is not convenient for actual application. SUMMARY

[0004] In order to overcome the defects and shortcomings of the traditional manual detection method for hole diameter, hole pitch and hole position, the present application provides a solution: the method of the present application moves along a predetermined path by carrying a detection device through a motion assembly, collects images of the hole diameter, hole pitch and hole position, and processes the images using a software system, including pre-processing using Gaussian filtering, finding the maximum inscribed circle of the hole using an edge detection algorithm, finding the fitting circle of the hole using the least squares method, finding the minimum circumscribed circle of the hole using a binarization algorithm and an edge detection algorithm, finding the boundary of the measured region using an edge detection algorithm, and then calculating the hole diameter, hole pitch and hole position using the above three circles, the boundary of the measured region and the center coordinates. Further, the hole quality is evaluated by a deep learning-based method. Meanwhile, the present application also relates to a hole diameter, hole pitch and hole position detection system and an automatic detection device.

[0005] In a first aspect, the present application provides a hole diameter, hole pitch and hole position detection method based on machine vision, which comprises the following steps:

[0006] (1) The detection device carried by the motion assembly moves along a pre-set detection path;

[0007] (2) The detection device collects images of the aperture, pitch and hole position during the movement;

[0008] (3) The collected images are pre-processed using Gaussian filtering in the software processing system;

[0009] (4) In the software processing system, the largest inscribed circle of the measured hole is found using the edge detection algorithm, the fitted circle of the measured hole is found using the least squares method, the smallest circumscribed circle of the measured hole is found using the binarization algorithm and the edge detection algorithm, and the boundary of the measured region is found using the edge detection algorithm;

[0010] (5) In the software processing system, the aperture, pitch and hole position are calculated using the largest inscribed circle, fitted circle, smallest circumscribed circle and boundary of the measured region found in the previous step, combined with the center coordinates;

[0011] (6) In the software processing system, the hole quality is graded and evaluated using a deep learning-based method.

[0012] Further, the execution method of steps (4)-(6) in the above detection method is as follows:

[0013] A. The canny edge detection algorithm is used to find the measured hole, first finding all edge-closed shapes in the measured region, and then judging whether the found shape is the measured hole, the criterion being:

[0014] a. Find the center (x, y) of each shape:

[0015]

[0016] Where n is the number of shape points, P is the set of all points in the shape, P ix is the x-coordinate of the point, and P iy is the y-coordinate of the point;

[0017] b. Calculate the set d of distances from each point on the shape to the center:

[0018]

[0019] Where C is the set of all points on the shape;

[0020] c. Calculate the mean and standard deviation of the set of distances from each point on the shape to the center:

[0021]

[0022]

[0023] d. Get the roundness result c:

[0024]

[0025] e. If the roundness result is greater than the threshold value, it is identified as a hole to be measured;

[0026] B. According to the contour shape of the hole to be measured found in the previous step, find the center and diameter of the fitting circle Fx, Fy, Fd, the center and diameter of the maximum inscribed circle Ix, Iy, Id, and the center and diameter of the minimum circumscribed circle Zx, Zy, Zd;

[0027] a. Fitting circle: take the center and diameter as variables, and fit the fitting circle by least squares method;

[0028] b. Maximum inscribed circle: traverse all points inside the contour, and calculate the distance of these points to the contour. The point farthest from the contour inside the contour is the center, and the maximum distance between the center and the contour is the radius. In this way, the maximum inscribed circle is obtained;

[0029] c. Minimum circumscribed circle: find a point on the upper, lower, left and right edges of the contour respectively and calculate its circumscribed circle. If any point in the contour is outside the circumscribed circle, introduce the point farthest from the circumscribed circle outside the circumscribed circle and calculate a new circumscribed circle. Traverse until all points in the contour are within the calculated circumscribed circle, and the minimum circumscribed circle is obtained;

[0030] C. Store the data results obtained in the previous step, and use them together with the true hole diameter of the hole to be measured as a training set. Use deep neural network to train and obtain weight parameter model w;

[0031] D. After repeating the training at least 10 times, the hole with unknown diameter can be measured and the center of the circle can be found:

[0032]

[0033] where w1, w2 … w 10 are the results of the weight parameter model w corresponding to each input item;

[0034] E. Hole quality grading evaluation: use the image of the hole taken by the detection equipment and the corresponding hole type as a training set, use convolutional neural network to train, then store the trained model, and use the model to grade and evaluate the quality of the hole during detection.

[0035] Preferably, the motion assembly in the above method is a robot or other automated motion device capable of carrying the detection equipment and moving along a predetermined path.

[0036] Preferably, the detection device in the above method is composed of a first camera, a first lens, a second camera, a second lens and a light source.

[0037] The detection device is connected with the motion assembly through a mechanical adapter and is carried by the motion assembly to move and take pictures according to a preset detection path.

[0038] The motion assembly and the detection device are connected with and exchange data with an upper computer fixedly arranged in a control box through a data transmission medium, the upper computer is internally provided with a control system, a receiving system and a software processing system, the control system is used to control the movement of the motion assembly and the image acquisition process of the detection device, the receiving system is used to receive the position information of the motion assembly and the image information transmitted back by the detection device, and the software processing system is used to pre-process, element analyze, calculate and result evaluate the image transmitted back by the detection device and feed back the processing result.

[0039] Preferably, the mechanical adapter in the above method is a flange, and the data transmission medium is a network cable.

[0040] In a second aspect, the application provides a hole diameter, hole distance and hole position detection system based on machine vision, which is composed of a motion assembly, a detection device, a data transmission medium, a control box and an upper computer, the upper computer is internally provided with a control system, a receiving system and a software processing system; the detection system is operated and run according to the above hole diameter, hole distance and hole position detection method based on machine vision.

[0041] In a third aspect, the application provides a hole diameter, hole distance and hole position detection device based on machine vision, which is composed of a motion assembly 101, a detection device 102, a control box 103 and an upper computer 104.

[0042] The detection device 102 is connected with the motion assembly 101 through a mechanical adapter, and the detection device 102 and the motion assembly 101 are connected with the control box 103 and the upper computer 104 through a network cable.

[0043] The upper computer 104 is fixedly arranged in the control box 103, and the upper computer 104 is internally provided with a control system, a receiving system and a software processing system, the control system is used to control the movement of the motion assembly 101 and the image acquisition process of the detection device 102, the receiving system is used to receive the position information of the motion assembly 101 and the image information transmitted back by the detection device 102 through the network cable, and the software processing system is used to pre-process, element analyze, calculate and result evaluate the image transmitted back by the detection device 102 and feed back the processing result.

[0044] The operation mode of the device is as follows:

[0045] (1) Detection device 102 carried by motion assembly 101 moves along the preset detection path;

[0046] (2) During the movement along the preset detection path, detection device 102 collects images of the to-be-tested area, collects images of the to-be-tested holes and related areas required for detection, and the images include images of hole diameter, hole pitch and hole position. Detection device 102 transmits the above images to host computer 104 through a network cable;

[0047] (3) Host computer 104 uses Gaussian filtering to pre-process the images transmitted back by detection device 102 through the network cable;

[0048] (4) Host computer 104 finds the largest inscribed circle of the to-be-tested hole through edge detection algorithm after pre-processing, finds the fitting circle of the to-be-tested hole through least square method, and finds the smallest circumscribed circle of the to-be-tested hole through binarization algorithm and edge detection algorithm. At the same time, host computer 104 finds the boundary of the to-be-tested area through edge detection algorithm;

[0049] (5) Host computer 104 calculates the hole diameter, hole pitch and hole position by combining the center coordinates of the largest inscribed circle, the fitting circle, the smallest circumscribed circle and the boundary of the to-be-tested area found in the previous step;

[0050] (6) Host computer 104 uses a deep learning-based method to make a grading assessment of the hole quality.

[0051] Further, the execution method of steps (4)-(6) in the operation mode of the above detection device is as follows:

[0052] A. Use canny edge detection algorithm to find the to-be-tested hole, first find all edge closed shapes in the to-be-tested area, and then judge whether the found shape is the to-be-tested hole, the criterion is:

[0053] a. Find the center (x, y) of each shape:

[0054]

[0055] Where n is the number of shape inner points, P is the set of all points in the shape, P ix is the x coordinate of the point, P iy is the y coordinate of the point;

[0056] b. Calculate the set d of distances from each point on the shape to the center:

[0057]

[0058] Where C is the set of all points on the shape;

[0059] c. Calculate the mean and standard deviation of the set of distances from each point on the shape to the center:

[0060]

[0061]

[0062] d. Get the roundness result c:

[0063]

[0064] e. If the roundness result is greater than the threshold value, it is identified as the hole to be measured;

[0065] B. According to the contour shape of the hole to be measured found in the previous step, find the center and diameter of the fitting circle Fx, Fy, Fd, the center and diameter of the largest inscribed circle Ix, Iy, Id, and the center and diameter of the smallest circumscribed circle Zx, Zy, Zd, respectively;

[0066] a. Fitting circle: take the center and diameter as variables, and fit the fitting circle by least squares method;

[0067] b. Largest inscribed circle: traverse all points inside the contour, and calculate the distance of these points to the contour. The point farthest from the contour inside the contour is the center, and the maximum distance between the center and the contour is the radius. In this way, the largest inscribed circle is obtained;

[0068] c. Smallest circumscribed circle: find a point on the upper, lower, left and right edges of the contour respectively and calculate its circumscribed circle. If there is a point in the contour outside the circumscribed circle, introduce the point farthest from the circumscribed circle outside the circumscribed circle and calculate a new circumscribed circle. Traverse until all points in the contour are within the calculated circumscribed circle, and the smallest circumscribed circle is obtained;

[0069] C. Store the data results obtained in the previous step and use them together with the true hole diameter of the hole to be measured as a training set. Train using a deep neural network to obtain a weight parameter model w;

[0070] D. After repeating the training at least 10 times, the hole with unknown hole diameter can be measured and the center of the circle can be found:

[0071]

[0072] where w1, w2 … w 10 are the results of the weight parameter model w corresponding to each input item;

[0073] E. Hole quality grading evaluation: the image of the hole taken by the detection equipment and the corresponding hole type are used as a training set to train a convolutional neural network, then the trained model is stored, and the hole quality is graded and evaluated by using the model during detection.

[0074] Preferably, the motion assembly 101 in the detection device is a robot or other automated motion device capable of carrying the detection equipment 102 and moving along a predetermined path.

[0075] Preferably, the host computer 104 in the detection device is a device comprising a processor, an operating system, a network interface, and a display terminal.

[0076] More preferably, the host computer 104 in the detection device is a PC, an industrial computer, or a PLC.

[0077] Preferably, the detection equipment 102 in the detection device consists of a first camera, a first lens, a second camera, a second lens, and a light source.

[0078] The detection equipment 102 is connected to the motion assembly 101 through a mechanical adapter and is carried by the motion assembly 101 to move along a pre-set detection path for photographing.

[0079] The motion assembly 101 and the detection equipment 102 are connected to and interact with the host computer 104 fixedly installed in the control box 103 through a network cable, and the host computer 104 has a control system, a receiving system, and a software processing system built-in, the control system is used to control the movement of the motion assembly 101 and the image acquisition process of the detection equipment 102, the receiving system is used to receive the position information of the motion assembly 101 and the image information transmitted back by the detection equipment 102, and the software processing system is used to pre-process, analyze, calculate, and evaluate the results of the images transmitted back by the detection equipment 102, and feedback the processing results.

[0080] More preferably, the mechanical adapter in the detection device is a flange.

[0081] In summary, the hole diameter, hole pitch, and hole position detection method and device based on machine vision have the following advantages:

[0082] (1) The full-automatic detection method has high detection efficiency, speed, and accuracy, and the detection precision and reproducibility are significantly improved compared with manual detection.

[0083] (2) The method is not affected by the material on the back of the measured object, has low requirements for detection conditions, and has high stability and detection accuracy.

[0084] (3) The non-contact detection method can effectively avoid accidental damage to the workpiece.

[0085] (4) The operation is convenient, supports online detection mode, and can well meet the production requirements of the mechanical manufacturing industry. BRIEF DESCRIPTION OF DRAWINGS

[0086] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following briefly introduces the drawings needed to be used in the embodiment description. Obviously, the drawings in the following description are only some embodiments of the present application, not all embodiments, and other drawings can be obtained by those skilled in the art without creative labor based on these drawings.

[0087] Figure 1 It is the implementation flowchart of the method of the present application.

[0088] Figure 2 It is the overall structure schematic diagram of the device of the present application.

[0089] Figure 3 It is the internal structure schematic diagram of the control box in the device of the present application.

[0090] Figure 4 It is the internal structure schematic diagram of the detection equipment in the device of the present application.

[0091] Figure 5A And Figure 5B It is the aperture measurement result schematic diagram of the method of the present application in actual cases. DETAILED DESCRIPTION

[0092] In order to make the purpose, technical scheme and advantages of the present application more clear, the technical scheme of the present application will be described clearly and completely in the following combined with specific embodiments and corresponding drawings. Obviously, the described embodiments are only some embodiments of the present application, not all embodiments, and the present application can also be implemented or applied by other different specific embodiments, and each detail in the specification can be modified or changed based on different views and applications without departing from the spirit of the present application.

[0093] At the same time, it should be understood that the protection scope of the present application is not limited to the following specific specific embodiments; it should also be understood that the terms used in the embodiments of the present application are for describing the specific specific embodiments, not for limiting the protection scope of the present application.

[0094] Embodiment 1: A hole diameter, hole pitch and hole position detection method based on machine vision (see Figure 1 ), the method comprises the following steps:

[0095] (1) The detection equipment is connected with the robot through the flange, and moves and takes pictures according to the pre-set detection path by the robot;

[0096] (2) The detection device is composed of a first camera, a first lens, a second camera, a second lens and a light source; during the movement, the images of the aperture, pitch and hole position are collected by the detection device;

[0097] (3) The robot and the detection device are connected to the host computer fixedly arranged in the control box through a network cable and data interaction is carried out, the host computer is internally provided with a control system, a receiving system and a software processing system, the control system is used for controlling the movement of the robot and the image collection process of the detection device, the receiving system is used for receiving the position information of the robot and the image information returned by the detection device, and the software processing system is used for pre-processing, element analysis, calculation and result evaluation on the image returned by the detection device, and feedback of the processing result; first, the collected image is pre-processed by using Gaussian filtering in the software processing system;

[0098] (4) In the software processing system, the largest inscribed circle of the measured hole is found out by using the edge detection algorithm, the fitting circle of the measured hole is found out by using the least square method, and the smallest circumscribed circle of the measured hole is found out by using the binarization algorithm and the edge detection algorithm; at the same time, the boundary of the measured region is found out by using the edge detection algorithm;

[0099] (5) In the software processing system, the hole diameter, pitch and position are calculated by combining the center coordinates of the largest inscribed circle, the fitting circle, the smallest circumscribed circle and the boundary of the measured region found out in the previous step;

[0100] (6) In the software processing system, the hole quality is graded and evaluated by using a deep learning-based method.

[0101] The execution methods of steps (4)-(6) are as follows:

[0102] A. The canny edge detection algorithm is used to find the measured hole, all edge closed shapes in the measured region are found out, and then it is judged whether the found shape is the measured hole, the criterion is:

[0103] a. The center (x, y) of each shape is found out:

[0104]

[0105] Where n is the number of shape points, P is the set of all points in the shape, P ix is the x coordinate of the point, P iy is the y coordinate of the point;

[0106] b. The set d of distances of each point on the shape to the center is calculated:

[0107]

[0108] Where C is the set of all points on the shape;

[0109] c. Calculate the mean mean and standard deviation std of the set of distances from each point to the center:

[0110]

[0111]

[0112] d. Get the roundness result c:

[0113]

[0114] e. If the roundness result is greater than the threshold value, it is identified as the hole to be measured;

[0115] B. According to the contour shape of the hole to be measured found in the previous step, find the center and diameter of the fitting circle Fx, Fy, Fd, the center and diameter of the largest inscribed circle Ix, Iy, Id and the center and diameter of the smallest circumscribed circle Zx, Zy, Zd;

[0116] a. Fitting circle: take the center and diameter as variables, and fit the fitting circle by least squares method;

[0117] b. Largest inscribed circle: traverse all points inside the contour, and calculate the distance between these points and the contour. The point farthest from the contour inside the contour is the center, and the maximum distance between the center and the contour is the radius. In this way, the largest inscribed circle is obtained;

[0118] c. Smallest circumscribed circle: find a point on the upper, lower, left and right edges of the contour respectively and calculate its circumscribed circle. If there is a point in the contour outside the circumscribed circle, introduce the point farthest from the circumscribed circle outside the circumscribed circle and calculate a new circumscribed circle. Traverse until all points in the contour are within the calculated circumscribed circle, and the smallest circumscribed circle is obtained;

[0119] C. Store the data results obtained in the previous step and use them together with the true hole diameter of the hole to be measured as a training set. Train using a deep neural network to obtain a weight parameter model w;

[0120] D. After repeating the training at least 10 times, the hole with unknown diameter can be measured and the center of the circle can be found:

[0121]

[0122] where w1, w2 … w 10 are the results of the weight parameter model w corresponding to each input item;

[0123] E. Hole quality grading evaluation: the image of the hole taken by the detection equipment and the corresponding hole type are used as a training set to train a convolutional neural network, and then the trained model is stored and used to grade the hole quality during detection.

[0124] Embodiment 2: a hole diameter, hole pitch and hole position detection device based on machine vision (see Figures 2-4 ), which consists of a robot 101, a detection equipment 102, a control box 103, and an upper computer 104;

[0125] The detection equipment 102 consists of a first camera, a first lens, a second camera, a second lens and a light source. The detection equipment 102 is connected to the robot 101 through a flange and is carried by the robot 101 to move and take pictures according to the pre-set detection path;

[0126] The detection equipment 102 and the robot 101 are connected to and interact with the upper computer 104 fixedly installed in the control box 103 through a network cable;

[0127] The upper computer 104 (PC computer) is fixedly installed in the control box 103. The upper computer 104 is built-in with a control system, a receiving system and a software processing system. The control system is used to control the movement of the robot 101 and the image acquisition process of the detection equipment 102. The receiving system is used to receive the position information of the robot 101 and the image information transmitted back by the detection equipment 102 through the network cable. The software processing system is used to preprocess, element analysis, calculation and result evaluation and feedback of the image transmitted back by the detection equipment 102;

[0128] The operation mode of the device is as follows:

[0129] (1) The detection equipment 102 is carried by the robot 101 to move according to the pre-set detection path;

[0130] (2) During the movement of the pre-determined detection path, the detection equipment 102 acquires images of the to-be-measured area, including the images of the to-be-measured holes and the related area, including the images of the hole diameter, hole pitch and hole position. The detection equipment 102 transmits the above images to the upper computer 104 through the network cable;

[0131] (3) The upper computer 104 uses Gaussian filter to preprocess the images transmitted back by the detection equipment 102 through the network cable;

[0132] (4) The upper computer 104 finds the maximum inscribed circle of the to-be-measured hole through the edge detection algorithm, finds the fitting circle of the to-be-measured hole through the least square method, and finds the minimum circumscribed circle of the to-be-measured hole through the binarization algorithm and the edge detection algorithm. At the same time, the upper computer 104 finds the boundary of the measured area through the edge detection algorithm;

[0133] (5) The host computer 104 calculates the hole diameter, hole pitch and hole position using the maximum inscribed circle, the fitting circle, the minimum circumscribed circle and the boundary of the measured area found in the previous step, in combination with the center coordinates;

[0134] (6) The host computer 104 uses a deep learning-based method to make a grading evaluation of the hole quality.

[0135] Among them, the execution method of steps (4)-(6) in the operation mode of the above detection device is as follows:

[0136] A. Use the canny edge detection algorithm to find the measured hole, first find all the edge closed shapes in the measured area, and then judge whether the found shape is the measured hole, the criterion is:

[0137] a. Find the center (x, y) of each shape:

[0138]

[0139] Where n is the number of shape points, P is the set of all points in the shape, P ix is the x coordinate of the point, P iy is the y coordinate of the point;

[0140] b. Calculate the set d of distances from each point on the shape to the center:

[0141]

[0142] Where C is the set of all points on the shape;

[0143] c. Calculate the mean and standard deviation of the set of distances from each point on the shape to the center:

[0144]

[0145]

[0146] d. Get the roundness result c:

[0147]

[0148] e. If the roundness result is greater than the threshold value, it is identified as the measured hole;

[0149] B. According to the contour shape of the measured hole found in the previous step, find the center and diameter of the fitting circle Fx, Fy, Fd, the center and diameter of the maximum inscribed circle Ix, Iy, Id and the center and diameter of the minimum circumscribed circle Zx, Zy, Zd;

[0150] a. Fitting circle: fitting a circle by least square method with center and diameter as variables;

[0151] b. Largest inscribed circle: traversing all points inside the contour and calculating the distance from these points to the contour, the point farthest from the contour is the center of the largest inscribed circle, and the largest distance between the center and the contour is the radius of the largest inscribed circle;

[0152] c. Smallest circumscribed circle: finding a point on the upper edge and a point on the lower edge of the contour respectively and calculating the circumscribed circle of these two points, if there is a point in the contour outside the circumscribed circle, then the point farthest from the circumscribed circle in the outside of the circumscribed circle is introduced and a new circumscribed circle is calculated, and the process is repeated until all points in the contour are inside the calculated circumscribed circle, and the smallest circumscribed circle is obtained;

[0153] C. Store the data obtained in the previous step and use it together with the real hole diameter of the hole to be measured as a training set, and use deep neural network to train and obtain weight parameter model w;

[0154] D. Repeat the training at least 10 times, and then measure the unknown hole diameter and find the center of the circle:

[0155]

[0156] where w1, w2 … w 10 is the result of each input item corresponding to the weight parameter model w;

[0157] E. Hole quality grading evaluation: use the image of the hole taken by the detection equipment and the corresponding hole type as a training set, use convolutional neural network to train, then store the trained model, and use the model to grade and evaluate the quality of the hole during detection.

[0158] Example 3: Application of the method of example 1 in the case of hole diameter measurement (see Figure 5A and Figure 5B ), wherein:

[0159] The left graph shows the hole distance and hole position measurement results. The four parameters listed on the left, right, top and bottom margins are calculated for the left hole. Among them, the left margin is the distance from the left edge to the center of the hole when the left hole is the first hole on the left of the measured object; the right margin is the distance between the two holes (i.e. hole distance), if the left hole is the last hole on the left of the measured object, then the right margin is the distance from the right edge to the center of the hole; the upper margin is the distance from the center of the hole to the upper edge; the lower margin is the distance from the center of the hole to the lower edge;

[0160] The right image shows the aperture measurement result. When measuring the aperture, the right image has only one hole in the field of view. The aperture size is obtained according to the above-mentioned aperture measurement method, and the hole quality is evaluated to obtain the conclusion of whether it is a standard hole.

[0161] Each embodiment in the present application is described in a progressive manner, and the same or similar parts between each embodiment can be referred to each other. Each embodiment mainly explains the difference from other embodiments.

[0162] The above only describes the embodiments of the present application and is not used to limit the present application. The present application can be variously changed and modified by those skilled in the art. Any modification, replacement, etc. within the spirit and principle of the present application should be included in the protection scope of the claims of the present application.

Claims

1. A method for detecting aperture, aperture spacing, and aperture position based on machine vision, characterized in that, The method includes the following steps: (1) The detection device is carried by the motion component and moves along a pre-set detection path; (2) During the movement, the detection equipment collects images of the aperture, aperture spacing, and aperture position; (3) Use Gaussian filtering in the software processing system to preprocess the acquired images; (4) In the software processing system, the edge detection algorithm is used to find the maximum inscribed circle of the hole to be measured, the least squares method is used to find the fitted circle of the hole to be measured, and the binarization algorithm and the edge detection algorithm are used to find the minimum circumscribed circle of the hole to be measured; at the same time, the edge detection algorithm is used to find the boundary of the measured area. (5) In the software processing system, the maximum inscribed circle, the fitted circle, the minimum circumscribed circle and the boundary of the measured area found in the previous step are used to calculate the hole diameter, hole spacing and hole position in combination with the center coordinates. (6) Use deep learning-based methods to grade and evaluate hole quality in the software processing system; The execution method for steps (4) to (6) is as follows: A. Use the Canny edge detection algorithm to find the hole to be tested. First, find all closed edge shapes in the area to be tested, and then determine whether the found shapes are the hole to be tested. The criterion is: a. Find the center (x, y) of each shape: Where n is the number of points within the shape, and P is the set of all points within the shape. ix Let P be the x-coordinate of the point. iy Let y be the coordinate of the point; b. Calculate the set d of distances from each point on the shape to the center: Where C is the set of all points in the shape; c. Calculate the mean and standard deviation (std) of the set of distances from each point on the shape to the center: d. Obtain the roundness result c: e. If the roundness result is greater than the threshold, it is identified as a hole to be tested; B. Based on the outline shape of the hole to be tested found in the previous step, find the center and diameter Fx, Fy, Fd of the fitted circle, the center and diameter Ix, Iy, Id of the largest inscribed circle, and the center and diameter Zx, Zy, Zd of the smallest circumscribed circle. a. Fitted circle: Using the center and diameter as variables, a fitted circle is obtained by fitting the circle using the least squares method; b. Maximum incircle: Traverse all points inside the contour and calculate the distance from these points to the contour. The point inside the contour that is farthest from the contour is the center of the circle, and the maximum distance between it and the contour is the radius. This is how to obtain the maximum incircle. c. Minimum circumcircle: Find one point on each of the top, bottom, left, and right edges of the contour and calculate its circumcircle. If a point in the contour is outside this circumcircle, then introduce the point outside the circumcircle that is farthest from the circumcircle and calculate a new circumcircle. Continue this process until all points in the contour are inside the calculated circumcircle, thus obtaining the minimum circumcircle. C. Store the data obtained in the previous step, and use it together with the actual aperture of the hole to be measured as a training set. Use a deep neural network to train the model and obtain the weight parameter model w. D. After repeating the training at least 10 times, measure the hole with the unknown diameter and find the center: Where w1, w2 … w 10 The result of the model w for the weight parameters corresponding to each input item; E. Hole Quality Grading Assessment: The images of holes captured by the detection equipment and their corresponding hole types are used as the training set. A convolutional neural network is then used to train the model. The trained model is then stored and used to grade and assess hole quality during the detection process.

2. The method for detecting aperture, aperture spacing, and aperture position according to claim 1, characterized in that, The motion component is a robot or other automated motion device capable of carrying detection equipment and moving along a predetermined path.

3. The method for detecting aperture, aperture spacing, and aperture position according to claim 1, characterized in that, The detection device consists of a first camera, a first lens, a second camera, a second lens, and a light source; The detection device is connected to the motion component via a mechanical transfer structure, and is carried by the motion component to move and take pictures according to a pre-set detection path; The motion component and the detection device are connected to a host computer fixed in the control box via a data transmission medium and interact with each other. The host computer has a built-in control system, a receiving system, and a software processing system. The control system is used to control the movement of the motion component and the image acquisition process of the detection device. The receiving system is used to receive the position information of the motion component and the image information transmitted back by the detection device. The software processing system is used to preprocess, analyze, calculate, and evaluate the images transmitted back by the detection device and provide feedback on the processing results.

4. The method for detecting aperture, aperture spacing, and aperture position according to claim 3, characterized in that, The mechanical adapter structure is a flange, and the data transmission medium is a network cable.

5. A machine vision-based system for detecting aperture, hole spacing, and hole position, characterized in that, The detection system comprises a motion component, a detection device, a data transmission medium, a control box, and a host computer. The host computer has a built-in control system, a receiving system, and a software processing system. The detection system operates and runs in accordance with the machine vision-based aperture, aperture spacing, and aperture position detection method as described in any one of claims 1-4.

6. A machine vision-based device for detecting aperture, hole spacing, and hole position, characterized in that, The device consists of a motion component (101), a detection device (102), a control box (103), and a host computer (104); The detection device (102) is connected to the motion component (101) through a mechanical transfer structure. The detection device (102) and the motion component (101) are connected to the control box (103) and the host computer (104) through a network cable. The host computer (104) is fixedly installed in the control box (103). The host computer (104) has a built-in control system, a receiving system and a software processing system. The control system controls the movement of the motion component (101) and the image acquisition process of the detection device (102). The receiving system receives the position information of the motion component (101) and the image information transmitted back by the detection device (102) through the network cable. The software processing system performs preprocessing, element analysis, calculation and result evaluation and feedback on the image transmitted back by the detection device (102). The device operates as follows: (1) The motion component (101) carries the detection device (102) and moves it along a pre-set detection path; (2) During the movement of the predetermined detection path, the detection device (102) acquires images of the area to be tested, and acquires images of the holes to be tested and related areas to be tested. The images include images of hole diameter, hole spacing and hole position. The detection device (102) transmits the above images to the host computer (104) via network cable. (3) The host computer (104) preprocesses the image transmitted back from the detection device (102) via the network cable using Gaussian filtering; (4) The host computer (104) uses the edge detection algorithm to find the maximum inscribed circle of the hole to be tested in the preprocessed image, uses the least squares method to find the fitted circle of the hole to be tested, and uses the binarization algorithm and the edge detection algorithm to find the minimum circumscribed circle of the hole to be tested; at the same time, the host computer (104) uses the edge detection algorithm to find the boundary of the area to be tested. (5) The host computer (104) uses the largest inscribed circle, fitted circle, smallest circumscribed circle and the boundary of the measured area found in the previous step, combined with the center coordinates of the circle, to calculate the hole diameter, hole spacing and hole position; (6) The host computer (104) uses a deep learning-based method to grade and evaluate the hole quality; The execution method of steps (4)-(6) in its operation mode is as follows: A. Use the Canny edge detection algorithm to find the hole to be tested. First, find all closed edge shapes in the area to be tested, and then determine whether the found shapes are the hole to be tested. The criterion is: a. Find the center (x, y) of each shape: Where n is the number of points within the shape, and P is the set of all points within the shape. ix Let P be the x-coordinate of the point. iy Let y be the coordinate of the point; b. Calculate the set d of distances from each point on the shape to the center: Where C is the set of all points in the shape; c. Calculate the mean and standard deviation (std) of the set of distances from each point on the shape to the center: d. Obtain the roundness result c: e. If the roundness result is greater than the threshold, it is identified as a hole to be tested; B. Based on the outline shape of the hole to be tested found in the previous step, find the center and diameter Fx, Fy, Fd of the fitted circle, the center and diameter Ix, Iy, Id of the largest inscribed circle, and the center and diameter Zx, Zy, Zd of the smallest circumscribed circle. a. Fitted circle: Using the center and diameter as variables, a fitted circle is obtained by fitting the circle using the least squares method; b. Maximum incircle: Traverse all points inside the contour and calculate the distance from these points to the contour. The point inside the contour that is farthest from the contour is the center of the circle, and the maximum distance between it and the contour is the radius. This is how to obtain the maximum incircle. c. Minimum circumcircle: Find one point on each of the top, bottom, left, and right edges of the contour and calculate its circumcircle. If a point in the contour is outside this circumcircle, then introduce the point outside the circumcircle that is farthest from the circumcircle and calculate a new circumcircle. Continue this process until all points in the contour are inside the calculated circumcircle, thus obtaining the minimum circumcircle. C. Store the data obtained in the previous step, and use it together with the actual aperture of the hole to be measured as a training set. Use a deep neural network to train the model and obtain the weight parameter model w. D. After repeating the training at least 10 times, measure the hole with the unknown diameter and find the center: Where w1, w2 … w 10 The result of the model w for the weight parameters corresponding to each input item; E. Hole Quality Grading Assessment: The images of holes captured by the detection equipment and their corresponding hole types are used as the training set. A convolutional neural network is then used to train the model. The trained model is then stored and used to grade and assess hole quality during the detection process.

7. The aperture, hole spacing, and hole position detection device according to claim 6, characterized in that, The motion component (101) is a robot or other automated motion device capable of carrying the detection device (102) and moving along a predetermined path.

8. The aperture, hole spacing, and hole position detection device according to claim 6, characterized in that, The detection device (102) consists of a first camera, a first lens, a second camera, a second lens, and a light source; The detection device (102) is connected to the motion component (101) through a mechanical transfer structure, and is carried by the motion component (101) to move and take pictures according to a pre-set detection path; The motion component (101) and the detection device (102) are connected to the host computer (104) fixed in the control box (103) via a network cable and interact with each other. The host computer (104) has a built-in control system, a receiving system and a software processing system. The control system is used to control the movement of the motion component (101) and the image acquisition process of the detection device (102). The receiving system is used to receive the position information of the motion component (101) and the image information transmitted back by the detection device (102). The software processing system is used to preprocess, analyze, calculate and evaluate the image transmitted back by the detection device (102) and provide feedback on the processing results.

Images