Numerical control drilling and milling machine working state detection method based on machine vision

Abstract

According to the method, the working state of the numerical control drilling and milling machine is evaluated by judging the grinding condition of the surface of the workpiece machined during working of the numerical control drilling and milling machine, the gradient distribution histogram of each pixel point corresponding to the surface of the workpiece is obtained from the gradient direction features of the pixel points, and the gradient feature matrix is obtained according to the gradient distribution histogram; obtaining a feature extraction result of the feature of the morphological representation at the position of each pixel point; region division is conducted on the image according to the similarity between the gradient feature matrixes extracted by all the pixel points, all damage positions of the surface of the workpiece are obtained, and the damage positions comprise normal grinding regions and suspected abnormal regions; and then the damage position is evaluated according to the direction consistency and linear morphological characteristics of damage caused by normal wear, whether abnormal damage exists or not is judged according to the evaluation result, and the work of the numerical control drilling and milling machine is judged, so that the judgment is more accurate.

Description

Technical Field

[0001] This application relates to the technical field of image data processing, and specifically to a method for detecting the working state of a numerically controlled drilling and milling machine based on machine vision. Background Technique

[0002] To meet the increasing precision requirements of equipment, people have higher and higher requirements for the machining precision of mechanical parts. Ordinary machine tools or highly specialized automated machine tools can no longer meet these requirements, and numerically controlled machine tools that comprehensively apply technical achievements in many aspects such as electronic computers, automatic control, precision measurement, and new mechanical structures have emerged as the times require. During the use of a numerically controlled drilling and milling machine, the shape of the target part is obtained by cutting and friction of the milling cutter on the metal material. Therefore, the numerically controlled drilling and milling machine has the advantages of strong adaptability, high machining precision, stable machining quality, and high production efficiency. However, if the milling cutter wears during use, it will damage the metal parts during the machining process, resulting in insufficient machining precision and affecting the subsequent use of the parts. Therefore, usually, the machining quality of the part surface is detected after the part machining is completed to detect the working state of the numerically controlled drilling and milling machine. The existing methods for detecting the working state of a numerically controlled drilling and milling machine generally use neural networks, and the detection effect is too dependent on a large number of training sets and manually labeled tags. There is a need for a method that can detect the working state of a numerically controlled drilling and milling machine according to the state of the part surface without relying on a large number of manual labeling requirements. Summary of the Invention

[0003] To solve the above problems, the present invention proposes a method for detecting the working state of a numerically controlled drilling and milling machine based on machine vision. The method includes the following steps:

[0004] Step S1: Collect the images after each process of the milling cutter processing the workpiece, record the obtained images as workpiece surface images, grayscale the workpiece surface images, and perform denoising;

[0005] Step S2: Use the GrabCut algorithm to segment the workpiece surface image obtained in Step S1, divide the area corresponding to the workpiece surface processed in the previous process in the image, and record the grayscale image that only contains the divided area as the grinding processing image;

[0006] Step S3: Use superpixel segmentation to divide the grinding processing image into n regions, record each region as a grinding trace region, and each region corresponds to a part of the workpiece surface with relatively uniform texture. Determine whether each grinding trace region belongs to a normal grinding region or a suspected tool mark region;

[0007] Step S4: Calculate the tool mark deviation degree for each suspected tool mark area determined in Step S3, and use LOF outlier detection for the above tool mark deviation degree. When an outlier is detected, the suspected tool mark area corresponding to the outlier is the damage position on the workpiece surface, and damage occurs in the corresponding grinding mark area; when no outlier is detected, no abnormal damage occurs in the corresponding grinding mark area.

[0008] Step S5: When there is damage in a certain grinding mark area in the workpiece surface image, it is considered that the working state of the CNC drill milling machine is abnormal, and at the same time, output the suspected tool mark at the damage position on the corresponding workpiece surface; when no outlier is detected in each grinding mark area in the image corresponding to the workpiece, it is considered that the working state of the CNC drill milling machine is good.

[0009] Preferably, in Step S3, to determine whether each grinding mark area belongs to a normal grinding area or a suspected tool mark area, the specific operation is as follows:

[0010] (1) Obtain the gradient vector corresponding to each pixel point in the grinding mark area according to the gray value of each pixel point in the grinding mark area. The direction value range of the gradient vector is There are a total of nine radian value ranges. Establish an a*a window with each pixel point as the center, and obtain the gradient direction distribution histogram according to the gradient vectors corresponding to each pixel point in the window.

[0011] (2) Denote the gradient direction included in the range with the largest frequency in the gradient direction distribution histogram corresponding to each pixel point as the first direction α1 corresponding to the pixel point, and denote the frequency of the first direction α1 as the first direction frequency Denote the gradient direction included in the range with the second largest frequency as the second direction α2 corresponding to the pixel point, and denote the frequency of the second direction α2 as the second direction frequency Similarly, obtain the third direction to the ninth direction and the third direction frequency to the ninth direction frequency. At the same time, denote the average value of the modulus length of the gradient vector corresponding to the i-th direction a i as the i-th direction intensity where i = 1, 2,..., 9;

[0012] (3) According to the various indicators obtained from the gradient direction distribution histogram corresponding to each pixel point, obtain the gradient feature matrix of each pixel point:

[0013]

[0014] In the formula, A (x,y) is the gradient feature matrix corresponding to the pixel point with coordinates (x, y) in the image;

[0015] (4) Obtain the gradient difference between each pixel point and other pixel points according to the gradient feature matrix of each pixel point. Use the DBSCAN algorithm for each gradient difference corresponding to each pixel point, with 4 as the minimum number of points and 7 as the radius for clustering to obtain multiple clusters. Calculate the number of times the pixel point is divided into each cluster, and record the cluster with the most division times as the cluster corresponding to the pixel point. Thus, the cluster corresponding to each pixel point within the grinding mark area can be determined;

[0016] (5) Generally, there are multiple tool marks within the grinding mark area. The pixel points corresponding to different tool marks will be divided into the same cluster. Obtain the positions of each pixel point included in each cluster, establish multiple binary images, and each binary image only contains the pixel points within the same cluster. Perform connected component analysis on each binary image to obtain multiple connected components. The connected component containing the most pixel points corresponds to the workpiece surface position, that is, the normal grinding area, and record the areas composed of the pixel points included in each of the other connected components as suspected tool mark areas.

[0017] Preferably, the calculation method of the tool mark deviation degree of each suspected tool mark area in step S4 is as follows:

[0018] Use the least squares method to fit a straight line to each pixel point included in each suspected tool mark area to obtain the corresponding goodness of fit. The goodness of fit is the similarity degree between the distribution and trend of the area and the straight line. The calculation formula for the tool mark deviation degree of each suspected tool mark area is as follows:

[0019]

[0020] In the formula, k is the tool mark deviation degree corresponding to the suspected tool mark area; is the mean value of the gradient differences between the l-th pixel point and other pixel points within the suspected tool mark area, where l = 1, 2,..., n1; n1 is the number of pixel points included in the suspected tool mark area; is the mean value of the corresponding for each pixel point within the suspected tool mark area; r l is the goodness of fit of the suspected tool mark area; is the mean value of the goodness of fit of each suspected tool mark area belonging to the same cluster.

[0021] The technical solution of the present invention can achieve the following beneficial effects: The present invention evaluates the working state of a numerically controlled drilling and milling machine by judging the grinding condition of the surface of the workpiece during the work of the numerically controlled drilling and milling machine. Starting from the gradient direction feature of pixel points, the gradient distribution histogram of each pixel point corresponding to the workpiece surface is obtained, and the gradient feature matrix is obtained according to the gradient distribution histogram, and the feature extraction result of the feature that highlights the morphological characterization at the position of each pixel point is obtained; Secondly, the image is regionally divided according to the similarity between the gradient feature matrices extracted from each pixel point, and each damage position on the workpiece surface is obtained. These damage positions include normal grinding areas and suspected abnormal areas; Then, the damage positions are evaluated according to the direction consistency and linear morphological characteristics of the damage caused by normal wear, and whether there is abnormal damage is judged according to the evaluation result, and the work of the numerically controlled drilling and milling machine is judged, making the judgment more accurate. Description of the Drawings

[0022] Figure 1 It is a grayscale image of the workpiece surface after being processed by milling cutters with different roughnesses. Detailed Embodiments

[0023] In order to enable those skilled in the art to better understand the present invention, the present invention will be described in conjunction with embodiments below.

[0024] Embodiment 1:

[0025] The present invention provides a method for detecting the working state of a numerically controlled drilling and milling machine based on machine vision, including the following steps:

[0026] Step S1: Collect the images after each process of the milling cutter processing the workpiece, record the obtained images as workpiece surface images, grayscale the workpiece surface images, and perform denoising. The specific operations are as follows:

[0027] An industrial camera is set at the end of the robotic arm that can move freely on the numerically controlled drilling and milling machine. The camera uses a microscopic lens, and a circular LED light is installed around the upper end of the microscopic lens. The function of the LED light is to supplement light to the workpiece during the shooting process to achieve the best lighting effect and improve the image quality. After each process of the milling cutter processing the workpiece, the industrial camera is used to shoot the image of the processed surface of the workpiece, and the obtained image is recorded as the workpiece surface image. The workpiece surface image is an RGB image. Since the workpiece is generally made of the same metal material, the color feature differences at different positions are not obvious after cutting and friction, so the analysis from the perspective of color features is not the focus of this solution in the subsequent analysis. Therefore, the workpiece surface image is converted into a grayscale image, denoted as the workpiece surface grayscale image. The workpiece surface grayscale image is denoised using Gaussian filtering. Gaussian filtering denoising is a well-known technology and will not be elaborated here.

[0028] Step S2: Segment the workpiece surface image obtained in Step S1 using the GrabCut algorithm to divide the area corresponding to the workpiece surface processed in the previous process in the image, and record the grayscale image that only contains the divided area as the grinding processing image. The specific operations are as follows:

[0029] The numerically controlled drilling and milling machine is mainly used to process workpieces with complex shapes. Therefore, the workpiece surface grayscale image may contain the workpiece surface processed in the previous process and the workpiece surface not processed in the previous process, and the image needs to be divided. Segment the workpiece surface image obtained in Step S1 using the GrabCut algorithm to divide the area corresponding to the workpiece surface processed in the previous process in the image, and record the grayscale image that only contains the divided area as the grinding processing image.

[0030] Step S3: Use superpixel segmentation to divide the grinding processing image into n regions, record each region as a grinding mark region, and each region corresponds to a part of the workpiece surface with relatively uniform texture. Determine whether each grinding mark region belongs to the normal grinding region or the suspected tool mark region. The specific operations are as follows:

[0031] The grinding processing image shows the condition of the workpiece surface after being ground in the previous process. The grayscale images of the workpiece surface after being processed with milling cutters of different roughness levels are also different. Regular repeated marks appear along the grinding direction of the milling cutter, as shown in the appendix Figure 1 as shown.

[0032] Use superpixel segmentation to divide the grinding processing image into n regions, and each region corresponds to a part of the workpiece surface with relatively uniform texture. Among them, the empirical value of n is 50. For the convenience of description, each region is recorded as a grinding mark region.

[0033] (1) Analyze each grinding mark region separately. Obtain the gradient vector corresponding to each pixel point in the grinding mark region according to the grayscale value of the pixel point. The range of the direction corresponding to the gradient vector is A total of nine radian values are taken. Establish an a*a window with each pixel point as the center, and obtain the gradient direction distribution histogram according to the gradient vectors corresponding to the pixel points in the window. Among them, the side length a of the window is an odd number, and the empirical value is 9. The horizontal axis of the gradient distribution histogram is the range of the direction corresponding to the gradient vector. To ensure that the direction of each gradient vector is within a range, the respective ranges corresponding to the horizontal axis are The vertical axis of the gradient distribution histogram is the frequency corresponding to the direction of each gradient vector within a certain range.

[0034] Record the gradient direction included in the range with the largest frequency in the gradient direction distribution histogram corresponding to each pixel point as the first direction α1 corresponding to the pixel point, and record the frequency of obtaining the first direction α1 as the first direction frequency The gradient direction included in the range with the second-highest frequency in the histogram of gradient direction distributions corresponding to each pixel is denoted as the second direction α2 corresponding to that pixel, and the frequency of obtaining the second direction α2 is denoted as the second direction frequency. Similarly, the third to ninth directions and the third to ninth direction frequencies are obtained. At the same time, the mean value of the modulus of the gradient vector corresponding to the i-th direction α i is denoted as the i-th direction intensity. where i = 1, 2,..., 9.

[0035] It should be noted that when the frequencies are the same, the judgment is made based on the distance from the range to the origin. For example, and the frequencies corresponding to both directions are 5. It is necessary to determine which range corresponds to the fourth and fifth directions. Since is closer to the origin, it is considered that is the fourth direction, and corresponds to the fifth direction.

[0036] Based on the various indicators obtained from the histogram of gradient direction distributions corresponding to each pixel, the gradient feature matrix of each pixel is obtained.

[0037]

[0038] In the formula, A (x，y) is the gradient feature matrix corresponding to the pixel with coordinates (x, y) in the image.

[0039] (2) Obtain the gradient difference between each pixel and other pixels based on the gradient feature matrix of each pixel.

[0040]

[0041] In the formula, (x1, y1) and (x2, y2) are the coordinates of pixels in the grinding mark area; is the gradient difference between the pixel with coordinates (x1, y1) and the pixel with coordinates (x2, y2) in the grinding mark area; (a1 i , b1 j ) is the value at the position of (a1 i , b1 j ) in the gradient feature matrix of the pixel with coordinates (x1, y1) in the grinding mark area. For example, in the gradient feature matrix A is the value at the position of (2, 3). When i = 2 and j = 3, (a1 i , b1 j ) corresponds to the value at the position of (2, 3); (a2 i , b2 j) is the value at the position of (a1 i , b1 j ) in the gradient feature matrix of the pixel at coordinates (x2, y2) in the image; where i = 1, 2,..., 9; j = 1, 2, 3. That is, it is the sum of the Euclidean distances between the values at the corresponding positions in the gradient feature matrix corresponding to the pixel at coordinates (x1, y1) and the gradient feature matrix corresponding to the pixel at coordinates (x2, y2) in the grinding mark area.

[0042] So far, a corresponding gradient difference can be obtained between every two pixels in the image. When the gradient distribution conditions of the two pixels are closer, the gradient distribution histograms and gradient feature matrices corresponding to the two pixels are closer, and the gradient difference is smaller.

[0043] (3) There are two types of pixels in the grinding mark area. One type corresponds to the sunken position below the workpiece surface formed after milling cutter grinding, that is, the tool mark, and the pixels corresponding to this position are denoted as tool mark pixels; the other type corresponds to the workpiece surface position, and the pixels corresponding to this position are denoted as surface pixels.

[0044] Use the DBSCAN algorithm for each gradient difference value corresponding to each pixel, with 4 as the minimum number of points and 7 as the radius for clustering to obtain multiple clusters. Since each pixel in the grinding mark area has a corresponding gradient difference value with other pixels, when clustering the gradient difference values corresponding to other pixels, the pixel may be divided into different clusters. Calculate the number of times the pixel is divided into each cluster, and denote the cluster with the most divided times as the cluster corresponding to the pixel. So far, the cluster corresponding to each pixel in the grinding mark area can be determined.

[0045] There are generally multiple tool marks in the grinding mark area. The depths of different tool marks are the same and parallel to each other. Therefore, the tool mark pixels corresponding to different tool marks will be divided into the same cluster. Obtain the positions of each pixel included in each cluster, establish multiple binary images, and each binary image only contains the pixels in the same cluster. Perform connected component analysis on each binary image to obtain multiple connected components. Since the workpiece tool marks are located at multiple different positions on the workpiece, corresponding to multiple regions, and the number of pixels included in each region is small, while the surface pixels included in the workpiece surface form a larger region, and the number of pixels included is significantly more than the number of pixels in the regions corresponding to each tool mark on the workpiece. Therefore, the connected component with the largest number of pixels corresponds to the workpiece surface position. The pixels in this connected component are surface pixels, and this connected component is the normal grinding area. Denote the regions composed of the pixels included in other connected components as suspected tool mark areas.

[0046] Step S4: Calculate the tool mark deviation degree for each suspected tool mark area determined in Step S3, and use LOF outlier detection for the above tool mark deviation degrees. When an outlier is detected, the suspected tool mark area corresponding to the outlier is the damage position on the workpiece surface, and damage occurs within the corresponding grinding mark area; when no outlier is detected, it means that no abnormal damage occurs in the corresponding grinding mark area. The specific operations are as follows:

[0047] For each pixel point contained in each suspected tool mark area, use the least squares method to fit a straight line to obtain the corresponding goodness of fit. The goodness of fit is the degree of similarity between the distribution and trend of this area and the straight line. The greater the degree of similarity, the greater the goodness of fit, and the value range is [0,1].

[0048]

[0049] In the formula, k is the tool mark deviation degree corresponding to the suspected tool mark area; is the mean value of the gradient differences between the l-th pixel point and other pixel points in this suspected tool mark area, where l = 1, 2,..., n1; n1 is the number of pixel points contained in this suspected tool mark area; is the corresponding mean value of; r l is the goodness of fit of this suspected tool mark area; is the mean value of the goodness of fit of each suspected tool mark area belonging to the same cluster.

[0050] reflects the degree of difference between pixel points in the suspected tool mark area; reflects the direction consistency between the suspected tool mark area and each suspected tool mark area within the same cluster; when the two are smaller, the significance of the tool mark position corresponding to this suspected tool mark area is greater, and each pixel point within the area is more likely to be a tool mark pixel point. When there is a problem with the working state of the CNC drilling and milling machine, the milling cutter will cause a small amount of damage to the workpiece. This damage does not have the characteristics of a tool mark area with strong consistency and a straight-line distribution, and the corresponding tool mark deviation degree is relatively large. Thus, multiple suspected tool mark areas and the corresponding tool mark deviation degrees of each suspected tool mark area are obtained in each grinding mark area.

[0051] When all pixel points in the suspected tool mark area correspond to tool mark pixel points, the tool mark deviation degrees corresponding to each suspected tool mark area are relatively small and the values are relatively close; when the suspected tool mark area contains damages caused by problems in the working state of the CNC drilling and milling machine, the tool mark deviation degree of the suspected tool mark area corresponding to the damage position is large, and there is a large difference from the tool mark deviation degree of the suspected tool mark area where the tool mark pixel points are located. The LOF outlier detection is used for each tool mark deviation degree. When an outlier is detected, the suspected tool mark area corresponding to the outlier corresponds to the damage position on the workpiece surface, and damages occur in the grinding mark area; when no outlier is detected, that is, there are no abnormal damages on the workpiece surface corresponding to the grinding mark area.

[0052] Step S5: When there are damages in a certain grinding mark area in the workpiece surface image, it is considered that the working state of the CNC drilling and milling machine is abnormal, and at the same time, the suspected tool marks at the damage positions on the corresponding workpiece surface are output; when no outliers are detected in each grinding mark area in the image corresponding to the workpiece, it is considered that the working state of the CNC drilling and milling machine is good.

[0053] Obviously, the above embodiments are only examples clearly described and not limitations on the implementation manners. For those of ordinary skill in the art, other different forms of changes or modifications can be made based on the above description. It is not necessary and impossible to list all implementation manners here. And the obvious changes or modifications derived therefrom are still within the protection scope of the present invention.

Claims

1. A method for detecting the working state of a numerically controlled drilling and milling machine based on machine vision, characterized in that, Including the following steps: Step S1: Collect the images after each process of the milling cutter processing the workpiece, record the obtained images as workpiece surface images, grayscale the workpiece surface images, and perform denoising. Step S2: Segment the workpiece surface image obtained in Step S1 using the GrabCut algorithm, divide the area corresponding to the workpiece surface processed in the previous process in the image, and record the grayscale image containing only the divided area as the grinding processing image. Step S3: Use superpixel segmentation to divide the grinding processing image into n regions, record each region as a grinding trace region, and each region corresponds to a part of the workpiece surface with relatively uniform texture. Determine whether each grinding trace region belongs to the normal grinding region or the suspected tool mark region. Step S4: Calculate the tool mark deviation degree of each suspected tool mark region determined in Step S3, use the LOF outlier detection for the above tool mark deviation degree. When an outlier is detected, the suspected tool mark region corresponding to the outlier is the damage position on the workpiece surface, and damage occurs in the corresponding grinding trace region. When no outlier is detected, no abnormal damage occurs in the corresponding grinding trace region. Step S5: When damage occurs in a certain grinding trace region in the workpiece surface image, it is considered that the working state of the CNC drill milling machine is abnormal, and at the same time, output the suspected tool marks at the damage position on the corresponding workpiece surface. When no outlier is detected in each grinding trace region in the image corresponding to the workpiece, it is considered that the working state of the CNC drill milling machine is good.

2. The method for detecting the working state of a numerically controlled drilling and milling machine based on machine vision according to claim 1, characterized in that, In Step S3, to determine whether each grinding trace region belongs to the normal grinding region or the suspected tool mark region, the specific operation is as follows: (1) Obtain the gradient vector corresponding to each pixel point in the grinding mark area according to the gray value of each pixel point in the grinding mark area. The direction value range corresponding to the gradient vector is There are a total of nine radian value selections. An a*a window is established with each pixel point as the center, and a gradient direction distribution histogram is obtained according to the gradient vectors corresponding to each pixel point in the window; (2) Denote the gradient direction included in the range with the largest frequency in the histogram of gradient direction distributions corresponding to each pixel as the first direction α1 corresponding to this pixel, and denote the frequency of the first direction α1 as the first direction frequency. Denote the gradient direction included in the range with the second largest frequency as the second direction α2 corresponding to this pixel, and denote the frequency of the second direction α2 as the second direction frequency. Similarly, obtain the third direction to the ninth direction and the third direction frequency to the ninth direction frequency. At the same time, denote the mean value of the modulus lengths of the gradient vectors corresponding to the i-th direction α i as the i-th direction intensity. where i = 1, 2,..., 9; (3) Obtain the gradient feature matrix of each pixel point according to the indicators obtained from the gradient direction distribution histogram corresponding to each pixel point. where A (x，y) is the gradient feature matrix corresponding to the pixel point with coordinates (x, y) in the image; (4) Obtain the gradient difference between each pixel point and other pixel points according to the gradient feature matrix of each pixel point. Use the DBSCAN algorithm for each gradient difference corresponding to each pixel point, with 4 as the minimum number of points and 7 as the radius for clustering to obtain multiple clusters. Calculate the number of times the pixel point is divided into each cluster, and record the cluster with the most divided times as the cluster corresponding to the pixel point. Thus, the cluster corresponding to each pixel point in the grinding trace region can be determined. (5) Generally, there are multiple tool marks in the grinding trace region. The tool mark pixel points corresponding to different tool marks will be divided into the same cluster. Obtain the positions of each pixel point included in each cluster, establish multiple binary images, and each binary image only contains the pixel points in the same cluster. Perform connected component analysis on each binary image to obtain multiple connected components. The connected component with the largest number of pixel points corresponds to the workpiece surface position, that is, the normal grinding region, and record the regions composed of the pixel points included in other connected components as suspected tool mark regions.

3. The method for detecting the working state of a numerically controlled drilling and milling machine based on machine vision according to claim 2, wherein, In Step S4, the calculation method of the tool mark deviation degree of each suspected tool mark region is as follows: Use the least squares method to fit each pixel point included in each suspected tool mark region with a straight line to obtain the corresponding goodness of fit. The goodness of fit is the similarity between the distribution and trend of the region and the straight line. The calculation formula of the tool mark deviation degree of each suspected tool mark region is as follows: Where k is the tool mark deviation corresponding to the suspected tool mark area; is the mean of the gradient differences between the l-th pixel point and other pixel points in the suspected tool mark area, where l = 1, 2,..., n1; n1 is the number of pixel points included in the suspected tool mark area; corresponds to each pixel point in the suspected tool mark area mean value of; r l is the goodness of fit of the suspected tool mark area; is the mean of the goodness of fit of each suspected tool mark area belonging to the same cluster.

Images