Microstructure parameter measurement method based on UNet network

Abstract

The invention relates to the technical field of precise instrument manufacturing and precise testing and metering, in particular to a microstructure parameter measuring method based on a UNet network. Firstly, aiming at a large number of prominent textures and scratches existing in an image background region of a to-be-detected piece, a region segmentation method based on a UNet network is adopted to improve the segmentation precision, and the influence of noise is overcome. Then a data set is collected and image enhancement is carried out so as to enhance the generalization ability of the model, the attitude of the microstructure is enriched through horizontal overturning and vertical overturning, and the robustness of the model to different illumination environments is enhanced through HSV color gamut random change. After a target area is obtained through segmentation, edge point extraction is carried out, then fitting is carried out, and outliers need to be removed before fitting due to the existence of machining defects. The outlier elimination mode for the microporous structure is as follows: edge sequence points are connected, so that convex parts are cut off, and finally, an area with the largest connected domain is taken as a repaired feature area; the outlier elimination mode for the micro-groove structure is as follows: performing multiple iteration elimination by using an RANSAC (Random Sample Consensus) algorithm, screening out all interior points meeting a'correct solution ', and fitting by taking data conforming to optimal fitting as input of a least square method, so that model parameters can be better estimated. According to the invention, the visual measurement precision of micro-pores and micro-grooves from micron level to millimeter level in the workpiece can be improved.

Description

Technical Field

[0001] The present invention mainly measures the parameters of micropores and microgrooves ranging from hundreds of micrometers to millimeters, integrates regional segmentation based on deep learning and geometric element fitting based on outlier elimination, and belongs to the field of precision instrument manufacturing and precision testing and measurement technology. Background Art

[0002] Arrayed microstructures ranging from hundreds of micrometers to millimeters have a wide range of applications and broad development prospects in the field of high-end equipment manufacturing. Microholes and microgrooves are particularly widely used in the aerospace field, especially the size and geometric characteristics of the microstructure have an important impact on the performance of the device; in addition, the development of ultra-precision measurement technology also restricts the development of precision ultra-precision machining technology. For this type of microstructure, visual measurement methods are widely used. The main steps are to first segment the area, then extract the edge, and finally fit and calculate the geometric parameters.

[0003] Among them, image segmentation is a more critical step. The earliest developed image segmentation algorithm is mainly based on the underlying features of the image, which include contours, edges, colors, textures, and shapes. The main representative algorithms are threshold-based image segmentation and region-based image segmentation. The above method is mainly based on the pixels of the image for segmentation, and the feature vector of the image can also be used. Commonly used methods include image segmentation based on clustering algorithms and image segmentation based on classifiers. When the prior knowledge of the image is known, the image can be divided into regions that meet the shape prior using template matching, deformation models, etc. according to the known or assumed target shape. This type of method is called a shape prior-based method. In addition, with the development of various disciplines, image segmentation algorithms based on wavelet transform, image segmentation algorithms based on graph theory, and segmentation methods based on energy functionals have emerged. However, the above traditional methods cannot take into account the underlying features and semantic information of the image at the same time, and it is easy to mis-segment the areas where the underlying features in the picture are close to the target. At the same time, during the processing process, the tool will leave scratches on the workpiece surface, presenting uneven morphology under a high-power magnification lens, and direct fitting will lead to inaccurate measurement results. In response to the above problems, the present invention provides a microstructure parameter measurement method based on a UNet network. Summary of the invention

[0004] The purpose of the present invention is to provide a microstructure parameter measurement method based on the UNet network, which solves the problem of inaccurate segmentation caused by a large number of protruding textures and scratches in the background area under a high-definition lens, and the problem that the extracted edges are not standard geometric figures due to morphological defects caused by the processing process.

[0005] To achieve the above effect, the technical solution provided by the present invention is as follows: First, since there are a large number of prominent textures and scratches in the image background area of ​​the test piece, it is necessary to segment the target area first. The present invention adopts a regional segmentation method based on the UNet network, collects data sets and enriches the microstructure posture through horizontal flipping and vertical flipping; through random changes in the HSV color gamut, the robustness of the model to different lighting environments is enhanced, and the enhanced data set is trained to obtain a segmentation model with stronger generalization ability. After segmenting the target area, edge point extraction is performed. The outlier removal method for the microporous structure is: connect the edge sequence points to cut off the protruding part, and finally take the area with the largest connected domain as the feature area after repair; the outlier removal method for the microgroove structure is: first use the RANSAC algorithm for multiple iterations to remove, screen out all the internal points that meet the "correct solution", and then use the data that meets the best fit as the input of the least squares method for fitting, which can better estimate the model parameters.

[0006] The microstructure parameter measurement method based on UNet network includes the following steps:

[0007] (1) Equip the camera with a ring light to collect the data set;

[0008] (2) Data enhancement is performed on the collected data set, mainly by enriching the postures of microholes and microgrooves through horizontal and vertical flipping; the robustness of the model to different lighting environments is enhanced by randomly changing the HSV color gamut.

[0009] (3) The data set is divided into training set and test set. The semantic segmentation and annotation software Labelme is used to annotate the images and separate the background and the target. The target is divided into two categories: circle and rectangle.

[0010] (4) Build the UNet network model, adjust the network parameters, and determine the parameters in the model such as loss function, optimizer, and learning rate.

[0011] (5) Train the network model in the configured hardware and software environment.

[0012] (6) Calculate the evaluation indicators and visualize the calculation results to evaluate the model.

[0013] (7) Use the trained model to segment the object to be tested, distinguish the background and target areas, and then extract the edge points.

[0014] (8) If the object to be tested is a microhole, connect the edge sequence points to remove the protruding parts, and finally take the area with the largest connected domain as the feature area after repair. After repair, use the least squares method to calculate the geometric parameters; if the object to be tested is a microgroove, first use the RANSAC algorithm to perform multiple iterations to eliminate all the internal points that meet the "correct solution", and then use the data that meets the best fit as the input of the least squares method for fitting, which can better estimate the model parameters.

[0015] (9) Calculate the actual size parameters based on the camera calibration results.

[0016] Compared with the prior art, the present invention proposes a microstructure parameter measurement method based on the UNet network, which has the following beneficial effects:

[0017] (1) The UNet-based region segmentation method makes full use of the underlying information and semantic information of the image, can overcome the impact of scratches and background noise around the workpiece to be tested, and will not be disturbed by non-target elements, and can completely segment the target elements. After adding two data enhancement methods, the trained model has better robustness.

[0018] (2) Methods for eliminating outliers are designed for the fitting of ellipses and lines respectively. After elimination, the least squares method is used to fit and obtain visual measurement parameters. This can overcome the problem of inaccurate fitting caused by processing defects and improve measurement accuracy.

[0019] The method proposed in the present invention is suitable for non-contact measurement of micropores and microgrooves in the range of hundreds of micrometers to millimeters. Compared with traditional methods, it has higher robustness and measurement accuracy, and can be applied in the fields of aerospace and precision manufacturing. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 This is a flow chart of the measurement method of microstructure parameters based on the UNet network.

[0021] Figure 2 This is a flowchart of random transformation of HSV color space

[0022] Figure 3 This is the model diagram of the UNet network

[0023] Figure 4 This is the segmentation effect diagram of micropores and microgrooves

[0024] Figure 5 This is a schematic diagram of defect ellipse outlier removal DETAILED DESCRIPTION

[0025] The technical solutions in the embodiments of the present invention will be described clearly and completely below in conjunction with the accompanying drawings in the embodiments of the present invention:

[0026] The measuring principle and process of the present invention are:

[0027] (1) Add a ring light to the industrial camera, adjust it to a suitable focal length, and collect the data set;

[0028] (2) Data enhancement is performed on the collected data set. The postures of micro-holes and micro-grooves are mainly enriched by horizontal flipping and vertical flipping; the robustness of the model to different lighting environments is enhanced by random changes in the HSV color domain. The process of random transformation of the HSV color domain is as follows: Figure 2 As shown, the hue is transformed by adding random perturbations to the three channels H, S, and V in the HSV color space.

[0029] (3) The data set is divided into a training set and a test set. The images are annotated using the semantic segmentation and annotation software Labelme to separate the background and the target. The target is divided into two categories: circle and rectangle.

[0030] (4) The present invention selects PyTorch as a deep learning framework to build the UNet network structure. The UNet network structure is as follows Figure 3 As shown, the overall structure consists of a contraction path on the left and an expansion path on the right. The contraction path follows the typical architecture of convolutional networks and consists of two 3×3 convolutional layers (ReLU) and a 2×2 max pooling operation with a stride of 2 for downsampling, doubling the number of feature channels in each downsampling step. Each step in the expansion path consists of upsampling of the feature map, but with the number of feature channels halved, a concatenation with the corresponding cropped feature map in the contraction path, and two 3×3 convolutional layers (ReLU). In the last layer, a 1×1 convolution is used to map each 64-component feature vector to the desired number of classes.

[0031] (4) When training the network, the batch size is set to 2 and the initial learning rate is set to 1×10 -6 The stochastic gradient descent algorithm selects the gradient descent with momentum, the momentum parameter is set to 0.999, the gradient clipping method is used to prevent the gradient explosion, the norm is specified to 1, and the number of epochs is set to 50.

[0032] (5) Calculate the evaluation index and visualize the index calculation results to evaluate the model. When evaluating the regional segmentation effect, the present invention uses the Dice coefficient for evaluation. The Dice coefficient is a set similarity measurement function used to calculate the similarity of two samples, and its value range is [0,1]. Its expression is shown in formula (1).

[0033]

[0034] In the formula, X represents the actual result and Y represents the predicted result.

[0035] (6) Use the trained model to segment the object to be tested, distinguish the background and target areas, and then extract the edge points.

[0036] (7) If the object to be tested is a micropore, connect the edge sequence points to remove the protruding parts, and finally take the area with the largest connected domain as the feature area after repair. After repair, the least squares method is used to calculate the geometric parameters. The specific repair schematic diagram is shown in the figure. Figure 5 As shown in the figure, first obtain the ordered edge sequence of the feature map to be repaired, and then traverse the edge sequence, and let the traversed edge point be A(x a ,y a ), its related point B(x b ,y b ) is the edge point whose serial number follows it and satisfies the conditions:

[0037]

[0038] Where r is the repair radius.

[0039] The edge repair algorithm is:

[0040]

[0041]

[0042] In the formula, (x c ,y c ) represents a point on the line segment formed by points A and B.

[0043] After the defect ellipse is repaired by removing outliers for many times, a smooth curve can be obtained. This curve can be used as a target for approximating a circle, and the circularity can be used to evaluate the repair effect. The circularity expression is shown in formula (5).

[0044]

[0045] Where S represents the area of ​​the figure after repair, and C represents the perimeter of the figure after repair.

[0046] After the repair is completed, the image is morphologically expanded, the intersection and matching of the image edge points and the original image edge points after each expansion operation are tracked, and the change law of the number of edge point matches is counted. When the number of intersecting edge points is greater than the matching rate threshold, the expansion is stopped, and the largest intersection point increment is recorded as the valid edge point. After the defect ellipse edge is repaired according to the above method, the least squares fitting is performed to obtain the parameters of the ellipse in the pixel coordinate system.

[0047] If the micro-grooves of the object to be tested are first eliminated by multiple iterations of the RANSAC algorithm to screen out all the internal points that meet the "correct solution", and then the data that meets the best fit is used as the input of the least squares method for fitting, which can better estimate the model parameters. Assuming that the probability of the internal point in the entire data set is t, then t can be expressed as:

[0048]

[0049] Where n in Indicates the number of interior points, n out Represents the number of external points.

[0050] According to the knowledge of probability, if n points are randomly selected, the probability that these n points are all interior points is t n , the probability of having at least one outlier is 1-t n , then in k random samplings, the probability that each sampling contains at least one outlier is (1-t n ) k , and the correct solution requires that the points obtained by each sampling are all interior points. From this, it can be inferred that the probability expression for obtaining the correct solution is shown in formula (7).

[0051] P=1-(1-t n ) k (7)

[0052] In the process of iterative calculation, the larger the P value is, the better the model effect is. According to formula (7), the size of the P value depends on the internal point probability n and the number of iterations k. Even when n is large, the probability P of obtaining the correct solution can be guaranteed by increasing the number of iterations k. The calculation formula of k can be derived from formula (7):

[0053]

[0054] The above method can be used to screen out the data points that meet the best fit, and then the least squares fitting is performed on them to obtain the parameters of the straight line in the pixel coordinate system.

[0055] (8) Calculate the actual size parameters based on the camera calibration results.

[0056] The above is only a preferred embodiment of the present application, and the present invention is not limited to the above embodiments. It can be understood that any changes made according to the technical solution of the present invention, as long as the resulting functions do not exceed the scope of the technical solution of the present invention, belong to the protection scope of the present invention.

Claims

1. A method for measuring microstructure parameters based on a UNet network, comprising the following steps: (1) A ring light is added to the camera to collect data sets of microholes and microgrooves. (2) Perform data enhancement on the collected data set. (3) The data set is divided into a training set and a test set. The images are annotated using the semantic segmentation and annotation software Labelme to separate the background and the target. The target is divided into two categories: circle and rectangle. (4) Build the UNet network model, adjust the network parameters, and determine the parameters in the model such as loss function, optimizer, and learning rate. Train the network model in the configured hardware and software environment. Calculate the evaluation indicators and visualize the calculation results of the indicators to evaluate the model. (5) Use the trained model to segment the object to be tested, distinguish the background and target areas, and then extract the edge points. (6) Based on the characteristics of the object to be measured, outliers are eliminated and then geometric parameter fitting is performed. (7) Calculate the actual size parameters based on the camera calibration results.

2. The microstructure parameter measurement method based on UNet network according to claim 1, characterized in that: In the step (2), data enhancement includes two methods: enriching the postures of micropores and microgrooves by horizontal flipping and vertical flipping; and enhancing the robustness of the model to different lighting environments by random changes in the HSV color gamut. Specifically, by adding random perturbations to the three channels of H, S, and V in the HSV color space, hue transformation is performed.

3. The microstructure parameter measurement method based on UNet network according to claim 1, characterized in that: In step (4), the parameters for training the network are set as follows: the batch number is set to 2, the initial learning rate is set to 1×10 -6 , the stochastic gradient descent algorithm selects the gradient descent with momentum, the momentum parameter is set to 0.999, the gradient clipping method is used to prevent gradient explosion, the norm is specified as 1, and the number of epochs is set to 50. Regarding the segmentation effect index, the Dice coefficient is used for evaluation. The expression of the Dice coefficient is shown in formula (1): In the formula, X represents the actual result and Y represents the predicted result.

4. The microstructure parameter measurement method based on UNet network according to claim 1, characterized in that: In the step (5), edge extraction specifically involves directly extracting pixel value coordinates of boundary points based on the binary image after segmentation.

5. The microstructure parameter measurement method based on UNet network according to claim 1, characterized in that: In the step (6), if the object to be tested is a microhole, the edge sequence points are connected to remove the protruding parts, and finally the area with the largest connected domain is taken as the feature area after repair. After repair, the least squares method is used to calculate the geometric parameters; if the object to be tested is a microgroove, first screen out all the internal points that meet the "correct solution", and then use the data that meets the best fit as the input of the least squares method for fitting.

6. The microstructure parameter measurement method based on UNet network according to claim 5 is characterized in that: In step (6), for the micropores, firstly, an ordered edge sequence of the features of the feature graph to be repaired is obtained, and then the edge sequence is traversed, and the traversed edge points are set as A(x a ,y a ), its related point B(x b ,y b ) is the edge point whose serial number follows it, and edge repair is performed according to formula (2) and formula (3). In the formula, (x c ,y c ) represents a point on the line segment formed by points A and B. After the repair is completed, the image is morphologically expanded, the intersection and matching of the image edge points and the original image edge points after each expansion operation are tracked, and the change law of the number of edge point matches is counted. When the number of intersecting edge points is greater than the matching rate threshold, the expansion is stopped, and the largest intersection point increment is recorded as the valid edge point. After the defect ellipse edge is repaired according to the above method, the least squares fitting is performed to obtain the parameters of the ellipse in the pixel coordinate system.

7. The microstructure parameter measurement method based on UNet network according to claim 5 is characterized in that: In step (6), the micro-grooves are firstly eliminated by using the RANSAC algorithm through multiple iterations. The iterative formula is shown in formula (4), and all the internal points that meet the "correct solution" are screened out. P=1-(1-t n ) k (4) Where n represents the interior point probability and k represents the number of iterations. The above method can be used to screen out the data points that meet the best fit, and then the least squares fitting is performed on them to obtain the parameters of the straight line in the pixel coordinate system.

8. The microstructure parameter measurement method based on UNet network according to claim 1, characterized in that: In the step (7), the camera is calibrated using the Zhang Zhengyou calibration method.

Images