A microstructure parameter measurement method based on a UNet network

Abstract

This invention relates to the fields of precision instrument manufacturing and precision testing and measurement technology, specifically a microstructure parameter measurement method based on a UNet network. First, addressing the presence of numerous prominent textures and scratches in the background region of the image of the test part, a region segmentation method based on a UNet network is employed to improve segmentation accuracy and overcome the influence of noise. Then, a dataset is collected and image enhancement is performed to improve the model's generalization ability. Horizontal and vertical flipping enriches the pose of the microstructure, and random variations in the HSV color gamut enhance the model's robustness to different lighting environments. After segmenting the target region, edge points are extracted and then fitted. Due to the presence of processing defects, outliers need to be removed before fitting. For micropore structures, outlier removal involves connecting edge sequence points to remove protrusions, and finally selecting the region with the largest connected component as the repaired feature region. For microgroove structures, outlier removal involves using the RANSAC algorithm for multiple iterations to filter out all interior points satisfying the "correct solution," and then using the data that best fits as input for least squares fitting, which can better estimate model parameters. This invention can improve the visual measurement accuracy of micro-holes and micro-grooves at the micrometer to millimeter level in workpieces.

Description

Technical Field

[0001] This invention is mainly for measuring the parameters of micropores and microgrooves at the level of hundreds of micrometers to millimeters. It integrates region segmentation based on deep learning and geometric element fitting based on outlier removal, and belongs to the field of precision instrument manufacturing and precision testing and measurement technology. Background Technology

[0002] Array-type microstructures ranging from hundreds of micrometers to millimeters have wide applications and broad development prospects in the field of high-end equipment manufacturing. Micropores and microgrooves are particularly widely used in the aerospace field, especially since the size and geometric characteristics of microstructures have a significant impact on device performance. Furthermore, the development of ultra-precision measurement technology also constrains the development of precision and ultra-precision machining technology. For these types of microstructures, visual measurement methods are widely used, the main steps of which involve first segmenting the region, then extracting edges, and finally fitting and calculating geometric parameters.

[0003] Image segmentation is a crucial step. Early image segmentation algorithms were primarily based on low-level image features, such as contours, edges, color, texture, and shape. Key algorithms include threshold-based and region-based segmentation. These methods mainly segment based on image pixels, but can also utilize image feature vectors. Common methods include clustering-based and classifier-based segmentation. When prior knowledge of the image is known, template matching, deformation models, etc., can be used to divide the image into regions conforming to the shape prior, based on the known or assumed target shape. These methods are called shape-prior-based methods. In addition, with the development of various disciplines, wavelet transform-based, graph theory-based, and energy functional-based segmentation algorithms have emerged. However, these traditional methods cannot simultaneously consider both low-level image features and semantic information, easily leading to missegmentation of regions whose low-level features are close to the target. Furthermore, during processing, tools leave scratches on the workpiece surface, resulting in an uneven morphology under high magnification, leading to inaccurate measurement results when directly fitted. To address these issues, this invention provides a microstructure parameter measurement method based on a UNet network. Summary of the Invention

[0004] The purpose of this invention is to provide a microstructure parameter measurement method based on UNet network, which solves the problem of inaccurate segmentation caused by a large number of prominent textures and scratches in the background area under high-definition lens, as well as the problem that the extracted edges are not regular geometric shapes due to morphological defects caused by the processing.

[0005] To achieve the above effects, the technical solution provided by this invention is as follows: First, since the background area of ​​the image of the test piece contains a large number of prominent textures and scratches, target region segmentation is required. This invention adopts a region segmentation method based on the UNet network, collects a dataset, and enriches the microstructure pose by horizontal and vertical flipping; by randomly changing the HSV color gamut, the robustness of the model to different lighting environments is enhanced, and the enhanced dataset is used for training to obtain a segmentation model with stronger generalization ability. After segmenting the target region, edge points are extracted. For outlier removal of micropore structures, the method is to connect the edge sequence points to remove the protruding parts, and finally take the region with the largest connected component as the repaired feature region; for outlier removal of microgroove structures, the method is to first use the RANSAC algorithm for multiple iterations to remove outliers, select all interior points that satisfy the "correct solution", and then use the data that meets the best fit as the input of the least squares method for fitting, which can better estimate the model parameters.

[0006] The method for measuring microstructure parameters based on UNet networks includes the following steps:

[0007] (1) Add a ring light to the camera and collect the dataset;

[0008] (2) Data augmentation is performed on the collected dataset, mainly by horizontal and vertical flipping to enrich the pose of micropores and microgrooves; the robustness of the model to different lighting environments is enhanced by random variation of the HSV color gamut.

[0009] (3) Divide the dataset into training and test sets, and use the semantic segmentation annotation software Labelme to annotate the images, dividing them into background and target. The target is divided into two categories: circle and rectangle.

[0010] (4) Build the UNet network model, tune the network parameters, and determine the parameters in the model such as loss function, optimizer, and learning rate.

[0011] (5) Train the network model in the configured hardware and software environment.

[0012] (6) Calculate the evaluation indicators and visualize the calculation results to evaluate the model.

[0013] (7) Use the trained model to segment the target area, distinguish the background and target areas, and then extract the edge points.

[0014] (8) If the object to be tested is a micropore, connect the edge sequence points to remove the protruding part, and finally take the region with the largest connected region as the feature region after repair. After repair, use the least squares method to calculate the geometric parameters. If the object to be tested is a microgroove, first use the RANSAC algorithm to perform multiple iterations to remove and select all the interior points that satisfy the "correct solution". Then use the data that meets the best fit as the input of the least squares method to fit, which can better estimate the model parameters.

[0015] (9) Calculate the actual size parameters based on the camera calibration results.

[0016] Compared with existing technologies, this invention proposes a method for measuring microstructure parameters based on UNet networks, which has the following advantages:

[0017] (1) The region segmentation method based on the UNet network makes full use of the low-level information and semantic information of the image, which can overcome the influence of scratches and background noise around the workpiece under test, and will not be affected by non-target elements, and can completely segment the target elements. After adding two data augmentation methods, the trained model has better robustness.

[0018] (2) Outlier removal methods were designed for fitting ellipses and lines respectively. After removal, the least squares method was used to fit the visual measurement parameters, which can overcome the problem of inaccurate fitting caused by processing defects and improve measurement accuracy.

[0019] The method proposed in this invention is suitable for non-contact measurement of microholes and microgrooves at the level of hundreds of micrometers to millimeters. Compared with traditional methods, it has higher robustness and measurement accuracy, and can be applied in the aerospace and precision manufacturing fields. Attached Figure Description

[0020] Figure 1 This is a flowchart of a method for measuring microstructure parameters based on UNet networks.

[0021] Figure 2 This is a flowchart of HSV color gamut random transformation.

[0022] Figure 3 This is a model diagram of the UNet network.

[0023] Figure 4 This is a diagram showing the segmentation effect of micropores and microgrooves.

[0024] Figure 5 This is a schematic diagram of outlier removal in defective ellipses. Detailed Implementation

[0025] The technical solutions of the present invention will now be clearly and completely described with reference to the accompanying drawings in the embodiments of the present invention:

[0026] The measurement principle and process of this invention are as follows:

[0027] (1) Install a ring light on the industrial camera, adjust it to a suitable focal length, and collect the dataset;

[0028] (2) Data augmentation was performed on the collected dataset. This primarily involved enriching the poses of the micropores and microgrooves through horizontal and vertical flipping; and enhancing the model's robustness to different lighting environments through random HSV color gamut transformation. The process of HSV color gamut random transformation is as follows: Figure 2 As shown, hue transformation is achieved by adding random perturbations to the H, S, and V channels in the HSV color space.

[0029] (3) Divide the dataset into training and test sets, and use the semantic segmentation annotation software Labelme to annotate the images, dividing them into background and target. The target is divided into two categories: circle and rectangle.

[0030] (4) This invention selects PyTorch as the deep learning framework to build the structure of the UNet network. The UNet network structure is as follows: Figure 3 As shown, the overall structure consists of a contraction path on the left and an expansion path on the right. The contraction path follows the typical architecture of a convolutional network, consisting of two 3×3 convolutional layers (ReLU) and a 2×2 max pooling operation with a stride of 2 for downsampling, doubling the number of feature channels in each downsampling step. Each step in the expansion path includes upsampling of the feature map, but with the number of feature channels halved, and a connection to the corresponding cropped feature map in the contraction path, along with two 3×3 convolutional layers (ReLU). In the final layer, a 1×1 convolution maps each 64-component feature vector to the desired number of classes.

[0031] (4) When training the network, the batch size is set to 2, and the initial learning rate is set to 1×10. -6 The stochastic gradient descent algorithm uses momentum gradient descent with the momentum parameter set to 0.999. Gradient clipping is used to prevent gradient explosion, the norm is set to 1, and the number of epochs is set to 50.

[0032] (5) Calculate the evaluation index and visualize the index calculation results to evaluate the model. When evaluating the region segmentation effect, this invention uses the Dice coefficient for evaluation. The Dice coefficient is a set similarity measurement function used to calculate the similarity between two samples, and its value ranges from [0,1]. Its expression is shown in equation (1).

[0033] (1)

[0034] In the formula, X represents the actual result and Y represents the predicted result.

[0035] (6) Use the trained model to segment the target area, distinguish the background and target areas, and then extract the edge points.

[0036] (7) If the object to be tested is a micropore, connect the edge sequence points to remove the protruding part, and finally take the region with the largest connected component as the feature region after repair. After repair, use the least squares method to calculate the geometric parameters. A specific repair diagram is shown below. Figure 5 As shown, firstly, an ordered edge sequence of features is obtained from the feature map that needs to be repaired. Then, the edge sequence is traversed, and the traversed edge points are denoted as A(x). a , y a ), its related point B(x) b , y b () is an edge point whose sequence number follows it, and satisfies the following condition:

[0037] (2)

[0038] In the formula, r represents the repair radius.

[0039] The edge repair algorithm is as follows:

[0040] (3)

[0041] (4)

[0042] In the formula, (x c , y c () represents a point on the line segment formed by points A and B.

[0043] After multiple rounds of outlier removal and repair, a smooth curve can be obtained from the defective ellipse. This curve can be used as an approximate circle as a target, and the roundness can be used to evaluate the repair effect. The roundness expression is shown in Equation (5).

[0044] (5)

[0045] In the formula, S represents the area of ​​the repaired figure, and C represents the perimeter of the repaired figure.

[0046] After restoration, morphological dilation is performed on the image. The intersection and matching of edge points in the original image with those in the image after each dilation operation are tracked, and the changing pattern of the number of matching edge points is statistically analyzed. Dilation stops when the number of intersecting edge points exceeds the matching rate threshold, and the edge point with the largest increase in intersection point is recorded as the valid edge point. After restoring the defective ellipse edge using the above method, least squares fitting is performed to obtain the parameters of the ellipse in the pixel coordinate system.

[0047] If the object under test has microgrooves, first use the RANSAC algorithm for multiple iterations to eliminate inliers, selecting all inliers that satisfy the "correct solution". Then, use the data that best fits as input for least squares fitting, which can better estimate the model parameters. Assuming the probability of an inlier in the entire dataset is t, then t can be expressed as:

[0048] (6)

[0049] In the formula, n in n represents the number of interior points. out Indicates the number of exterior points.

[0050] According to probability theory, the probability that all n randomly selected points are interior points is t. n The probability of having at least one outlier is 1-t. n Then, in k random samplings, the probability that each sampling includes at least one outlier is (1-t). n ) k The correct solution requires that each sampled point be an interior point. Therefore, the probability expression for obtaining the correct solution can be deduced as shown in equation (7).

[0051] (7)

[0052] During the iterative calculation, a larger P value indicates a better model performance. According to equation (7), the size of the P value depends on the inlier probability n and the number of iterations k. Even when n is large, the probability P of obtaining the correct solution can be guaranteed by increasing the number of iterations k. The formula for calculating k can be derived from equation (7):

[0053] (8)

[0054] The above method can be used to select data points that meet the best fit. Next, the least squares fit is performed to obtain the parameters of the line in the pixel coordinate system.

[0055] (8) Calculate the actual size parameters based on the camera calibration results.

[0056] The above descriptions are merely preferred embodiments of this application, and the present invention is not limited to the above embodiments. It is understood that any changes made to the technical solution of this invention that do not exceed the scope of the technical solution of this invention are within the protection scope of this invention.

Claims

1. A method for measuring microstructure parameters based on UNet networks, targeting micropores and microgrooves with background noise and processing defects at the hundred-micrometer to millimeter scale, includes the following steps: (1) Add a ring light to the camera to collect data on micro-holes and micro-grooves; (2) Perform data augmentation on the collected dataset; (3) Divide the dataset into training and test sets, and use semantic segmentation annotation software to annotate the images to divide them into background and target. The target is divided into two categories: circle and rectangle. (4) Build a UNet network model, tune the network parameters, determine the parameters in the model such as loss function, optimizer, and learning rate, train the network model in the configured hardware and software environment, calculate the evaluation index, and visualize the index calculation results to evaluate the model. (5) Use the trained model to perform region segmentation on the test object. After distinguishing the background and target regions, extract the pixel coordinates of the boundary points of the target region based on the segmented binary image. (6) For the micropores or microgrooves, the corresponding repair algorithm is used to remove outliers caused by processing defects, and then geometric parameters are fitted. If the object to be tested is a micropore, the edge sequence points are connected to remove the protruding part. Finally, the region with the largest connected component is taken as the feature region after repair. After repair, the least squares method is used to calculate the geometric parameters. If the object to be tested is a microgroove, all interior points that satisfy the correct solution are first selected according to the RANSAC algorithm, and then the data that meets the best fit is used as the input of the least squares method for fitting. (7) Calculate the actual size parameters based on the camera calibration results.

2. The method for measuring microstructure parameters based on UNet network according to claim 1, characterized in that: In step (2), data augmentation includes two methods: enriching the posture of micropores and microgrooves by horizontal and vertical flipping; and enhancing the robustness of the model to different lighting environments by random variation of the HSV color gamut. Specifically, hue transformation is performed by adding random perturbations to the H, S, and V channels in the HSV color space.

3. The method for measuring microstructure parameters based on UNet network according to claim 1, characterized in that: In step (4), the parameters for training the network are set as follows: the batch size is set to 2, and the initial learning rate is set to 1×10. -6 The stochastic gradient descent algorithm uses momentum gradient descent, with the momentum parameter set to 0.

999. Gradient clipping is used to prevent gradient explosion, the norm is set to 1, and the number of epochs is set to 50. The Dice coefficient is used to evaluate the segmentation performance, and its expression is shown in equation (1). (1) In the formula, X represents the actual result and Y represents the predicted result.

4. The method for measuring microstructure parameters based on UNet network according to claim 1, characterized in that: In step (6), for micropores, firstly, an ordered edge sequence of features is obtained from the feature map that needs to be repaired, and then the edge sequence is traversed, with the traversed edge points being A(x). a , y a ), its related point B(x) b , y b The edge points with serial numbers following them are repaired according to equations (2) and (3). (2) (3) In the formula, (x c , y c () represents a point on the line segment formed by points A and B; After the repair is completed, morphological dilation is performed on the image. The intersection and matching of the edge points of the image after each dilation operation with the edge points of the original image are tracked. The change pattern of the number of matching edge points is statistically analyzed. Dilation is stopped when the number of intersecting edge points is greater than the matching rate threshold. The one with the largest increase in intersection points is recorded as the effective edge point. After repairing the defective ellipse edge, least squares fitting is performed to obtain the parameters of the ellipse in the pixel coordinate system.

5. The method for measuring microstructure parameters based on UNet network according to claim 1, characterized in that: In step (6), for micro-grooves, the RANSAC algorithm is first used to perform multiple iterations to eliminate them. The iterative formula is shown in equation (4), and all interior points that satisfy the correct solution are selected. （4） In the formula, n represents the interior point probability, and k represents the number of iterations; After selecting the data points that best fit the criteria, the line is then fitted using least squares to obtain its parameters in the pixel coordinate system.

Images