A method for correcting distorted light spots in three-dimensional topography measurement of highly reflective elements

Abstract

This invention relates to the field of precision optical inspection engineering technology, and more specifically, to a method for correcting distorted spot images in the three-dimensional topography measurement of highly reflective components. The key technical points are: using a complex beam angle sensor to scan a cylindrical metal component and acquire a multi-spot image dataset; due to the high reflectivity and roughness of the metal material, nonlinear distortion occurs in the multi-spot images, affecting the accuracy of three-dimensional topography reconstruction; therefore, a distortion spot correction network is designed to correct the images to obtain ideal spot images. The center coordinates of the spots in the image are accurately extracted using the Hough circle detection algorithm; the processed image is segmented to extract multiple sub-spot images, achieving sub-pixel level spot center coordinate extraction with a coordinate positioning accuracy of 0.02 pixels, thus improving the accuracy of subsequent three-dimensional topography reconstruction. The advantages of this invention are strong anti-interference capability, fast calculation speed, and the ability to achieve sub-pixel level spot center extraction.

Description

Technical Field

[0001] This invention relates to the field of precision optical inspection engineering technology, and more specifically, to a method for correcting distorted light spots in the three-dimensional morphology measurement of highly reflective elements. Background Technology

[0002] Currently, cylindrical metal components are widely used in industrial manufacturing, primarily to support transmission parts, transmit torque, and bear loads. Therefore, they require extremely high precision in terms of roundness error and surface roughness. To determine whether the precision of machined cylindrical metal components meets the requirements, a self-developed multi-beam angle sensor (MBAS) is used to detect the three-dimensional shape of the cylindrical metal components and acquire multi-spot images. The center coordinates of the spots in the images are then accurately extracted to improve the accuracy of wavefront reconstruction, thereby precisely reconstructing a high-precision three-dimensional model of the cylindrical metal component. However, when using MBAS to detect cylindrical components, the high reflectivity of the metal material and surface roughness can cause nonlinear distortion in the spot images acquired by MBAS. This makes it impossible to accurately extract the center coordinates of the distorted spots, affecting subsequent angle difference calculations and thus reducing detection accuracy. Therefore, to prevent distorted spots from affecting the accuracy of MBAS in detecting the three-dimensional shape of high-precision cylindrical metal components, it is necessary to study how to correct the distorted spot images into ideal Gaussian spot images and then accurately extract the sub-pixel level center coordinates of the spots from the ideal image.

[0003] Currently, methods for locating the center coordinates of point-like light spots are mainly divided into two categories: gray-level conversion-based methods and edge-based methods. Gray-level conversion-based methods utilize the gray-level distribution information of the target, such as the gray-level centroid method and multi-moment geometry method, and are mainly suitable for situations where the spot radius is small and the brightness distribution is uniform. Edge-based methods utilize the edge shape information of the target for processing and calculation, such as ellipse fitting and Hough transform methods, and are suitable for light spot images with relatively large radii. Among them, the gray-scale centroid method has a fast calculation speed and can achieve relatively accurate positioning for uniform light spots, but its anti-interference ability is poor, and its positioning accuracy is very poor for real images with large noise or complex conditions; the multi-moment geometry method is not sensitive to noise, and the calculated centroid will not deviate much when there is external noise interference, and the calculation speed is also fast, but its accuracy is also very low; the ellipse fitting method has high calculation accuracy and low time complexity, but its anti-interference ability for random noise is very poor, and the accuracy of the center calculation will be significantly reduced when random noise is present; the Hough transform method has good anti-interference ability for noise interference, but its disadvantages are large calculation volume, large memory occupation, and the need for discretization of parameter space, which limits the detection accuracy.

[0004] Therefore, in order to address the problem of reduced detection accuracy caused by spot distortion due to the high reflectivity and surface roughness of the metal material when measuring cylindrical metal components by MBAS, there is an urgent need for a method that can correct distorted spot images and extract the center coordinates of the spot with high accuracy, strong anti-interference ability, and low computational cost. Summary of the Invention

[0005] To address the shortcomings of existing technologies, this invention provides a method for correcting distorted light spots in the three-dimensional morphology measurement of highly reflective components, which reduces the accuracy of MBAS measurements due to light spot distortion caused by the high reflectivity and surface roughness of the metal material.

[0006] The above-mentioned technical objective of the present invention is achieved through the following technical solution: a method for correcting distorted light spots in three-dimensional topography measurement of highly reflective elements, comprising the following steps:

[0007] S1. Decompose the MBAS measurement system into an optical path calibration module and an image acquisition module, and then build a system simulation model to acquire an ideal Gaussian spot image;

[0008] S2. Noise is added to the simulated optical path in the system simulation model, and defocus surfaces are set to acquire distorted light spot images;

[0009] S3. Perform image preprocessing and data augmentation, using the ideal Gaussian spot image and the distorted spot image as the training dataset and labels for the subsequent training network;

[0010] S4. Build a distortion spot correction network architecture in the PyCharm framework;

[0011] S5. Input the training dataset into the correction network architecture for training, and use the ideal Gaussian spot image as a label for backpropagation, and optimize it in the ADAM optimizer to obtain the weight model.

[0012] S6. Use the MBAS device to acquire images to obtain a real multi-spot image dataset;

[0013] S7. After smoothing the actual multi-spot image dataset, load it into the weight model for correction to obtain the corrected ideal spot image;

[0014] S8. Use the Hough circle detection algorithm in the ideal spot image to extract the center coordinates of each spot.

[0015] In one embodiment, the step of extracting the center coordinates of the light spot further includes:

[0016] S801. Perform edge detection on the corrected ideal spot image, detect the spot edges in the image and extract the pixel coordinates of the spot edges;

[0017] S802. Add a counter to the pixel coordinates of the light spot edge to count the number of corresponding points between the pixel coordinates of the light spot edge and the edge pixel points;

[0018] S803. Based on the edge pixel coordinates, search for the highest position of the counter in Hough space to obtain the center coordinates and radius of the light spot.

[0019] In one embodiment, the optical path calibration module includes calibration parameters for the microlens array and glass components, used to establish an optical model of the main components so that the light they produce is ideal parallel light.

[0020] In one embodiment, the image acquisition module includes a microlens array and a CMOS camera for acquiring a multi-spot image dataset generated by the object being picked up.

[0021] In one embodiment, the correction network architecture includes an input layer, a convolutional layer, a pooling layer, and an upsampling layer.

[0022] In one embodiment, the defocusing surfaces are set to 20µm, 50µm, and 100µm, respectively.

[0023] The above-mentioned distortion spot correction method, applicable to the three-dimensional topography measurement of highly reflective elements, has the following advantages:

[0024] Firstly, by utilizing the powerful feature learning capabilities of multi-layer convolutional neural networks, redundant information in the light spot image is removed, thereby achieving the goal of accurately extracting multi-light spot information from the image.

[0025] Secondly, it can correct distorted spot images into ideal spot images by using the Hough circle detection algorithm to accurately extract the center coordinates of the spot in the ideal spot image;

[0026] Third, the distortion spot correction method has strong anti-interference ability, fast calculation speed, and can achieve sub-pixel level spot center extraction. Attached Figure Description

[0027] Figure 1 This is a flowchart illustrating the specific implementation steps of this embodiment;

[0028] Figure 2 This is the structure and optical path diagram of the MBAS in this embodiment;

[0029] Figure 3 This is a schematic diagram of the architecture of the distortion spot correction network applied in this embodiment;

[0030] Figure 4This is a schematic diagram illustrating the effects of distortion spot correction and spot center coordinate extraction in this embodiment. Detailed Implementation

[0031] The present invention will now be described in detail with reference to the accompanying drawings and embodiments.

[0032] In the description of this invention, it should be understood that the terms "center," "longitudinal," "lateral," "length," "width," "thickness," "upper," "lower," "front," "rear," "left," "right," "vertical," "horizontal," "top," "bottom," "inner," "outer," "clockwise," "counterclockwise," "axial," "radial," and "circumferential" indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. They are used only for the convenience of describing this invention and are not intended to indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on this invention.

[0033] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this invention, "a plurality of" means at least two, unless otherwise explicitly specified.

[0034] In this invention, unless otherwise explicitly specified and limited, the terms "installation," "connection," "linking," and "fixing," etc., should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral part; they can refer to a mechanical connection, an electrical connection, or a connection that allows communication between them; they can refer to a direct connection or an indirect connection through an intermediate medium; they can refer to the internal communication of two components or the interaction between two components, unless otherwise explicitly limited. Those skilled in the art can understand the specific meaning of the above terms in this invention according to the specific circumstances.

[0035] In this invention, unless otherwise explicitly specified and limited, "above" or "below" the second feature can mean that the first feature is in direct contact with the second feature, or that the first feature is in indirect contact with the second feature through an intermediate medium. Furthermore, "above," "over," and "on top" of the second feature can mean that the first feature is directly above or diagonally above the second feature, or simply that the first feature is at a higher horizontal level than the second feature. "Below," "below," and "under" the second feature can mean that the first feature is directly below or diagonally below the second feature, or simply that the first feature is at a lower horizontal level than the second feature.

[0036] This invention utilizes a self-developed MBAS device and employs Zemax simulation software to simulate MBAS for acquiring light spot images. The structure and optical path diagram of the MBAS are shown below. Figure 2 As shown, the principle is as follows: the laser emitted by the laser diode forms a parallel light path after passing through the collimation unit, and then is reflected by the mirror to the beam splitter in the detection unit. The beam splitter disperses the light, and part of the dispersed light is reflected to the cylindrical lens and then focused onto the sample. The light reflected by the sample returns to the cylindrical lens and beam splitter, and then passes through the microlens array, finally reaching the camera and where the camera collects the light spot array data.

[0037] A method for correcting distorted light spots in the three-dimensional topography measurement of highly reflective elements, such as Figure 1 As shown, it includes the following steps:

[0038] S1. Decompose the MBAS measurement system into an optical path calibration module and an image acquisition module, and then build a system simulation model to acquire an ideal Gaussian spot image;

[0039] S2. Add noise to the simulated optical path in the system simulation model, and set defocus surfaces to acquire distorted light spot images;

[0040] S3. Perform image preprocessing and data augmentation, using the ideal Gaussian spot image and the distorted spot image as the training dataset and labels for the subsequent training network.

[0041] S4. Build a distortion spot correction network architecture in the PyCharm framework;

[0042] S5. Input the training dataset into the calibration network architecture for training, and perform backpropagation with the ideal Gaussian spot image as the label. Optimize the model in the ADAM optimizer to obtain the weight model.

[0043] S6. Use the MBAS device to acquire images to obtain a real multi-spot image dataset;

[0044] S7. After smoothing the actual multi-spot image dataset, load it into the weight model for correction to obtain the corrected ideal spot image.

[0045] S8. Use the Hough circle detection algorithm in the ideal spot image to extract the center coordinates of each spot.

[0046] Specifically, the steps for extracting the center coordinates of the light spot also include:

[0047] S801. Perform edge detection on the corrected ideal spot image, detect the spot edges in the image and extract the pixel coordinates of the spot edges;

[0048] S802. Add a counter to the pixel coordinates of the light spot edge to count the number of corresponding points between the pixel coordinates of the light spot edge and the edge pixel points;

[0049] S803. Based on the edge pixel coordinates, search for the highest position of the counter in Hough space to obtain the center coordinates and radius of the light spot.

[0050] Specifically, the optical path calibration module includes calibration parameters for the microlens array and glass components, which are used to establish the optical model of the main components so that the light they produce is ideal parallel light.

[0051] Specifically, the image acquisition module includes a microlens array and a CMOS camera, used to acquire a multi-spot image dataset generated by the object being picked up.

[0052] Specifically, the calibration network architecture includes an input layer, convolutional layers, pooling layers, and upsampling layers.

[0053] Specifically, the defocus surfaces are set to 20um, 50um, and 100um, respectively.

[0054] Based on the above scheme, in one example, highly reflective cylindrical metal elements with radii of 10mm, 20mm, 35mm, and 50mm are used as the objects under test. First, a laser diode emits a laser with a wavelength of 658nm. The laser passes through a condenser lens, a pinhole, and a collimating lens in sequence to form parallel light. Then, the main light energy passes through an aperture, a reflector, a beam splitter, and a cylindrical lens to reach the cylindrical metal element. The light reflected from the element then passes through a beam splitter and a microlens array to reach the CMOS camera, which can then collect the light spot array data.

[0055] The MBAS (Magnetic Bandwidth Interchange) measurement system was simulated using Zemax software. During modeling, the MBAS system was decomposed into an optical path calibration module and an image acquisition module. The optical path calibration module mainly includes calibration parameters for glass components such as microlens arrays, establishing optical models of the main components to enable them to acquire ideal Gaussian spot images. The image acquisition module, used to acquire multi-spot images generated by the inspected object, mainly consists of a microlens array and a CMOS camera. A system simulation model was built by combining the component optical models and optical path structure parameters, enabling the simulation model to have image acquisition capabilities. The optical path calibration module maps the multi-spot images formed by different inspected objects after focusing through the microlens array onto a rectangular detector and saves them. After acquiring the ideal Gaussian spot image, defocusing surfaces of 20µm, 50µm, and 100µm were set in the simulated optical path, and corresponding distorted spot images were acquired, with varying degrees of noise added. The simulated ideal Gaussian spot images and distorted spot images were used as training datasets for subsequent network training.

[0056] Subsequently, a distortion correction network was built in PyCharm based on the PyTorch framework, such as... Figure 3 As shown, the network architecture includes an input layer, convolutional layers, pooling layers, upsampling layers, and an output layer, mainly composed of a combination of encoder and decoder. The first layer is the input layer, where the input data is the distorted spot image generated when detecting cylindrical metal components using MBAS. The second and third layers are composite convolutional layers, containing a convolution operation with a kernel size of 3x3 and a stride of 1, a batch normalization, and a LeakyReLU activation function. Then, in the fourth layer, a max pooling layer with a kernel size of 2x2 is used to obtain high-level semantic features of the image, remove redundant information in the spot image, and compress image features. The second to twelfth layers of the network consist of the aforementioned composite convolutional layers and max pooling layers, forming the encoder part of the network. This encoder can progressively reduce the spatial resolution of the input distorted multi-spot image, extract and compress the low-level features in the distorted spot image, filter out the distortion points around a single spot through composite convolutional layers, and generate a low-dimensional feature vector, which is then passed to the subsequent decoder part. Layers 13 to 23 of the network consist of composite pooling layers and upsampling layers, forming the decoder portion of the network. This decoder transforms the low-dimensional feature vectors extracted by the encoder back into a Gaussian multi-spot image of the same size as the input distorted image. Finally, a high-resolution output image is generated through a 3x3 convolutional output layer in layer 24. Therefore, the combination of the encoder and decoder can perform feature extraction, compression, distortion correction, and restoration on the input distorted multi-spot image, ultimately generating a high-quality spot-corrected image.

[0057] Based on this operation, the advantages are that batch normalization makes the distribution of input data in each layer of the network relatively stable, which speeds up the model learning speed, simplifies the parameter tuning process, and makes the network learning more stable. At the same time, the LeakyReLu activation function solves the problem of neuron "death". When the input is negative, it gives the input value a very small slope, and can also calculate the gradient of the part of the input that is less than zero during backpropagation.

[0058] The ideal Gaussian spot image and the distorted spot image obtained through simulation using Zemax software are used as training datasets and input into the aforementioned distortion spot correction network for training. The ideal spot image is used as a label for backpropagation, and the network is optimized using the ADAM optimizer to obtain a reliable and stable weight model. In the training process, the dataset contains 5000 sets, which are divided into training and validation sets in a 9:1 ratio. Cross-validation is used for training, and a training loss function is set. The network is trained for 30,000 epochs. After training, the actual distorted spot images acquired by the MBAS device are smoothed and then input into the trained correction network architecture to obtain the corrected ideal spot image. The loss function is as follows:

[0059]

[0060] After obtaining the corrected ideal spot image, the actual spot image acquired by the MBAS device is processed through a trained distortion spot correction network to obtain the corresponding ideal spot image. Then, the center coordinates of each spot can be accurately extracted from the ideal spot image using the Hough circle detection algorithm. First, an edge detection algorithm (Canny algorithm) is used to detect the spots in the ideal spot image and extract the coordinates of the edge pixels. For each detected edge pixel (x, y), the coordinates can be calculated using the formula... Calculate the coordinates (x, y) of all possible light spot centers. c ,y c ) and radius r, and all (x) c ,y c The data (x, r) form a Hough space, and in each (x, r) c ,y c Add a counter to (r) to count how many edge pixels this data corresponds to. After calculating the number of edge pixels corresponding to all the light spot pixels, (x...)... c ,y c After obtaining the data (r), search for the position of the highest counter in the Hough space, where (x) is the highest value. c ,y c ,r) represents the most probable coordinates of the spot center and radius. Therefore, the Hough circle detection algorithm can be used to extract the center coordinates of each spot in the corrected distorted spot image with high accuracy, such as Figure 4 As shown, the present invention can correct light spots with different degrees of distortion and obtain high-precision center coordinates, proving the effectiveness and accuracy of the proposed method.

[0061] In practical operation, this invention utilizes a multi-beam angle sensor (MBAS) to scan a cylindrical metal component and acquire a multi-spot image dataset. However, due to the high reflectivity and surface roughness of the metal material, the multi-spot images suffer from nonlinear distortion, affecting the accuracy of subsequent 3D topography reconstruction. Therefore, a distortion spot correction network is designed to correct the images and obtain ideal spot images. The Hough circle detection algorithm is used to accurately extract the center coordinates of the spots in the images. The processed images are segmented to extract multiple sub-spot images and achieve sub-pixel level spot center coordinate extraction. The coordinate positioning accuracy can reach 0.02 pixels, improving the accuracy of subsequent 3D topography reconstruction of the component.

[0062] Based on the above scheme, the advantages are that the method has good robustness, can perform high-precision correction on irregular and blurred light spots in the image, and extract the center coordinates of the light spots. The light spot center extraction accuracy obtained by this method reaches the sub-pixel level, which greatly improves the measurement accuracy of MBAS.

[0063] The embodiments described above are merely illustrative of several implementations of the present invention, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of the present invention. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the present invention, and these modifications and improvements all fall within the scope of protection of the present invention. Therefore, the scope of protection of this patent should be determined by the appended claims.

Claims

1. A method for correcting distorted light spots in three-dimensional topography measurement of highly reflective elements, characterized in that, Includes the following steps: S1. Decompose the MBAS measurement system into an optical path calibration module and an image acquisition module, and then build a system simulation model to acquire an ideal Gaussian spot image; S2. Noise is added to the simulated optical path in the system simulation model, and defocus surfaces are set to acquire distorted light spot images; S3. Perform image preprocessing and data augmentation, using the ideal Gaussian spot image and the distorted spot image as the training dataset and labels for the subsequent training network; S4. Build a distortion spot correction network architecture in the PyCharm framework; S5. Input the training dataset into the correction network architecture for training, and use the ideal Gaussian spot image as a label for backpropagation, and optimize it in the ADAM optimizer to obtain the weight model. S6. Use the MBAS device to acquire images to obtain a real multi-spot image dataset; S7. After smoothing the actual multi-spot image dataset, load it into the weight model for correction to obtain the corrected ideal spot image; S8. Use the Hough circle detection algorithm in the ideal spot image to extract the center coordinates of each spot; The step of extracting the center coordinates of the light spot further includes: S801. Perform edge detection on the corrected ideal spot image, detect the spot edges in the image and extract the pixel coordinates of the spot edges; S802. Add a counter to the pixel coordinates of the light spot edge to count the number of corresponding points between the pixel coordinates of the light spot edge and the edge pixel points; S803. Based on the edge pixel coordinates, search for the highest position of the counter in Hough space to obtain the center coordinates and radius of the light spot.

2. The distortion spot correction method for three-dimensional topography measurement of high reflectivity elements according to claim 1, characterized in that: The optical path calibration module includes calibration parameters for the microlens array and glass components, which are used to establish the optical model of the main components so that the light they produce is ideal parallel light.

3. The distortion spot correction method for three-dimensional topography measurement of high reflectivity elements according to claim 1, characterized in that: The image acquisition module includes a microlens array and a CMOS camera, used to acquire a multi-spot image dataset generated by the object being picked up.

4. The distortion spot correction method for three-dimensional topography measurement of high reflectivity elements according to claim 1, characterized in that: The corrected network architecture includes an input layer, a convolutional layer, a pooling layer, and an upsampling layer.

5. The method for correcting distorted light spots in three-dimensional topography measurement of high-reflectivity elements according to claim 1, characterized in that: The defocus surfaces are set to 20um, 50um and 100um respectively.

Images