Object three-dimensional topography measurement method based on unet convolutional neural network adaptive spatial filtering

Abstract

The application discloses a kind of object three-dimensional topography measurement methods based on Unet convolutional neural network adaptive spatial filtering.The computer simulation generates random holographic interferogram conversion into frequency spectrum chart;Iterative threshold segmentation processing obtains frequency spectrum binary segmentation chart;Unet convolutional neural network is built;Frequency spectrum chart is input and frequency spectrum binary segmentation chart is label and is trained;The holographic interferogram of the object to be measured is converted into test frequency spectrum chart, and the object image spectrum center coordinates are determined by phase information;Test frequency spectrum chart is input into the network trained and output frequency spectrum binary segmentation chart, and the object image spectrum region binary segmentation mask is extracted according to the object image spectrum center coordinates in test frequency spectrum chart, and the object image spectrum filtering chart is obtained by filtering test frequency spectrum chart, and the three-dimensional topography chart of the object to be measured is reconstructed.The method has the excellent performance of threshold iterative segmentation method and the rapid processing capacity of neural network, which is beneficial to realize the high-quality and rapid reconstruction of object three-dimensional topography.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of digital holography, and particularly relates to a three-dimensional topography measurement method based on Unet convolutional neural network adaptive spatial filtering. BACKGROUND

[0002] Digital holographic interferometry (DHI) is a technology that records the holographic interference pattern generated by the interference of object light and reference light through a photoelectric detector, and then obtains the three-dimensional topography of the measured object through numerical reconstruction. However, the interference of the zero-order term, the conjugate term and the stray spectrum in the frequency spectrum space seriously hinders the realization of high-quality object image reconstruction by DHI. So far, many automatic spectral filtering methods have been designed by researchers at home and abroad based on image processing technology, but these methods face the contradiction between processing speed and filtering effect, and often need a lot of time for iterative search operation in order to obtain a better filtering window. How to automatically obtain the best shape of the spectrum as the filtering window while reducing the computational load and speeding up the processing speed is still an area to be improved for spatial filtering technology. SUMMARY

[0003] In order to solve the above technical problems, the present application provides a three-dimensional topography measurement method of an object based on Unet convolutional neural network adaptive spatial filtering.

[0004] The present application is realized by the following technical solutions:

[0005] Step one: generate multiple random holographic interference patterns by computer simulation, and convert them into frequency spectrum P0; specifically, generate 30,000 random holographic interference patterns.

[0006] Step two: perform iterative threshold segmentation on the frequency spectrum P0 of each random holographic interference pattern to obtain a frequency spectrum binary segmentation P1;

[0007] Step three: build an Unet convolutional neural network that integrates a multi-scale feature fusion module and a bottleneck residual module;

[0008] Step four: take the frequency spectrum P0 as the input of the Unet convolutional neural network, and take the frequency spectrum binary segmentation P1 corresponding to the frequency spectrum P0 as the learning label, and train the Unet convolutional neural network;

[0009] Step five: collect the holographic interference pattern of the measured object, and then convert it into a test frequency spectrum P 01 , determine the object image spectrum center coordinate (x1, y1) through the phase information of the test frequency spectrum P 01 ;

[0010] Step Six: Input the test spectrum P into the trained Unet convolutional neural network. 01 The output obtains the test spectrum P. 01 Spectral binary segmentation P 11 Then, based on the center coordinates (x1, y1) of the object's spectrum, in the test spectrum P 01 Spectral binary segmentation P 11 Search and extract the object's spectral region binary segmentation mask P2 in the foreground region, where the foreground region is the white region with an intensity value of 1 for each pixel;

[0011] Step 7: Using the binary segmentation mask P2 of the object image spectrum region as the filtering window, filter the test spectrum image P. 01 Spectral spatial filtering is performed to obtain the object image spectral filtering map P3. Finally, the three-dimensional morphology map of the object under test is reconstructed through phase unwrapping and distortion compensation operations.

[0012] The object to be tested is a micro- or nano-structured object, such as an ultra-precision component, a MEMS chip, or a biological cell.

[0013] In this invention, a resolution test target is used as the test object in the embodiment, and a holographic interferogram of the surface of the test object is acquired.

[0014] Step one specifically includes:

[0015] 1.1) Objects with different three-dimensional morphological features are simulated using computer simulation. Based on the Fresnel diffraction principle, the object light signal of the laser passing through the simulated object and the reference light signal that does not pass through the simulated object are calculated. Then, the object light signal and the reference light signal interfere at random angles on the same plane to generate a simulated holographic interference pattern.

[0016] I = |O| 2 +|R| 2 +OR * +O * R

[0017] Where I represents the holographic interference signal, O is the object light signal formed by diffraction after the laser passes through the object, and R is the reference light signal when the laser does not pass through the object; O* and R* are the conjugate of the object light signal and the conjugate of the reference light signal, respectively.

[0018] The objects mentioned include objects with different shapes.

[0019] 1.2) Convert the holographic interferogram into a spectrogram P0 using a two-dimensional Fourier transform:

[0020] P0 = FFT{I} = FFT{|O| 2}+FFT{|R| 2}+FFT{OR *}+FFT{O* R}

[0021] Where FFT{} denotes the two-dimensional Fourier transform; FFT{|O| 2}+FFT{|R| 2 Together, they constitute the zero-order term spectrum in the spectrum diagram P0, FFT{O(x,y)R * (x,y)} and FFT{O * (x,y)R(x,y)} represents the image spectrum and the conjugate term spectrum, respectively.

[0022] Step two specifically involves:

[0023] 2.1) Pre-set the global threshold T0 of the spectrum P0 and the critical area S to be deleted, where T0 is greater than 0 and less than 1;

[0024] 2.2) After applying mean filtering to the spectrum P0, iterative threshold segmentation is performed based on the global threshold T0 and the critical area S to be deleted to obtain the spectrum segmentation map;

[0025] 2.3) The number of foreground regions in the spectral segmentation map obtained from the first threshold segmentation operation is automatically identified by computer, and a judgment is made:

[0026] If the number of foreground regions in the spectrum segmentation map obtained by the first threshold segmentation operation is less than 3, then the global threshold T0 is increased by 0.01 and returned to step 2.2) for processing, and the process is iterated until the number of foreground regions is not less than 3. The global threshold T0 of the last iteration is recorded as the segmentation threshold T.

[0027] If the number of foreground regions in the spectrum segmentation map obtained by the first threshold segmentation operation is greater than 3, the current global threshold T0 is directly used as the segmentation threshold T.

[0028] If the number of foreground regions in the spectrum segmentation map obtained by the first threshold segmentation operation is equal to 3, then the global threshold T0 is reduced by 0.01 and returned to step 2.2) for processing, and iterated continuously until the number of foreground regions is not equal to 3. The global threshold T0 of the penultimate iteration is recorded as the segmentation threshold T.

[0029] 2.4) Use the segmentation threshold T obtained in the final iteration to perform threshold segmentation on the spectrum map P0. Finally, sort all the foreground regions obtained after segmentation according to their area size, and keep only the three with the largest area, namely the zero-level term region, the conjugate term region and the object image spectrum region, so as to obtain the spectrum binary segmentation map P1.

[0030] In section 2.2), the threshold segmentation operation specifically includes:

[0031] Perform iterative threshold segmentation on the spectrogram P0, setting the intensity values ​​of pixels smaller than the global threshold T0 to 0, and setting the intensity values ​​of the remaining pixels to 1, where 1 represents the foreground region and 0 represents the background region; then delete all foreground regions whose area is smaller than the critical area S to be deleted.

[0032] Represented as:

[0033]

[0034] Where P1(x,y) represents each pixel in the binary segmentation map of the spectrum, P0(x,y) represents the corresponding pixel in the spectrum map, and T0 represents the initial segmentation threshold.

[0035] In step three, the Unet convolutional neural network mainly consists of an encoding subnetwork, a decoding subnetwork, and a bridging path connecting the encoding subnetwork and the decoding subnetwork.

[0036] The encoding subnetwork comprises four consecutive downsampling modules, each connected by a 2×2 max pooling operation. The decoding subnetwork comprises four consecutive upsampling modules, each connected by a 2×2 upconvolution operation. The input of the first downsampling module serves as the input to the Unet convolutional neural network. The output of the fourth downsampling module is concatenated with the result of a 2×2 max pooling operation, two consecutive 3×3 convolutional modules, and a 2×2 upconvolution operation, before being input to the first... In each upsampling module, the output of the first upsampling module, after undergoing a 2×2 upconvolution operation, is concatenated with the output of the third downsampling module and then input into the second upsampling module. The output of the second upsampling module, after undergoing a 2×2 upconvolution operation, is concatenated with the output of the second downsampling module and then input into the third upsampling module. The output of the third upsampling module, after undergoing a 2×2 upconvolution operation, is concatenated with the output of the first downsampling module and then input into the fourth upsampling module. The output of the fourth upsampling module, after being processed by a 1*1 convolutional layer, serves as the output of the Unet convolutional neural network.

[0037] Each downsampling module consists of a multi-scale feature fusion module and a bottleneck residual module connected in sequence; each upsampling module consists of two 3*3 convolution modules connected in sequence; the multi-scale feature fusion module includes five 3*3 convolution modules. The input of the multi-scale feature fusion module passes through only one 3*3 convolution module to obtain the first feature, the input of the multi-scale feature fusion module passes through two consecutive 3*3 convolution modules to obtain the second feature, and the input of the multi-scale feature fusion module passes through three consecutive 3*3 convolution modules to obtain the third feature. The first feature, the second feature, and the third feature are concatenated in sequence to obtain the comprehensive feature of image shape and position at multiple scales as the output of the multi-scale feature fusion module; the bottleneck residual module includes one 3*3 convolution module and two 1*1 convolution modules. The input of the bottleneck residual module passes through the first 1*1 convolution module, the 3*3 convolution module, and the second 1*1 convolution module in sequence, and the result is added to the input of the bottleneck residual module. The result after the sum is passed through an activation function is the output of the bottleneck residual module.

[0038] The 3x3 convolutional module is composed of a 3x3 convolutional layer, a normalization layer, and an activation function connected sequentially. The 1x1 convolutional module is composed of a 1x1 convolutional layer, a normalization layer, and an activation function connected sequentially.

[0039] Step four specifically involves: first, using a small batch of 1,000 spectrograms P0 as input, and the corresponding binary segmented spectrogram P1 as the learning label, training the Unet convolutional neural network using a hybrid loss function combining Focal Loss and Dice Loss as the target for backpropagation calculation; then, adjusting the various parameter settings of the Unet convolutional neural network by analyzing the evaluation curve after training; after parameter tuning, using a large batch of 30,000 spectrograms P0 as input, and the corresponding binary segmented spectrogram P1 as the learning label, using a hybrid loss function combining Focal Loss and Dice Loss as the target for backpropagation calculation to complete the Unet convolutional neural network training.

[0040] Step five specifically involves:

[0041] 5.1) A CCD (charge-coupled device) photosensitive electronic imaging device is used to record the holographic interferogram generated by the interference between the object light wave and the reference light wave of the target for the test object resolution test. Then, the holographic interferogram is converted into a spectrum diagram as the test spectrum diagram P through a two-dimensional Fourier transform. 01 ;

[0042] 5.2) Test spectrum P 01The signal is a complex amplitude signal, containing both intensity and phase information. The test spectrum P is extracted. 01 The wrapped phase signal:

[0043] φ FFT =arctan{Im(P 01 ) / Re(P 01 )}

[0044] Where, φ FFT Re represents the wrapped phase signal of the spectrum P0, and Re() represents the extraction of the spectrum P. 01 The real part information, Im() represents the extraction of the spectrogram P 01 The imaginary part of the information;

[0045] 5.3) The wrapped phase signal is further unwrapped to obtain the unfolded phase. The position of the unfolded phase maximum value of the holographic interferogram is searched and used as the center coordinate (x1, y1) of the object image spectrum. The unfolded phase distribution shows an extreme value corresponding to the carrier frequency.

[0046] Step six specifically involves inputting the test spectrum P into the trained Unet convolutional neural network. 01 Obtain the corresponding binary segmentation map P of the spectrum. 11 Obtain the binary segmentation map P of the spectrum. 11 The centroid coordinates (x) of all foreground regions i ,y i ), calculate the centroid coordinates (x, y) of each foreground region. i ,y i The distance between the foreground region and the center coordinates (x1, y1) of the object's spectrum is used to determine the object's spectrum region. A binary segmentation mask P2 for the object's spectrum region is then established.

[0047] The distance calculation is as follows:

[0048]

[0049] Among them, L i This represents the distance from the centroid coordinates of each segmented foreground region to the center coordinates of the object image's spectral region, (x i ,y i (x1, y1) represents the centroid coordinates of the foreground region, and (x1, y1) represents the center coordinates of the image spectrum.

[0050] Step seven specifically involves: testing the spectrum P. 01In the above method, the intensity values ​​of all pixels within the background region of the binary segmentation mask P2 in the image spectrum region are set to 0 to obtain the image spectrum filter map P3. Finally, the three-dimensional shape map of the object under test is reconstructed by performing phase unwrapping and distortion compensation operations on the image spectrum filter map P3.

[0051] The object image spectrum filtering map P3 is obtained according to the following formula:

[0052]

[0053] Wherein, P3(x,y) represents each pixel of the object image spectrum filter map, P0(x,y) represents the corresponding pixel of spectrum map P0, and P2(x,y) represents the corresponding pixel of object image spectrum region binary segmentation mask P2.

[0054] This invention generates a simulated dataset of 30,000 pairs of spectrograms and binary spectrograms using an iterative threshold segmentation method that can automatically generate high-quality filtering windows but is time-consuming. Then, an improved Unet convolutional neural network model integrating a multi-scale feature fusion module and a bottleneck residual module is designed and trained. The trained neural network model is used to perform adaptive spatial filtering operations in digital holography, which is beneficial for achieving high-quality and rapid reconstruction of the three-dimensional shape of objects.

[0055] Compared with existing technologies, the beneficial effects of this invention are:

[0056] This invention generates tens of thousands of simulated datasets consisting of spectrograms and binary spectrograms using an iterative threshold segmentation method that can automatically generate high-quality filtering windows but is time-consuming. Then, an improved Unet convolutional neural network model that integrates a multi-scale feature fusion module and a bottleneck residual module is designed and trained. The final neural network model combines the excellent performance of the threshold iterative segmentation method with the fast processing capability of neural networks, which is beneficial for achieving high-quality and fast reconstruction of the three-dimensional shape of objects. Attached Figure Description

[0057] Figure 1 Here is a flowchart of the iterative threshold segmentation algorithm;

[0058] Figure 2 The flowchart shows a method for measuring the 3D shape of an object based on adaptive spatial filtering using a Unet convolutional neural network.

[0059] Figure 3 This describes the main process of automatically obtaining a spectrum segmentation map by simulating a holographic spectrum map using a computer and then using a threshold iteration method.

[0060] Figure 4 This is a schematic diagram of the overall architecture of a neural network;

[0061] Figure 5 This is a schematic diagram of the architecture of the multi-scale feature fusion module;

[0062] Figure 6 This is a schematic diagram of the bottleneck residual module architecture;

[0063] Figure 7 To compare the segmentation output results of holographic spectrograms with random shape, size, and position distribution, the threshold iteration method and the neural network model were used respectively.

[0064] Figure 8 For example, the test spectrum P 01 ;

[0065] Figure 9 Example: Spectral binary segmentation diagram P 11 ;

[0066] Figure 10 Example: Binary segmentation mask P2 for the object image spectral region;

[0067] Figure 11 Image spectrum filtering diagram P3 for the embodiment;

[0068] Figure 12 The image shows the result of the three-dimensional shape reconstruction of the object in the example. Detailed Implementation

[0069] The present invention will be further described below with reference to the accompanying drawings and embodiments.

[0070] The iterative threshold segmentation algorithm in this invention is as follows: Figure 1 As shown in the flowchart, the complete operation of the embodiment is as follows: Figure 2 As shown in the flowchart, the specific steps are as follows:

[0071] Step 1: Generate multiple random holographic interferograms through computer simulation, and convert them into spectrum diagrams P0.

[0072] Specifically:

[0073] 1.1) Objects with different three-dimensional morphological features are simulated using computer simulation. Based on the Fresnel diffraction principle, the object light signal of the laser passing through the simulated object and the reference light signal not passing through the object are calculated. Then, the object light signal and the reference light signal interfere at random angles on the same plane to generate a simulated holographic interference pattern.

[0074] I = |O| 2 +|R| 2 +OR * +O * R

[0075] 1.2) Convert the holographic interferogram into a spectrogram P0 using a two-dimensional Fourier transform:

[0076] P0 = FFT{I} = FFT{|O| 2}+FFT{|R| 2}+FFT{OR *}+FFT{O * R}

[0077] The specific process used in this embodiment is as follows: First, various simulated objects with different three-dimensional morphological features are simulated and constructed. Then, based on the mathematical model of Fresnel diffraction, the simulated laser light signal passing through the simulated object and the reference light signal not passing through the simulated object interfere at random angles on the recording plane to generate 30,000 different random simulated holographic interferograms. Then, Fourier transform is performed on the simulated holographic interferograms to obtain the spectrum P0.

[0078] Step 2: Perform iterative threshold segmentation on the spectrum P0 of each random holographic interferogram to obtain a binary spectrum segmentation map P1. Specifically:

[0079] 2.1) Pre-set the global threshold T0 and the critical area S to be deleted for the spectrogram P0, where the global threshold T0 is greater than 0 and less than 1;

[0080] 2.2) After applying mean filtering to the spectrum P0, iterative threshold segmentation is performed based on the global threshold T0 and the critical area S to be deleted to obtain the spectrum segmentation map;

[0081] The global threshold T0 is greater than 0 and less than 1. Specifically, the global threshold T0 of the spectrogram P0 is obtained using the OTSU threshold segmentation method as the initial value, and the critical area S to be deleted is set to 100.

[0082] In section 2.2), the threshold segmentation operation is as follows: iterative threshold segmentation is performed on the spectrogram P0, the intensity values ​​of pixels with values ​​less than the global threshold T0 are set to 0, and the intensity values ​​of the remaining pixels are set to 1, where 1 represents the foreground region and 0 represents the background region; then all foreground regions with areas smaller than the critical area S to be deleted are deleted, which can prevent some special small regions from affecting the iterative condition judgment.

[0083] 2.3) The computer automatically identifies and determines the number of foreground regions in the current spectral segmentation map:

[0084] (a) If the number of foreground regions in the first iteration is less than 3, it means that the initial threshold T0 is too small, causing the zero-order term, conjugate term and object spectrum to be connected together to form the same foreground region, making it impossible to extract the binary mask of the object spectrum region separately in subsequent iterations. Therefore, the threshold needs to be increased by 0.01 each time until the number of foreground regions identified is not less than 3, and the iteration stops. Record the final segmentation threshold T obtained at this time.

[0085] (b) If the number of foreground regions in the first iteration is greater than 3, this threshold T is directly recorded as the segmentation threshold. At this time, the spectral segmentation map contains the zero-order term, the conjugate term, the object spectrum, and some stray spectrum regions. Subsequently, the operation of extracting the binary mask of the object spectrum region can be realized separately.

[0086] (c) If the number of foreground regions is equal to 3 in the first iteration, reduce the global threshold T0 by 0.01 and return to step 2.2) for processing. Iterate continuously until the number of foreground regions is not equal to 3. Record the global threshold T0 of the penultimate iteration as the segmentation threshold T.

[0087] In this case, although the initial threshold T0 has successfully separated the zero-order term, conjugate term, and object spectrum, satisfying the condition for subsequently extracting the binary mask of the object spectrum region separately, it is not the maximum spectrum segmentation result at this point. By iteratively searching with continuously decreasing initial threshold, there are two scenarios: Scenario 1: Finding the critical threshold at which the zero-order term, conjugate term, and object spectrum are connected to synthesize the same foreground region, manifested as the number of identified foreground regions being less than 3; Scenario 2: Due to the iterative decrease of the segmentation threshold T, the area of ​​some stray spectrum regions continuously increases, exceeding the critical area S of the stray spectrum, manifested as the number of identified foreground regions being greater than 3.

[0088] Therefore, in (c), the iteration termination condition is that the number of identified foreground regions is not equal to 3, and after the iteration ends, the threshold of the previous iteration needs to be recorded as the final segmentation threshold T. This can ensure that the zero-order term, the conjugate term and the object image spectrum are not connected together to synthesize the same foreground region.

[0089] 2.4) Use the segmentation threshold T obtained in the final iteration to perform threshold segmentation on the spectrum map P0. Finally, obtain the area information of all foreground regions on the segmentation map, arrange them in descending order, take the third-ranked area as the critical area S1, delete all foreground regions with areas smaller than the critical area S1, and leave only the three regions with the largest area: the zero-level term, the conjugate term, and the object spectrum. Thus, the binary spectrum segmentation map P1 is obtained.

[0090] like Figure 3 The main process is shown from generating a holographic interferogram from a computer-simulated object to obtaining a spectral binary segmentation map through an iterative threshold segmentation algorithm. Figure 2 (c1) and Figure 2 (c2) represents two different distributions of the spectrum. Figure 2 (f1) and Figure 2 (f2) shows the segmentation result of leaving only three main spectral regions by searching for the optimal segmentation threshold, so as to facilitate the subsequent binary mask operation of extracting the spectral regions of the object separately.

[0091] Step 3: AsFigure 4 As shown, the constructed Unet convolutional neural network mainly consists of an encoder subnetwork, a decoder subnetwork, and a bridging path connecting the encoder and decoder subnetworks;

[0092] The encoding subnetwork contains four consecutive downsampling modules, each connected by a 2×2 max pooling operation. The decoding subnetwork contains four consecutive upsampling modules, each connected by a 2×2 upconvolution operation. The input of the first downsampling module is used as the input of the Unet convolutional neural network. The output of the fourth downsampling module is concatenated with the output of the fourth downsampling module itself after passing through a 2×2 max pooling operation, two consecutive 3×3 convolutional modules, and a 2×2 upconvolution operation. This concatenation is then used as the input to the first upsampling module. In the sampling module, the output of the first upsampling module, after undergoing a 2×2 upconvolution operation, is concatenated with the output of the third downsampling module and then input into the second upsampling module. The output of the second upsampling module, after undergoing a 2×2 upconvolution operation, is concatenated with the output of the second downsampling module and then input into the third upsampling module. The output of the third upsampling module, after undergoing a 2×2 upconvolution operation, is concatenated with the output of the first downsampling module and then input into the fourth upsampling module. The output of the fourth upsampling module, after being processed by a 1*1 convolutional layer, serves as the output of the Unet convolutional neural network.

[0093] After the spectrogram P0 is input into the Unet convolutional neural network, it is processed by four downsampling modules in the encoding subnetwork to obtain the image position and shape features after size reduction. The image position and shape features after size reduction are then processed by two 3×3 convolutional modules for further feature integration and removal of redundant parameters to prevent overfitting and improve the model's generalization ability, thereby obtaining automatically integrated image shape and position features. The automatically integrated image shape and position features are used as input to the decoding subnetwork, which uses four upsampling modules. Each upsampling module first upsamples the input feature map through a 2×2 upconvolution operation, then concatenates the upsampled feature map with the corresponding feature map from the downsampling module of the encoding subnetwork through a bridging path, and finally performs feature fusion through two 3×3 convolutional modules. After passing through the four upsampling modules in sequence, a 1×1 convolutional layer is used to process the final output binary segmentation map of the spectrogram.

[0094] Each downsampling module consists of a multi-scale feature fusion module and a bottleneck residual module connected in sequence; each upsampling module consists of two 3*3 convolutional modules connected in sequence.

[0095] like Figure 5As shown, the multi-scale feature fusion module includes five 3*3 convolutional modules. The input of the multi-scale feature fusion module is processed by only one 3*3 convolutional module to obtain the first feature. The input of the multi-scale feature fusion module is processed by two consecutive 3*3 convolutional modules to obtain the second feature. The input of the multi-scale feature fusion module is processed by three consecutive 3*3 convolutional modules to obtain the third feature. The first feature, the second feature, and the third feature are concatenated in sequence to obtain the comprehensive feature of image shape and position at multiple scales as the output of the multi-scale feature fusion module.

[0096] like Figure 6 As shown, the bottleneck residual module includes a 3*3 convolution module and two 1*1 convolution modules. The input of the bottleneck residual module is sequentially passed through the first 1*1 convolution module, the 3*3 convolution module, and the second 1*1 convolution module. The result of this process is added to the input of the bottleneck residual module. The result of the summation and then passing through the activation function is used as the output of the bottleneck residual module.

[0097] A 3x3 convolutional module consists of a 3x3 convolutional layer, a normalization layer, and an activation function connected in sequence.

[0098] A 1x1 convolutional module consists of a 1x1 convolutional layer, a normalization layer, and an activation function connected in sequence.

[0099] The specific process used in this embodiment is as follows: The neural network is built based on the open-source deep learning library TensorFlow framework. The network structure consists of an encoding sub-network, a symmetrical decoding sub-network, and a bridging path connecting the two networks. The encoding sub-network uses four downsampling modules. Each downsampling module consists of a multi-scale feature fusion module, a bottleneck residual module, and a 2×2 max pooling operation. Each 2×2 max pooling operation reduces the image size by a factor of 4. Then, two 3×3 convolutional layers and a Dropout layer are used for further feature integration and redundant parameter removal to prevent overfitting and improve the model's generalization ability. The output is used as the input to the decoding sub-network, which uses a symmetrical upsampling module. Each upsampling module first upsamples the feature map using a 2×2 upconvolution operation, increasing the image size by a factor of 4 each time. Then, the upsampled feature map is concatenated with the corresponding feature map from the downsampling path through the bridging path. Finally, feature fusion is performed through two 3×3 convolutional layers. After passing through four upsampling modules in sequence, a 1×1 convolutional layer is used to process the binary segmentation map of the final output spectrogram. The specific structure of the neural network is as follows: Figure 4 As shown.

[0100] The multi-scale fusion module automatically identifies features by using different convolution ranges simultaneously on the same input image and then merges the results. Here, the input image is sequentially passed through three 3×3 convolution operations, and the results are merged. This makes it possible for the model to handle complex inputs and enhance its adaptability to various fuzzy scales. It not only increases the receptive field but also improves the robustness of the neural network, showing good recognition performance for features of different scales. The bottleneck residual module, when the input image has 3N channels, sequentially passes it through a 1×1 convolutional layer with N kernels, a 3×3 convolutional layer with N kernels, and a 1×1 convolutional layer with 3N kernels. Finally, the convolution result is merged with the initial input image and output using the ReLU activation function. Its purpose is to solve the gradient vanishing and gradient exploding problems that occur in neural networks, while also accelerating the training speed of the neural network.

[0101] Step 4: First, use a small batch of 1,000 spectrograms P0 as input, and the corresponding binary segmentation spectrogram P1 as the learning label. Use a hybrid loss function, combining Focal Loss and Dice Loss, as the target for backpropagation calculation to train the neural network. Analyze the training evaluation curves and adjust the various parameters of the neural network. After parameter tuning, use a dataset of 30,000 pairs to perform a complete neural network training.

[0102] The specific process in this embodiment is as follows: A hybrid loss function, combining Focal Loss and Dice Loss, is used as the target for backpropagation calculation. 1,000 pairs of random spectrograms P0 and their corresponding binary segmented spectrograms P1 are used as the training dataset to train a Unet convolutional neural network with added multi-scale feature fusion and bottleneck residual modules. The curves of the training loss function and evaluation function changing with the training cycle are analyzed, and the various parameters of the neural network are adjusted. Parameter tuning is mainly done using the controlled variable method, adjusting one parameter at a time, and observing the stability of the training curve and the magnitude of the evaluation index after retraining. The parameters that need to be adjusted mainly include: activation function, optimization function, initial learning rate and its decay condition settings, training batch size, and training cycle. After parameter tuning, a complete neural network training is performed using a dataset of 30,000 pairs of data.

[0103] like Figure 7 The image shows a comparison of segmentation results processed by a threshold iteration method and a neural network model for holographic spectrograms with random shapes, sizes, and positional distributions. It can be seen that the segmented shapes of the two methods are roughly the same, except that the edges of the segmented image output by the neural network model are slightly blurred due to misidentification of some edge pixels in the foreground region. However, this error can be eliminated by a simple morphological dilation operation.

[0104] Step 5: Acquire the holographic interferogram of the object under test, and then convert it into a test spectrum P. 01 By testing the spectrum P 01 The phase information is used to determine the coordinates (x1, y1) of the center of the object image spectrum. Specifically:

[0105] 5.1) A CCD (charge-coupled device) photosensitive electronic imaging device is used to record the holographic interferogram generated by the interference between the object light wave and the reference light wave of the target. Then, the holographic interferogram is converted into a test spectrum P by a two-dimensional Fourier transform. 01 .

[0106] 5.2) The spectrum P0 is a complex amplitude signal, containing both intensity and phase information. Extract the phase signal encapsulated in the spectrum P0:

[0107] φ FFT =arctan{Im(P0) / Re(P0)}

[0108] 5.3) The wrapped phase signal is further unwrapped to obtain the unfolded phase. The position of the unfolded phase maximum value of the holographic interferogram is searched and used as the center coordinate (x1, y1) of the object image spectrum. The unfolded phase distribution shows an extreme value corresponding to the carrier frequency.

[0109] The specific process used in this embodiment is as follows: First, a holographic interferogram of the target object's resolution is captured using a CCD industrial camera. Then, a Fourier transform is performed on the holographic interferogram to obtain the test spectrum P. 01 ,like Figure 8 As shown. Then extract the test spectrum P. 01 The wrapped phase signal, after phase unwrapping, yields the true three-dimensional phase spectrum, which always exhibits extreme values ​​corresponding to the carrier frequency. The position of the maximum phase value in the spectrum phase diagram is recorded as the center coordinate (x1, y1) of the image spectrum.

[0110] Step Six: Input the test spectrum P into the trained neural network. 01 Obtain the spectral binary segmentation map P of the neural network output. 11 Obtain the centroid coordinates (x, y) of all foreground regions. i ,y i ), calculate (x) respectively i ,y i The distance between the foreground region and the center coordinates (x1, y1) of the image spectrum is calculated. The foreground region with the smallest distance is the image spectrum region. The binary segmentation mask P2 of the image spectrum region is extracted separately.

[0111]

[0112] The specific process used in this embodiment is as follows: First, the test spectrum P is... 01 The input is fed into the trained spectrum segmentation Unet network model to obtain the spectrum binary segmentation map P output by the neural network. 11 ,like Figure 9 As shown. Then, the centroid coordinates (x, y) of all foreground regions are obtained respectively. i ,y i ), and calculate (x) i ,y i The distance between the object's spectral center coordinates (x1, y1) and the foreground region with the smallest distance is the object's spectral region. Finally, the binary segmentation mask P2 of the object's spectral region is extracted separately, as shown below. Figure 10 As shown.

[0113] Step 7: Test the spectrum P 01 In step 1, the intensity values ​​of all pixels within the background region of the binary segmentation mask P2 are set to 0 to obtain the object's spectral filter map P3. Finally, a high-quality 3D topographic image of the object is reconstructed through phase unwrapping and distortion compensation operations.

[0114]

[0115] Wherein, P3(x,y) represents each pixel of the object image spectrum filter map, P0(x,y) represents the corresponding pixel of the spectrum map, and P2(x,y) represents the corresponding pixel of the object image spectrum region binary segmentation mask.

[0116] The specific process used in this embodiment is as follows: The computer iterates through each pixel of the binary segmentation mask P2 of the object's spectral region; each pixel's intensity value is either 0 or 1. When the intensity value of a pixel is 0, the intensity value of the corresponding pixel in the spectrum map P0 is set to 0; when the intensity value of a pixel is 1, the spectral information of the corresponding pixel in the spectrum map P0 remains unchanged. This achieves spectral spatial filtering of the spectrum map P0 using the binary segmentation mask P2 as a filtering window, thereby obtaining the object's filtered spectral map P3, as shown below. Figure 11 As shown in the figure. Finally, a high-quality 3D topographic image of the object is reconstructed through phase unwrapping and distortion compensation operations.

[0117] The phase unfolding result of this embodiment is as follows: Figure 12 As shown, the unfolded phase obtained by the present invention clearly shows the surface morphology of the object and has good uniformity, which confirms the effectiveness of the present invention.

[0118] This invention addresses the trade-off between processing speed and filtering effectiveness in adaptive spatial filtering methods based on image processing technology, where obtaining a good filtering window requires significant time for iterative search operations. A simulated dataset of 30,000 pairs of spectrograms and binary spectrogram segmentation maps was generated using an iterative threshold segmentation method that can automatically generate high-quality filtering windows but is time-consuming. Then, an improved Unet convolutional neural network model, integrating a multi-scale feature fusion module and a bottleneck residual module, was designed and trained. This trained neural network model was used for adaptive spatial filtering operations in digital holography, facilitating high-quality and rapid reconstruction of the three-dimensional shape of objects.

Claims

1. A method for measuring the three-dimensional shape of an object based on adaptive spatial filtering using a Unet convolutional neural network, characterized in that: Step 1: Generate multiple random holographic interferograms using computer simulation, and convert them all into spectrum diagrams P0; Step 2: Perform iterative threshold segmentation on the spectrum P0 of each random holographic interferogram to obtain the binary spectrum segmentation P1; In step two, the threshold segmentation operation specifically involves: Perform iterative threshold segmentation on the spectrogram P0, setting the intensity values ​​of pixels smaller than the global threshold T0 to 0 and the intensity values ​​of the remaining pixels to 1, where 1 represents the foreground region and 0 represents the background region; then delete all foreground regions whose area is smaller than the critical area S to be deleted. Step 3: Construct a Unet convolutional neural network that integrates a multi-scale feature fusion module and a bottleneck residual module; Step 4: Use the spectrogram P0 as the input to the Unet convolutional neural network and the corresponding binary segmentation map P1 as the learning label to train the Unet convolutional neural network. Step 5: Acquire the holographic interferogram of the object under test, and then convert it into a test spectrum P. 01 By testing the spectrum P 01 The phase information is used to determine the coordinates (x1, y1) of the center of the image spectrum; Step Six: Input the test spectrum P into the trained Unet convolutional neural network. 01 The output obtained is the test spectrum P. 01 Spectral binary segmentation P 11 Then, based on the center coordinates (x1, y1) of the object's spectrum, in the test spectrum P 01 Spectral binary segmentation P 11 Search and extract the object's spectral region individually in the foreground region to create a binary segmentation mask P2; Step 7: Using the binary segmentation mask P2 of the object image spectrum region as the filtering window, filter the test spectrum image P. 01 Spectral spatial filtering is performed to obtain the object image spectral filtering map P3. Finally, the three-dimensional morphology map of the object under test is reconstructed through phase unwrapping and distortion compensation operations.

2. The method for measuring the three-dimensional shape of an object based on adaptive spatial filtering using a Unet convolutional neural network according to claim 1, characterized in that: Step one specifically includes: 1.1) Objects with different three-dimensional morphological features are simulated using computer simulation. Based on the Fresnel diffraction principle, the object light signal of the laser passing through the simulated object and the reference light signal that does not pass through the simulated object are calculated. Then, the object light signal and the reference light signal interfere at random angles on the same plane to generate a simulated holographic interference pattern. I=|O| 2 +|R| 2 +OR * +O * R Where I represents the holographic interference signal, O is the object light signal formed by diffraction after the laser passes through the object, and R is the reference light signal where the laser does not pass through the object; 1.2) Convert the holographic interferogram into a spectrogram P0 using a two-dimensional Fourier transform: P0=FFT{I}=FFT{|O| 2 }+FFT{|R| 2 }+FFT{OR * }+FFT{O * R} Where FFT{} denotes the two-dimensional Fourier transform.

3. The method for measuring the three-dimensional shape of an object based on adaptive spatial filtering using a Unet convolutional neural network according to claim 1, characterized in that: Step two specifically involves: 2.1) Pre-set the global threshold T0 and the critical area S to be deleted for the spectrogram P0; 2.2) After applying mean filtering to the spectrum P0, iterative threshold segmentation is performed based on the global threshold T0 and the critical area S to be deleted to obtain the spectrum segmentation map; 2.3) The number of foreground regions in the spectral segmentation map obtained from the first threshold segmentation operation is identified by computer, and a judgment is made: If the number of foreground regions in the spectrum segmentation map obtained by the first threshold segmentation operation is less than 3, then the global threshold T0 is increased by 0.01 and returned to step 2.2) for processing, and the process is iterated until the number of foreground regions is not less than 3. The global threshold T0 of the last iteration is recorded as the segmentation threshold T. If the number of foreground regions in the spectrum segmentation map obtained by the first threshold segmentation operation is greater than 3, the current global threshold T0 is directly used as the segmentation threshold T. If the number of foreground regions in the spectrum segmentation map obtained by the first threshold segmentation operation is equal to 3, then the global threshold T0 is reduced by 0.01 and returned to step 2.2) for processing, and iterated continuously until the number of foreground regions is not equal to 3. The global threshold T0 of the penultimate iteration is recorded as the segmentation threshold T. 2.4) Use the segmentation threshold T to perform threshold segmentation on the spectrum map P0. Finally, sort all the foreground regions obtained after segmentation according to their area size, and keep only the three with the largest area, namely the zero-level term region, the conjugate term region and the object image spectrum region, so as to obtain the spectrum binary segmentation map P1.

4. The method for measuring the three-dimensional shape of an object based on adaptive spatial filtering using a Unet convolutional neural network according to claim 1, characterized in that: In step three, the Unet convolutional neural network mainly consists of an encoding subnetwork, a decoding subnetwork, and connections. The encoding subnetwork comprises four consecutive downsampling modules, each connected to the next by a 2×2 max pooling operation. The decoding subnetwork comprises four consecutive upsampling modules, each connected to the next by a 2×2 upconvolution operation. The output of the fourth downsampling module is concatenated with the output of the fourth downsampling module itself after passing through a 2×2 max pooling operation, two consecutive 3×3 convolution modules, and a 2×2 upconvolution operation. This concatenation is then fed into the first upsampling module. The output of the module is concatenated with the output of the third downsampling module after a 2×2 upconvolution operation and then input into the second upsampling module. The output of the second upsampling module is concatenated with the output of the second downsampling module after a 2×2 upconvolution operation and then input into the third upsampling module. The output of the third upsampling module is concatenated with the output of the first downsampling module after a 2×2 upconvolution operation and then input into the fourth upsampling module. The output of the fourth upsampling module is processed by a 1*1 convolutional layer and then used as the output of the Unet convolutional neural network.

5. The method for measuring the three-dimensional shape of an object based on adaptive spatial filtering using a Unet convolutional neural network according to claim 4, characterized in that: Each downsampling module consists of a multi-scale feature fusion module and a bottleneck residual module connected in sequence; each upsampling module consists of two 3*3 convolution modules connected in sequence; the multi-scale feature fusion module includes five 3*3 convolution modules. The input of the multi-scale feature fusion module passes through only one 3*3 convolution module to obtain the first feature, the input of the multi-scale feature fusion module passes through two consecutive 3*3 convolution modules to obtain the second feature, and the input of the multi-scale feature fusion module passes through three consecutive 3*3 convolution modules to obtain the third feature. The first feature, the second feature, and the third feature are concatenated in sequence to obtain the comprehensive feature of image shape and position at multiple scales as the output of the multi-scale feature fusion module; the bottleneck residual module includes one 3*3 convolution module and two 1*1 convolution modules. The input of the bottleneck residual module passes through the first 1*1 convolution module, the 3*3 convolution module, and the second 1*1 convolution module in sequence, and the result is added to the input of the bottleneck residual module. The result after the sum is passed through an activation function is the output of the bottleneck residual module.

6. The method for measuring the three-dimensional shape of an object based on adaptive spatial filtering using a Unet convolutional neural network according to claim 1, characterized in that: Step four specifically involves: first, using 1,000 spectrograms P0 as input and the corresponding binary segmentation spectrogram P1 as the learning label, training the Unet convolutional neural network using a hybrid loss function combining Focal Loss and Dice Loss as the target for backpropagation calculation; then, adjusting the various parameter settings of the Unet convolutional neural network by analyzing the evaluation curve after training; after parameter tuning, using 30,000 spectrograms P0 as input and the corresponding binary segmentation spectrogram P1 as the learning label, and using a hybrid loss function combining Focal Loss and Dice Loss as the target for backpropagation calculation to complete the Unet convolutional neural network training.

7. The method for measuring the three-dimensional shape of an object based on adaptive spatial filtering using a Unet convolutional neural network according to claim 1, characterized in that: Step five specifically involves: 5.1) A CCD photosensitive electronic imaging device is used to record the holographic interferogram generated by the mutual interference of the object light wave and the reference light wave of the target for the resolution test of the object under test. Then, the holographic interferogram is converted into a spectrum diagram as the test spectrum diagram P01 by two-dimensional Fourier transform. 5.2) Test spectrum P01 is a complex amplitude signal, containing both intensity and phase information. Extract the encapsulated phase signal from test spectrum P01: φ FFT =arctan{Im(P 01 ) / Re(P 01 )} Where, φ FFT Represents the wrapped phase signal of spectrum P0, Re() represents extracting the real part information of spectrum P01, and Im() represents extracting the imaginary part information of spectrum P01. 5.3) Obtain the unfolded phase by further unwrapping the wrapped phase signal, and use the position of the maximum value of the unfolded phase as the center coordinate (x1, y1) of the object image spectrum.

8. The method for measuring the three-dimensional shape of an object based on adaptive spatial filtering using a Unet convolutional neural network according to claim 1, characterized in that: Step six specifically involves: inputting the test spectrum map P01 into the trained Unet convolutional neural network to obtain the corresponding binary spectrum segmentation map P11; obtaining the centroid coordinates (xi, yi) of all foreground regions in the binary spectrum segmentation map P11; calculating the distance between the centroid coordinates (xi, yi) of each foreground region and the object spectrum center coordinates (x1, y1); taking the foreground region with the smallest distance as the object spectrum region; and establishing an object spectrum region binary segmentation mask P2 for the object spectrum region.

9. The method for measuring the three-dimensional shape of an object based on adaptive spatial filtering using a Unet convolutional neural network according to claim 1, characterized in that: Step seven specifically involves: setting the intensity values ​​of all pixels within the background region of the object image spectrum region binary segmentation mask P2 to 0 on the test spectrum map P01 to obtain the object image spectrum filter map P3; and finally reconstructing the three-dimensional shape map of the object under test by performing phase unwrapping and distortion compensation operations on the object image spectrum filter map P3.

Images