Precise structural member three-dimensional reconstruction method based on digital holographic reconstruction adversarial network

Abstract

The invention provides a precision structural member three-dimensional reconstruction method based on a digital holographic reconstruction adversarial network. The method comprises the following steps: completing two-dimensional image preprocessing; training a model of the digital holographic reconstruction adversarial network, converting a holographic image into a data matrix form, inputting the data matrix form into a generator, and inputting an image result generated by the generator and a label image corresponding to the holographic image into a discriminator at the same time to distinguish a network generation image and a real object field image; the loss function constraint network training is iterated along a set image style, and weight parameters of the network are updated; the generator and the discriminator learn in a mutual confrontation mode; and inputting the verification data set image and the test data set image into the trained digital holographic reconstruction adversarial network model in the step 2 to generate a reconstructed real three-dimensional image. According to the technical scheme, the technical problem that in the field of traditional holographic image three-dimensional reconstruction, a reconstruction algorithm often needs to depend on accurate reconstruction target three-dimensional parameters to serve as prior information is solved.

Description

Technical Field

[0001] The present invention relates to the technical field of holographic image three-dimensional reconstruction, and in particular to a method for three-dimensional reconstruction of precision structural parts based on a digital holographic reconstruction adversarial network. Background Art

[0002] As one of the core components used in aerospace, shipbuilding, electronic communications, biomedicine and high-precision machinery manufacturing, whether the production of precision structural parts can meet product indicators is related to the quality of the product. With the rapid development of imaging technology and manufacturing processes, the requirements for the processing quality of precision structural parts are becoming more and more stringent. Surface error, flatness, surface defects, etc. are important indicators for the quality standard inspection of ultra-thin precision structural parts, which have a serious impact on the performance of structural parts and product systems. For the inspection of ultra-thin precision structural parts, magnetic particle inspection, penetrant inspection, eddy current inspection, ultrasonic inspection, machine vision and other inspection technologies have been carried out to improve the product quality of ultra-thin precision structural parts.

[0003] With the continuous updating and iteration of digital image simulation algorithms and the widespread application of high-sensitivity optical signal detectors, digital holographic 3D reconstruction technology is gradually playing an important role in many fields such as medical diagnosis, aerospace navigation, industrial manufacturing, and security monitoring. However, in the field of traditional holographic image 3D reconstruction, reconstruction algorithms often need to rely on accurate 3D parameters of the reconstruction target as prior information. These algorithms use physical models to simplify the integration process to simulate reconstruction, but also face problems such as insufficient sampling, low resolution, and many iterations.

[0004] Traditional digital holographic 3D imaging technology requires accurate experimental parameters as prior information to reconstruct holographic images, and the 3D reconstruction of precision structural parts is achieved through the Fresnel transformation method, angular spectrum method and convolution method. The mapping relationship between the holographic image and the reconstructed 3D image can be achieved through a neural network, reducing the reliance of 3D reconstruction of holographic images on prior information and simplifying complex physical processes to simulate and reconstruct holographic images. Summary of the invention

[0005] The present invention provides a method for three-dimensional reconstruction of precision structural parts based on a digital holographic reconstruction adversarial network, which can solve the technical problem in the field of traditional three-dimensional reconstruction of holographic images that the reconstruction algorithm often needs to rely on accurate three-dimensional parameters of the reconstruction target as prior information.

[0006] The present invention provides a three-dimensional reconstruction method for precision structural parts based on a digital holographic reconstruction adversarial network. The three-dimensional reconstruction method for precision structural parts based on a digital holographic reconstruction adversarial network comprises: step one, a source image pair comprises a coherent image for reconstruction and a real object field image, image denoising and contrast enhancement, image feature extraction, image matching and image cropping are performed on the source image, and two-dimensional image preprocessing of a network model data set is completed, wherein the preprocessed image data set comprises a training data set image, a verification data set image and a test data set image; step two, based on the image data set samples preprocessed in step one, a model of a digital holographic reconstruction adversarial network is trained, and in the training stage of the network, an algorithm automatically calls the data set, converts the holographic image into a data matrix form and inputs it into a generator, and the image result generated by the generator and the label image corresponding to the holographic image are simultaneously input into a discriminator to distinguish between the network generated image and the real object field image; When the trained discriminator can correctly distinguish between generated data and labeled data, the feature image similarity results output by the discriminator will be used to guide the next round of training of the generator, guiding the generator to generate more realistic image data along the direction where the gradient of the loss function decreases fastest. As the training iterates, the image features of the holographic image are learned by the network; the loss function constrains the network training to iterate along the set image style and updates the weight parameters of the network; the generator and the discriminator learn in an adversarial manner and are trained in an alternating iterative manner until the discriminator cannot classify whether the image is generated by the generator or the real object field image, thereby obtaining the final trained digital holographic reconstruction adversarial network model; in step three, the verification data set images and the test data set images are input into the digital holographic reconstruction adversarial network model that has been trained in step two to generate a reconstructed real three-dimensional image and complete the three-dimensional reconstruction of precision structural parts based on the digital holographic reconstruction adversarial network.

[0007] Furthermore, the adversarial network model for digital holographic reconstruction includes a generator based on a U-net architecture and a discriminator based on a Markov Patch-GAN architecture. Given an original domain X (coherent image) and a generated domain Y (real object field image), the generator attempts to achieve a mapping G:X→Y through iteration and a maximum-minimization game with the adversarial discriminator. The generator uses an Encoder-Decoder algorithm to design a generator structure, including a downsampling unit and a symmetrical upsampling unit. The downsampling part of the network uses a stack of VGG convolutional layers and pooling layers. There are jump connections between the mirror layers of the network, which can record and transmit part of the spatial information. The upsampling layer fuses the acquired features with the high-frequency information transmitted by the jump connection to complete the final network prediction. When the given input is a real object field image, the discriminator calculates and outputs a real number close to 1 through classification, indicating that the input is real. Conversely, when the given input is an image generated by the generator, the discriminator outputs a real number close to 0, indicating that the input is false.

[0008] Furthermore, in step one, the image denoising and contrast enhancement are performed on the source image, specifically including: using the zero mean method to process and remove the DC component of the image spatial frequency domain, and for the label image from which the DC noise has been removed, using grayscale stretching and gamma correction to achieve contrast enhancement and dark field detail restoration. By denoising and contrast enhancement of the source image, the extraction rate of image feature points can be effectively improved.

[0009] Furthermore, in step 1, the image feature extraction of the source image specifically includes: the Hessian matrix eigenvalue of the (x, y) coordinate value of any point in the image space represents the anisotropy of the image change in the direction indicated by the two eigenvectors, and the image is transformed, and the transformation graph is composed of the approximate value of the Hessian matrix determinant of each pixel in the original image. When extracting the local maximum or minimum value of the transformation graph, it can be determined whether the current point is brighter or darker than other points in the surrounding neighborhood, thereby completing the image feature point extraction.

[0010] Further, in step 1, image matching of the source image specifically includes: locating feature points by interpolating feature point proximity information, and on the basis of the located feature points, calculating the Haar wavelet transform of the pixels around the feature points in the x and y directions, and adding the transform values ​​in the x and y directions in a certain angle interval of the xy plane to form a vector, and the longest of all vectors (i.e., the largest x and y components) is the direction of the feature point. After the direction of the feature point is selected, the surrounding pixels need to use this direction as a reference to establish a descriptor. At this time, 55 pixels are used as a sub-region, and the range of 2020 pixels around the feature point is taken, a total of 16 sub-regions, and the sum of the Haar wavelet transforms in the x and y directions in the sub-regions Σdx, ΣdyΣdx, Σdy and the sum of their vector lengths Σ|dx|, Σ|dy|Σ|dx|, Σ|dy|, a total of four values, can generate a 64-dimensional descriptor. Use BurteForceMatcher to match feature points, and sort them according to Euclidean distance to complete the image matching.

[0011] Furthermore, in step 2, the mapping relationship between the input image and the generated image is: G:X(Y), where X represents the input image in the original domain, Y represents the desired domain, and Z represents the distribution of random noise; the standard loss function For E X,Y [logD(X,Y)]+E X,Z [log(1-D(X,G(X,Z)))], where E X,Y [logD(X,Y)] is the probability that the discriminator determines the real data as the real data, E X,Z[log(1-D(X,G(X,Z)))] is the probability that the discriminator determines false data as false data; under the constraints of the generative network adversarial network principle, the generator attempts to minimize The discriminator tries to maximize E X,Y Represents the expected value of the real data sample, E X,Z Represents the expected value of all generated fake samples G(X,Z); from the image content and overall image similarity The loss function is optimized in two directions to improve the network reconstruction effect; the optimized loss function is Among them, λ1=0.7 and λ2=0.3 determine the weight ratio between different items in network training.

[0012] According to another aspect of the present invention, a three-dimensional reconstruction system for precision structural parts based on a digital holographic reconstruction adversarial network is provided. The three-dimensional reconstruction system for precision structural parts based on a digital holographic reconstruction adversarial network uses the three-dimensional reconstruction method for precision structural parts based on a digital holographic reconstruction adversarial network as described above to perform three-dimensional reconstruction of precision structural parts.

[0013] According to another aspect of the present invention, there is provided a computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the method for three-dimensional reconstruction of precision structural parts based on a digital holographic reconstruction adversarial network as described above.

[0014] According to another aspect of the present invention, a computer-readable storage medium is provided, which stores a computer program. When the computer program is executed by a processor, the steps of the method for three-dimensional reconstruction of precision structural parts based on a digital holographic reconstruction adversarial network are implemented as described above.

[0015] By applying the technical solution of the present invention, a method for three-dimensional reconstruction of precision structural parts based on a digital holographic reconstruction adversarial network is provided. Compared with the digital holographic image reconstruction algorithm based on the physical model, the data holographic reconstruction adversarial network does not require accurate prior information. Compared with the reconstruction scheme based on the generative adversarial network, the digital holographic reconstruction adversarial network uses a simple model with fewer parameters for fast reasoning. On this basis, starting from the more dimensional influencing factors of the reconstructed image, the loss function for holographic reconstruction is optimized. While taking into account the training model with a small memory storage capacity, the network model achieves faster and more efficient convergence. Secondly, the digital holographic reconstruction adversarial network technology plays an important role in the detection and analysis of precision structural parts. The three-dimensional structural reconstruction of the micron-sized detection target helps the structural inspector to more intuitively detect whether the overall morphology of the structural part meets the production standards. Based on the training of the data set paired with the optical coherence image and the real object field, the optical coherence image can restore the micron-level change in the thickness of the structural part through the phase change of the imaging beam coherence, so as to realize the detection of ultra-thin precision structural parts. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] The included drawings are used to provide a further understanding of the embodiments of the present invention, which constitute a part of the specification, are used to illustrate the embodiments of the present invention, and together with the text description, explain the principles of the present invention. Obviously, the drawings in the following description are only some embodiments of the present invention, and for ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0017] Figure 1 It is a flow chart of the overall scheme of the present invention.

[0018] Figure 2 It is the experimental scheme for acquiring dataset images.

[0019] Figure 3 It is a schematic diagram of dataset image preprocessing.

[0020] Figure 4 It is a structural diagram of the digital holographic reconstruction adversarial network model.

[0021] Figure 5 This is a comparison chart of the results of digital holographic reconstruction. DETAILED DESCRIPTION

[0022] It should be noted that, in the absence of conflict, the embodiments in this application and the features in the embodiments can be combined with each other. The technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. The following description of at least one exemplary embodiment is actually only illustrative and is by no means intended to limit the present invention and its application or use. Based on the embodiments in the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0023] It should be noted that the terms used herein are only for describing specific embodiments and are not intended to limit the exemplary embodiments according to the present application. As used herein, unless the context clearly indicates otherwise, the singular form is also intended to include the plural form. In addition, it should be understood that when the terms "comprise" and / or "include" are used in this specification, it indicates the presence of features, steps, operations, devices, components and / or combinations thereof.

[0024] Unless otherwise specifically stated, the relative arrangement of the parts and steps described in these embodiments, numerical expressions and numerical values ​​do not limit the scope of the present invention. At the same time, it should be understood that, for ease of description, the sizes of the various parts shown in the accompanying drawings are not drawn according to the actual proportional relationship. The technology, method and equipment known to ordinary technicians in the relevant field may not be discussed in detail, but in appropriate cases, the technology, method and equipment should be regarded as a part of the authorization specification. In all examples shown and discussed here, any specific value should be interpreted as being merely exemplary, rather than as a limitation. Therefore, other examples of exemplary embodiments may have different values. It should be noted that similar reference numerals and letters represent similar items in the following drawings, so once a certain item is defined in an accompanying drawing, it does not need to be further discussed in subsequent drawings.

[0025] like Figures 1 to 5As shown, according to a specific embodiment of the present invention, a method for three-dimensional reconstruction of precision structural parts based on a digital holographic reconstruction adversarial network is provided, and the method for three-dimensional reconstruction of precision structural parts based on a digital holographic reconstruction adversarial network includes: step one, the source image pair includes a coherent image for reconstruction and a real object field image, the source image is subjected to image denoising and contrast enhancement, image feature extraction, image matching and image cropping, and two-dimensional image preprocessing of the network model data set is completed, and the preprocessed image data set includes a training data set image, a verification data set image and a test data set image; step two, based on the image data set samples preprocessed in step one, the model of the digital holographic reconstruction adversarial network is trained, and in the training stage of the network, the algorithm automatically calls the data set, converts the holographic image into a data matrix form and inputs it into the generator, and the image result generated by the generator and the label image corresponding to the holographic image are simultaneously input into the discriminator to distinguish between the network generated image and the real real object field image; when the trained discriminator can correctly distinguish between generated data and labeled data, the feature image similarity results output by the discriminator will be used to guide the next round of training of the generator, guiding the generator to generate more realistic image data along the direction where the gradient of the loss function decreases fastest. As the training iterates, the image features of the holographic image are learned by the network; the loss function constrains the network training to iterate along the set image style and updates the weight parameters of the network; the generator and the discriminator learn in an adversarial manner and are trained in an alternating iterative manner until the discriminator cannot classify whether the image is generated by the generator or the real object field image, and the final trained digital holographic reconstruction adversarial network model is obtained; step three, input the verification data set images and the test data set images into the digital holographic reconstruction adversarial network model that has been trained in step two, generate a reconstructed real three-dimensional image, and complete the three-dimensional reconstruction of precision structural parts based on the digital holographic reconstruction adversarial network.

[0026] By applying this configuration, a method for 3D reconstruction of precision structural parts based on digital holographic reconstruction adversarial network is provided. Compared with the digital holographic image reconstruction algorithm based on physical model, the data holographic reconstruction adversarial network does not require accurate prior information. Compared with the reconstruction scheme based on generative adversarial network, the digital holographic reconstruction adversarial network uses a simple model with fewer parameters for fast reasoning. On this basis, starting from the factors affecting more dimensions of the reconstructed image, the loss function for holographic reconstruction is optimized. While taking into account the training model with small memory storage capacity, the network model achieves faster and more efficient convergence. Secondly, the digital holographic reconstruction adversarial network technology plays an important role in the detection and analysis of precision structural parts. The 3D structural reconstruction of the micron-sized detection target helps the structural inspector to more intuitively detect whether the overall morphology of the structural part meets the production standards. Based on the training of the data set of optical coherence image and real object field pairing, the optical coherence image can restore the micron-level change in the thickness of the structural part through the phase change of the imaging beam coherence, so as to realize the detection of ultra-thin precision structural parts.

[0027] Furthermore, in the present invention, the adversarial network model for digital holographic reconstruction includes a generator based on a U-net architecture and a discriminator based on a Markov Patch-GAN architecture. Given an original domain X (coherent image) and a generated domain Y (real object field image), the generator attempts to achieve a mapping G:X→Y through iteration and a maximum-minimization game of the adversarial discriminator. The generator uses an Encoder-Decoder algorithm to design a generator structure, including a downsampling unit and a symmetrical upsampling unit. The downsampling part of the network uses a stack of convolutional layers and pooling layers of VGG. There are jump connections between the mirror layers of the network, which can record and transmit part of the spatial information. The upsampling layer fuses the acquired features with the high-frequency information transmitted by the jump connection to complete the final network prediction. When the given input is a real object field image, the discriminator calculates and outputs a real number close to 1 through classification, indicating that the input is real; conversely, when the given input is an image generated by the generator, the discriminator outputs a real number close to 0, indicating that the input is false.

[0028] In addition, in step one, the image denoising and contrast enhancement of the source image specifically include: using the zero mean method to process and remove the DC component of the image spatial frequency domain, and for the label image with DC noise removed, using grayscale stretching and gamma correction to achieve contrast enhancement and dark field detail restoration. By denoising and contrast enhancement of the source image, the extraction rate of image feature points can be effectively improved.

[0029] In step 1, the image feature extraction of the source image specifically includes: the Hessian matrix eigenvalue of the (x, y) coordinate value of any point in the image space represents the anisotropy of the image change in the direction indicated by the two eigenvectors, and the image is transformed. The transformation map is composed of the approximate value of the Hessian matrix determinant of each pixel in the original image. When extracting the local maximum or minimum value of the transformation map, it can be determined whether the current point is brighter or darker than other points in the surrounding neighborhood, thereby completing the image feature point extraction.

[0030] In step 1, image matching of the source image specifically includes: locating feature points by interpolating feature point proximity information, and on the basis of the located feature points, calculating the Haar wavelet transform of the pixels around the feature points in the x and y directions, and adding the transformation values ​​in the x and y directions in a certain angle interval of the xy plane to form a vector. The longest vector (i.e., the largest x and y components) is the direction of the feature point. After the direction of the feature point is selected, the surrounding pixels need to use this direction as a reference to establish a descriptor. At this time, 55 pixels are used as a sub-region, and the range of 2020 pixels around the feature point is taken, a total of 16 sub-regions. The sum of the Haar wavelet transforms in the x and y directions in the sub-regions is calculated. Σdx, ΣdyΣdx, Σdy and the sum of their vector lengths Σ|dx|, Σ|dy|Σ|dx|, Σ|dy|, a total of four values, a total of a 64-dimensional descriptor can be generated. Use BurteForceMatcher to match feature points, and sort them according to Euclidean distance to complete the image matching.

[0031] Furthermore, in step 2, the mapping relationship between the input image and the generated image is: G:X(Y), where X represents the input image in the original domain, Y represents the desired domain, and Z represents the distribution of random noise; the standard loss function For E X,Y [logD(X,Y)]+E X,Z [log(1-D(X,G(X,Z)))], where E X,Y [logD(X,Y)] is the probability that the discriminator determines the real data as the real data, E X,Z [log(1-D(X,G(X,Z)))] is the probability that the discriminator determines false data as false data; under the constraints of the generative network adversarial network principle, the generator attempts to minimize The discriminator tries to maximize E X,Y Represents the expected value of the real data sample, E X,Z Represents the expected value of all generated fake samples G(X,Z); from the image content and overall image similarity The loss function is optimized in two directions to improve the network reconstruction effect; the optimized loss function is Among them, λ1=0.7 and λ2=0.3 determine the weight ratio between different items in network training.

[0032] According to another aspect of the present invention, a three-dimensional reconstruction system for precision structural parts based on a digital holographic reconstruction adversarial network is provided. The three-dimensional reconstruction system for precision structural parts based on a digital holographic reconstruction adversarial network uses the three-dimensional reconstruction method for precision structural parts based on a digital holographic reconstruction adversarial network as described above to perform three-dimensional reconstruction of precision structural parts.

[0033] By applying this configuration, a three-dimensional reconstruction system for precision structural parts based on a digital holographic reconstruction adversarial network is provided. Compared with the digital holographic image reconstruction algorithm based on the physical model, the method adopted by the system does not require precise prior information for the data holographic reconstruction adversarial network. Compared with the reconstruction scheme based on the generative adversarial network, the digital holographic reconstruction adversarial network uses a simple model with fewer parameters for fast reasoning. On this basis, starting from the factors affecting more dimensions of the reconstructed image, the loss function for holographic reconstruction is optimized. While taking into account the training model with a small memory storage capacity, the network model achieves faster and more efficient convergence. Secondly, the digital holographic reconstruction adversarial network technology plays an important role in the detection and analysis of precision structural parts. The three-dimensional structural reconstruction of the micron-sized detection target helps the structural inspector to more intuitively detect whether the overall morphology of the structural part meets the production standards. Based on the training of the data set of optical coherence images paired with the real object field, the optical coherence image can restore the micron-level changes in the thickness of the structural part through the phase change of the imaging beam coherence, so as to realize the detection of ultra-thin precision structural parts.

[0034] According to another aspect of the present invention, there is provided a computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the method for three-dimensional reconstruction of precision structural parts based on a digital holographic reconstruction adversarial network as described above.

[0035] According to another aspect of the present invention, a computer-readable storage medium is provided, which stores a computer program. When the computer program is executed by a processor, the steps of the method for three-dimensional reconstruction of precision structural parts based on a digital holographic reconstruction adversarial network are implemented as described above.

[0036] In order to further understand the present invention, the following Figures 1 to 5 The three-dimensional reconstruction method of precision structural parts based on digital holographic reconstruction adversarial network provided by the present invention is described in detail.

[0037] In order to overcome the above problems existing in the prior art, the present invention proposes a solution of digital holographic reconstruction adversarial network based on the generative-discriminative model to meet the detection requirements of ultra-thin precision structural parts. This solution effectively solves the problem of three-dimensional reconstruction of holographic images of ultra-thin precision structural parts by utilizing the model advantages of the generative adversarial model.

[0038] The technical solution adopted by the patent of this invention is as follows:

[0039] The present invention provides a three-dimensional reconstruction of a digital holographic image for an ultra-thin precision structural part, which comprises the following steps:

[0040] The present invention includes a digital holographic reconstruction adversarial network model architecture design applied to ultra-thin structural parts:

[0041] The 3D reconstruction network model of holographic images based on the digital holographic reconstruction adversarial network includes a generator and a discriminator, including the Encoder-Decoder algorithm and the image receptive field Patch unit. The loss function of the network includes image content loss term, local texture style loss term and overall image similarity loss term.

[0042] Step 1: 2D image preprocessing of network model dataset

[0043] The image source is an image pair consisting of an optical coherence image of the reconstructed target and a real object field image. After being processed by algorithms such as image denoising and feature matching, the source image pair is cropped into multiple paired images of size 256 × 256 × 3. All the original image pairs are processed as above to form an image dataset, and are randomly allocated for training, verification, and testing in a ratio of 8:1:1.

[0044] Step 2: Model training of digital holographic reconstruction adversarial network

[0045] The output of the model is a reconstructed three-dimensional ultra-thin precision structural component image. The three-dimensional ultra-thin precision structural component image generated by the generative model and the real object field image are respectively input into the discriminant model, and the similarity of the two images is calculated. When the given input is a real object field image, the discriminator will output a real number close to 1, indicating that the input is real; conversely, when the given input is an image generated by the generator, the discriminator will output a real number close to 0, indicating that the input is false.

[0046] The generative model and the discriminative model are learned in an adversarial manner and trained in an alternating iterative manner until the discriminator cannot classify whether the image is generated by the generator or the real object field image, thus obtaining the final trained digital holographic reconstruction adversarial network model.

[0047] Step 3: Image reconstruction of digital holographic reconstruction adversarial network model

[0048] The test and verification dataset images completed in step 2 are input into the trained generation network in step 3, and the output is the reconstructed real 3D image.

[0049] First, compared with the digital holographic image reconstruction algorithm based on physical models, the data holographic reconstruction adversarial network does not require precise prior information. Compared with the reconstruction scheme based on the generative adversarial network, the digital holographic reconstruction adversarial network uses a simple model with fewer parameters for fast reasoning. On this basis, starting from the factors affecting the reconstructed image in more dimensions, the loss function for holographic reconstruction is optimized. This network model achieves faster and more efficient convergence while taking into account the training model with smaller memory storage.

[0050] Secondly, digital holographic reconstruction adversarial network technology plays an important role in the detection and analysis of precision structural parts. The three-dimensional structural reconstruction of micron-sized detection targets helps structural inspectors to more intuitively detect whether the overall shape of the structural parts meets the production standards. Based on the training of the data set of optical coherence images paired with real object fields, optical coherence images can restore the micron-level changes in the thickness of the structural parts through the phase changes of the coherent imaging beam, thereby realizing the detection of ultra-thin precision structural parts.

[0051] To facilitate the description of the technical means and use process of the present invention, the technical solution of the present invention is further described below in conjunction with the accompanying drawings and specific examples.

[0052] The digital holographic reconstruction adversarial network model includes a generator based on the U-net architecture and a discriminator based on the MarkovPatch-GAN architecture.

[0053] As the generator of the adversarial network architecture for digital holographic reconstruction, given the original domain X (coherent image) and the generated domain Y (real object field image), the generator attempts to achieve the mapping G: X→Y through iteration and the maximum minimization game of the adversarial discriminator. The generator uses the Encoder-Decoder algorithm to design the generator structure, including downsampling units and symmetrical upsampling units, such as Figure 3 As shown in the figure. In the digital holographic reconstruction adversarial network model, the preprocessed coherent image is input into the generative network. The downsampling part of the network adopts a stacking method of convolutional layers and pooling layers similar to VGG, which reduces the network parameters and realizes more image feature extraction. There are jump connections between the mirror layers of the network, which can record and transmit part of the spatial information, which is very effective in image-to-image conversion and image quality enhancement. The upsampling layer fuses the acquired features with the high-frequency information transmitted by the jump connection to complete the final network prediction. In addition, each layer is also set with a 4×4 two-dimensional convolution filter unit, a Leaky-ReLU nonlinear unit, and a batch normalization unit.

[0054] For the network's discriminator, the present invention optimizes the classic Markov Patch-GAN architecture. The optimized architecture can capture more image detail information in capturing high-frequency features such as local texture and image style. When the given input is a real object field image, the discriminator calculates through classification and outputs a real number close to 1, indicating that the input is real; conversely, when the given input is an image generated by the generator, the discriminator outputs a real number close to 0, indicating that the input is false. Its structure is essentially a classification model. Four convolutional layers convert the input image into a 16×16×1 output, which represents the average effective response of the discriminator. Each layer of the discriminator has a similar structure to the generator, using a 3×3 convolution filter, Leaky-ReLU nonlinearity, and batch size (BN) to distinguish between samples generated by the generator or labeled samples.

[0055] like Figure 1 As shown, a method for 3D reconstruction of ultra-thin precision structural parts images based on a generative adversarial network comprises the following steps:

[0056] Step 1: Dataset acquisition and preprocessing

[0057] like Figure 2 (c) The source image pair includes the coherent image for reconstruction and the real object field image. These two images will be used as the input image and label image in the training of the generative adversarial network. The coherent image and the real object field image are obtained using an optimized Mach-Zendel optical system to obtain part of the data set, see Figure 2 (a). Figure 2 As shown in (b), micron-level precision structural parts and USAF 1951 are used as examples to demonstrate the process of acquiring precision structural parts datasets.

[0058] Before inputting the image into the reconstruction network model, in order to improve the reconstruction effect of the model, the original image needs to be processed. Taking the USAF resolution plate as the target instance for reconstruction, the preprocessing process of the source image mainly includes three steps: image denoising and contrast enhancement, image feature extraction, image matching and image cropping. The specific process is as follows: Figure 3 shown.

[0059] (1) In order to avoid the image noise being amplified as the generative adversarial network model is trained, which further causes image distortion, the present invention uses a zero-mean method to process and remove the DC component in the image spatial frequency domain, such as Figure 3 As shown in (b). For the label image with DC noise removed, the present invention uses grayscale stretching and gamma correction to achieve contrast enhancement and dark field detail recovery, as shown in Figure 3 (d) and Figure 3As shown in (e), by denoising and contrast enhancement of the source image, the extraction rate of image feature points can be effectively improved.

[0060] (2) The calculation method based on the Hessian matrix at any point in the two-dimensional space of the image is that the Hessian matrix eigenvalue of the (x, y) coordinate value of any point in the image space represents the anisotropy of the image change in the direction indicated by the two eigenvectors. The image is transformed, and the transformation map is composed of the approximate value of the Hessian matrix determinant of each pixel in the original image. When extracting the local maximum or minimum of the transformation map, it can be determined whether the current point is brighter or darker than other points in the surrounding neighborhood, thereby completing the image feature point extraction.

[0061] (3) Based on step (2), the feature points are located by interpolating the neighboring information of the feature points. Based on the located feature points, the Haar wavelet transform of the pixels around the feature points in the x and y directions is calculated, and the transformation values ​​in the x and y directions are added in a certain angle interval in the xy plane to form a vector. The longest vector (i.e., the vector with the largest x and y components) is the direction of the feature point. After the direction of the feature point is selected, the surrounding pixels need to use this direction as a reference to establish a descriptor. At this time, 55 pixels are taken as a sub-region, and the range of 2020 pixels around the feature point is taken as a total of 16 sub-regions. The Haar wavelet transform sum Σdx, ΣdyΣdx, Σdy in the x and y directions (the parallel feature point direction is x and the perpendicular feature point direction is y) in the sub-region is calculated, and the sum of their vector lengths Σ|dx|, Σ|dy|Σ|dx|, Σ|dy| is a total of four values, which can generate a 64-dimensional descriptor. Use BurteForceMatcher to match feature points and sort them according to Euclidean distance to complete image matching.

[0062] During training, paired images are input into the adversarial network in pairs, while during validation and testing, real object field images are used to compare the reconstruction results with those generated by the network.

[0063] Step 2: Model training of digital holographic reconstruction adversarial network;

[0064] During the training phase of the network, the algorithm automatically calls the data set and converts the holographic image into a data matrix form and inputs it into the generation network. The image results generated by the generator and the label image corresponding to the holographic image are simultaneously input into the discriminator to distinguish the network-generated image from the real object field image. When the trained discriminator can correctly distinguish the generated data from the label data, the feature image similarity results output by the discriminator will be used to guide the next round of training of the generator, guiding the generator to generate more realistic image data along the direction where the gradient of the loss function decreases fastest. As the training iterates, the image features of the holographic image are learned by the network. The loss function constrains the network training to iterate along the set image style and updates the weight parameters of the network.

[0065] During the verification and testing phases, when the network's loss function value no longer decreases significantly, the updated network model weights and the test and verification subset images that have never been input into the network are input into the digital holographic adversarial network for prediction.

[0066] The mapping relationship between the input image and the generated image is: G:X(Y), where X represents the input image in the original domain, Y represents the desired domain, and Z represents the distribution of random noise. Standard loss function For E X,Y [logD(X,Y)]+E X,Z [log(1-D(X,G(X,Z)))], where E X,Y [logD(X,Y)] is the probability that the discriminator determines the real data as the real data, E X,Z [log(1-D(X,G(X,Z)))] is the probability that the discriminator determines false data as false data. Under the constraints of the generative network adversarial network principle, the generator attempts to minimize The discriminator tries to maximize E X,Y Represents the expected value of the real data sample, E X,Z Represents the expected value of all generated fake samples G(X,Z). This study uses image content Overall image similarity The loss function is optimized in two directions to improve the network reconstruction effect. The optimized loss function is Among them, λ1=0.7 and λ2=0.3 determine the weight ratio between different items in network training. In order to avoid poor training results due to network convergence difficulties during training, the batch size of each iterative training is set to 8.

[0067] Step 3: Image reconstruction of digital holographic reconstruction adversarial network model

[0068] The test set and validation set images are input into the generative network trained in step 3. After feature extraction and reconstruction output, the reconstructed real 3D image is obtained, as shown in Figure 5 (a). The reconstructed 3D image and the label image ( Figure 5 (e)) to evaluate the image peak signal-to-noise ratio and image structure similarity. On this basis, and with the pix2pix network ( Figure 5 (b)), resnet network ( Figure 5 (c)) and angular spectrum reconstruction results ( Figure 5 (d)) for comparative analysis. Figure 5 The reconstruction results of pix2pix, resnet and angular spectrum algorithms show artifacts and noise, and low image contrast. In contrast, the reconstruction results of the digital holographic reconstruction adversarial network model are more accurate. Figure 5 In the three-dimensional view of (f), the three-dimensional morphological features of the target can be obtained more intuitively and accurately.

[0069] For ease of description, spatially relative terms such as "above", "above", "on the upper surface of", "above", etc. may be used here to describe the spatial positional relationship between a device or feature and other devices or features as shown in the figure. It should be understood that spatially relative terms are intended to include different orientations of the device in use or operation in addition to the orientation described in the figure. For example, if the device in the accompanying drawings is inverted, the device described as "above other devices or structures" or "above other devices or structures" will be positioned as "below other devices or structures" or "below other devices or structures". Thus, the exemplary term "above" can include both "above" and "below". The device can also be positioned in other different ways (rotated 90 degrees or in other orientations), and the spatially relative descriptions used here are interpreted accordingly.

[0070] In addition, it should be noted that the use of terms such as "first" and "second" to limit components is only for the convenience of distinguishing the corresponding components. If not otherwise stated, the above terms have no special meaning and therefore cannot be understood as limiting the scope of protection of the present invention.

[0071] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A method for three-dimensional reconstruction of precision structural parts based on digital holographic reconstruction adversarial network, characterized in that: The method for three-dimensional reconstruction of precision structural parts based on a digital holographic reconstruction adversarial network comprises: Step 1: The source image pair includes a coherent image for reconstruction and a real object field image, and the source image is subjected to image denoising and contrast enhancement, image feature extraction, image matching and image cropping to complete the two-dimensional image preprocessing of the network model data set. The preprocessed image data set includes a training data set image, a verification data set image and a test data set image; Step 2: Based on the sample of the image data set preprocessed in step 1, the model of the digital holographic reconstruction adversarial network is trained. In the training stage of the network, the algorithm automatically calls the data set, converts the holographic image into a data matrix form and inputs it into the generator. The image result generated by the generator and the label image corresponding to the holographic image are simultaneously input into the discriminator to distinguish the network-generated image from the real object field image. When the trained discriminator can correctly distinguish the generated data from the label data, the feature image similarity result output by the discriminator will be used to guide the next round of training of the generator, guiding the generator to generate more realistic image data along the direction where the gradient of the loss function decreases fastest. With the iteration of training, the image features of the holographic image are learned by the network. The loss function constrains the network training to iterate along the set image style and updates the weight parameters of the network. The generator and the discriminator learn in a mutually adversarial manner and train in an alternating iterative manner until the discriminator cannot classify whether it is an image generated by the generator or a real object field image, thereby obtaining a finally trained digital holographic reconstruction adversarial network model. Step three, input the verification data set image and the test data set image into the digital holographic reconstruction adversarial network model that has been trained in step two, generate a reconstructed real three-dimensional image, and complete the three-dimensional reconstruction of the precision structural parts based on the digital holographic reconstruction adversarial network.

2. The method for 3D reconstruction of precision structural parts based on digital holographic reconstruction adversarial network according to claim 1, characterized in that: The digital holographic reconstruction adversarial network model includes a generator based on a U-net architecture and a discriminator based on a MarkovPatch-GAN architecture. Given an original domain X (coherent image) and a generated domain Y (real object field image), the generator attempts to achieve a mapping G:X→Y through iteration and a maximum minimization game of the adversarial discriminator. The generator uses an Encoder-Decoder algorithm to design a generator structure, including a downsampling unit and a symmetrical upsampling unit. The downsampling part of the network uses a stacking method of VGG convolutional layers and pooling layers. There are jump connections between the image layers of the network, which can record and transmit part of the spatial information. The upsampling layer fuses the acquired features with the high-frequency information transmitted by the jump connection to complete the final network prediction. When the given input is a real object field image, the discriminator calculates and outputs a real number close to 1 through classification, indicating that the input is real; Conversely, when the given input is an image generated by the generator, the discriminator outputs a real number close to 0, indicating that the input is fake.

3. The method for 3D reconstruction of precision structural parts based on digital holographic reconstruction adversarial network according to claim 1, characterized in that: In the step one, the image denoising and contrast enhancement of the source image specifically includes: using the zero mean method to process and remove the DC component of the image spatial frequency domain, and for the label image with DC noise removed, using grayscale stretching and gamma correction to achieve contrast enhancement and dark field detail restoration. By denoising and contrast enhancement of the source image, the extraction rate of image feature points can be effectively improved.

4. The method for 3D reconstruction of precision structural parts based on digital holographic reconstruction adversarial network according to claim 3, characterized in that: In the step 1, the image feature extraction of the source image specifically includes: the Hessian matrix eigenvalue of the (x, y) coordinate value of any point in the image space represents the anisotropy of the image change in the direction indicated by the two eigenvectors, and the image is transformed, and the transformation graph is composed of the approximate value of the Hessian matrix determinant of each pixel in the original image. When extracting the local maximum or minimum value of the transformation graph, it can be determined whether the current point is brighter or darker than other points in the surrounding neighborhood, thereby completing the image feature point extraction.

5. The method for 3D reconstruction of precision structural parts based on digital holographic reconstruction adversarial network according to claim 4, characterized in that: In the step 1, the image matching of the source image specifically includes: locating the feature point by interpolating the feature point proximity information, and on the basis of the located feature point, calculating the Haar wavelet transform of the pixels around the feature point in the x and y directions, and adding the transformation values ​​in the x and y directions in a certain angle interval of the xy plane to form a vector, and the longest of all vectors (i.e., the largest x and y components) is the direction of the feature point. After the direction of the feature point is selected, the surrounding pixels need to use this direction as a reference to establish a descriptor. At this time, 55 pixels are used as a sub-region, and the range of 2020 pixels around the feature point is taken, a total of 16 sub-regions, and the sum of the Haar wavelet transforms in the x and y directions in the sub-regions Σdx, ΣdyΣdx, Σdy and the sum of their vector lengths Σ|dx|, Σ|dy|Σ|dx|, Σ|dy|, a total of four values, can generate a 64-dimensional descriptor. Use BurteForceMatcher to match the feature points, and sort them according to the Euclidean distance to complete the image matching.

6. The method for 3D reconstruction of precision structural parts based on digital holographic reconstruction adversarial network according to claim 5, characterized in that: In step 2, the mapping relationship between the input image and the generated image is: G:X(Y), where X represents the input image in the original domain, Y represents the desired domain, and Z represents the distribution of random noise; the standard loss function For E X,Y [logD(X,Y)]+E X,Z [log(1-D(X,G(X,Z)))], where E X,Y [logD(X,Y)] is the probability that the discriminator determines the real data as the real data, E X,Z [log(1-D(X,G(X,Z)))] is the probability that the discriminator determines false data as false data; under the constraints of the generative network adversarial network principle, the generator attempts to minimize The discriminator tries to maximize E X,Y Represents the expected value of the real data sample, E X,Z Represents the expected value of all generated fake samples G(X,Z); from the image content and overall image similarity The loss function is optimized in two directions to improve the network reconstruction effect; the optimized loss function is Among them, λ1=0.7 and λ2=0.3 determine the weight ratio between different items in network training.

7. A three-dimensional reconstruction system for precision structural parts based on digital holographic reconstruction adversarial network, characterized in that: The system for three-dimensional reconstruction of precision structural parts based on a digital holographic reconstruction adversarial network uses the method for three-dimensional reconstruction of precision structural parts based on a digital holographic reconstruction adversarial network as described in claims 1 to 6 to perform three-dimensional reconstruction of precision structural parts.

8. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: The processor executes the computer program to implement the steps of the method for three-dimensional reconstruction of precision structural parts based on a digital holographic reconstruction adversarial network as described in claims 1 to 6.

9. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the steps of the method for three-dimensional reconstruction of precision structural parts based on a digital holographic reconstruction adversarial network as described in claims 1 to 6 are implemented.

Images