Spectrum-image combined measurement equipment and system based on double networks

Abstract

The invention discloses a spectrum-image combined measurement device and system based on double networks, and relates to the field of wide spectrum coherent diffraction imaging, the spectrum-image combined measurement device comprises a light source, a first lens, an aperture diaphragm, a second lens, a third lens, an imaging object and a CMOS camera which are arranged along a light path direction, the CMOS camera receives a diffraction pattern and transmits the diffraction pattern to a processor, and the processor is used for processing the diffraction pattern. The processor reconstructs the received diffraction pattern by adopting a spectrum-image combined measurement method based on double networks, including collecting the diffraction pattern, and obtaining a fused diffraction pattern after high dynamic range fusion; a monochromatic diffraction pattern is collected, an actual diffraction distance is obtained in combination with the monochromatic diffraction propagation model, and an actual wide-spectrum physical propagation model is constructed based on the actual diffraction distance; and based on the fused diffraction pattern, reconstructing the image and the spectrum by using a spectrum-image joint reconstruction algorithm based on double networks to obtain a reconstructed diffraction pattern and spectrum. Spectrum calibration is not needed, the image and the spectrum can be reconstructed at the same time, and the difficulty of wide-spectrum diffraction imaging is greatly lowered.

Description

Technical Field

[0001] This invention relates to the field of broadband coherent diffraction imaging, and more particularly to a spectral-image joint measurement device and system based on a dual-network. Background Technology

[0002] The statements in this section are merely background information relating to this disclosure and do not necessarily constitute prior art.

[0003] Coherent diffraction imaging is an imaging technique that reconstructs the amplitude and phase of a target object from recorded light intensity information through iterative calculations. It boasts advantages such as high resolution, simple structure, and low cost, and has significant application value in fields such as quantitative phase reconstruction, high-resolution imaging, and X-ray imaging. Traditional coherent diffraction imaging requires extremely high coherence of the light source. However, in practice, due to uncertainties in the energy-time relationship and beam defects, the light source often faces certain coherence limitations. Traditional broadband reconstruction algorithms suffer from problems such as the need for pre-calibration of the spectrum and slow or even non-convergent convergence.

[0004] To address the aforementioned issues, existing technologies have proposed deep learning algorithms constrained by physical models. Deep learning typically employs two approaches: one is to train the network model using a dataset, enabling it to learn the mapping relationship between two data types, thereby achieving the goal of predicting new data; the other is to directly constrain the convergence direction of the network model parameters using a known physical model, thereby obtaining the reconstructed target. Given that the method of training through a dataset has poor generalization performance, and that diffraction imaging requires observing a wide variety of targets, constructing a large dataset is time-consuming and laborious. Therefore, the method of constraining by physical models is generally chosen.

[0005] However, existing deep learning algorithms constrained by physical models are only suitable for image reconstruction at monochromatic wavelengths. For continuous spectrum light sources, such as broadband, the diffraction propagation model requires appropriate discretization of the spectrum to minimize reconstruction time while maintaining reconstruction quality. During the processing of actual data, if the selected wavelength interval is too large, the ignored wavelength information is treated as noise during convergence, resulting in poor image quality. Conversely, if the selected wavelength interval is too small, it does not conform to the interference phenomena caused by similar wavelengths, also degrading the reconstruction results. Furthermore, traditional broadband reconstruction algorithms only reconstruct the image and require prior spectral calibration. Summary of the Invention

[0006] To overcome the shortcomings of the prior art, the present invention provides a spectrum-image joint measurement device and system based on dual networks, which eliminates the need for spectral calibration and can simultaneously reconstruct images and spectra, greatly simplifying the difficulty of broadband diffraction imaging.

[0007] To achieve the above objectives, one or more embodiments of the present invention provide the following technical solutions: In a first aspect, the present invention provides a spectrum-image joint measurement device based on dual networks, including a light source, a first lens, an aperture stop, a second lens, a third lens, an imaging object, and a CMOS camera arranged along the optical path direction; The light source is collimated after being emitted by the first lens, and then illuminates the aperture stop to form a probe spot. The second and third lenses form a 4f system to relay the probe spot to the imaging object. The light field carrying the information of the imaging object diffracts and propagates to generate a diffraction pattern. The CMOS camera receives the diffraction pattern and transmits it to the processor. The processor is configured to reconstruct the received diffraction pattern using a dual-network-based spectral-image joint measurement method to obtain the reconstructed diffraction pattern and spectrum.

[0008] A further technical solution, the specific steps of the dual-network-based spectral-image joint measurement method are as follows: Diffraction patterns were collected at various exposure times and then fused using high dynamic range fusion to obtain a fused diffraction pattern. Collect monochromatic diffraction patterns, combine them with a monochromatic diffraction propagation model to obtain the actual diffraction distance, and construct an actual broadband physical propagation model based on the actual diffraction distance; Based on the fused diffraction pattern, a dual-network-based spectral-image joint reconstruction algorithm is used to reconstruct the image and spectrum, obtaining the reconstructed diffraction pattern and spectrum. The processing procedure of the dual-network-based spectral-image joint reconstruction algorithm is as follows: The fused diffraction pattern and Gaussian distribution are input into the improved residual neural network and fully connected neural network for processing, respectively, and the two-dimensional image and spectrum of the fused diffraction pattern are output. The two-dimensional image and spectrum of the fused diffraction pattern are used as the initial conjecture functions of the imaging object and the transmission spectrum, respectively. The corresponding estimated broadband aliasing intensity map is obtained through the actual broadband physical propagation model. A loss function is constructed based on the estimated broadband aliasing intensity map and the experimentally measured broadband aliasing intensity map; Based on the corresponding loss function, the improved residual neural network and the fully connected neural network are iteratively optimized alternately. When the network converges, the reconstructed diffraction pattern and spectrum are output.

[0009] A further technical solution involves acquiring a monochromatic diffraction pattern and combining it with a monochromatic diffraction propagation model to obtain the actual diffraction distance, specifically as follows: The monochromatic diffraction pattern is input into an improved residual neural network for processing, and its two-dimensional image is output. The two-dimensional image of the monochromatic diffraction pattern is used as the initial conjecture function of the imaging object, and the diffraction estimate is obtained through the monochromatic diffraction propagation model. A loss function is constructed based on diffraction estimation and experimentally measured monochromatic diffraction patterns. The parameters of the improved residual neural network are optimized based on the loss function, and the reconstructed monochromatic diffraction pattern is output. The actual diffraction distance is calculated based on the reconstructed monochromatic diffraction pattern using a contrast autofocus algorithm.

[0010] A further technical solution involves constructing an actual broadband physical propagation model based on the actual diffraction distance using the Fresnel diffraction propagation model and partial coherent mode decomposition.

[0011] A further technical solution yields the following estimated broadband aliasing intensity map: The two-dimensional image of the fused diffraction pattern is used as the initial conjecture function for the imaged object, and the spectrum is used as the initial conjecture function for the transmission spectrum. Based on the initial conjecture function of the imaged object combined with the actual broadband physical propagation model, the estimated broadband aliasing intensity map of the image is obtained. Based on the initial conjecture function of the transmission spectrum combined with the actual broadband physical propagation model, the estimated broadband aliasing intensity map of the spectrum is obtained.

[0012] A further technical solution involves constructing a first loss function based on the estimated broadband aliasing intensity map of the image and the experimentally measured broadband aliasing intensity map, and then optimizing the parameters of the improved residual neural network based on the first loss function.

[0013] A further technical solution involves constructing a second loss function based on the estimated broadband aliasing intensity map of the spectrum and the experimentally measured broadband aliasing intensity map, and then optimizing the parameters of the fully connected neural network based on the second loss function.

[0014] A further technical solution involves alternating iterative optimization of the improved residual neural network and the fully connected neural network. When reconstructing the diffraction pattern, the parameters of the fully connected neural network are not optimized, and when reconstructing the spectrum, the parameters of the improved residual neural network are not optimized.

[0015] A further technical solution involves using the Laplace energy as a standard to determine whether the diffraction pattern is in positive optimization during the alternating iterative optimization process of the improved residual neural network and the fully connected neural network, and using the first-order difference mean as a standard to determine whether the transmission spectrum is in positive optimization.

[0016] Secondly, the present invention provides a spectral-image joint measurement system based on a dual-network architecture, comprising: The image acquisition module is configured to acquire diffraction patterns at various exposure times and obtain a fused diffraction pattern after high dynamic range fusion. The model calibration module is configured to: acquire a monochromatic diffraction pattern, combine it with a monochromatic diffraction propagation model to obtain the actual diffraction distance, and construct an actual broadband physical propagation model based on the actual diffraction distance; The algorithm reconstruction module is configured to: reconstruct the image and spectrum based on the fused diffraction pattern using a dual-network-based spectral-image joint reconstruction algorithm, thereby obtaining the reconstructed diffraction pattern and spectrum; the processing procedure of the dual-network-based spectral-image joint reconstruction algorithm is as follows: The fused diffraction pattern and Gaussian distribution are input into the improved residual neural network and fully connected neural network for processing, respectively, and the two-dimensional image and spectrum of the fused diffraction pattern are output. The two-dimensional image and spectrum of the fused diffraction pattern are used as the initial conjecture functions of the imaging object and the transmission spectrum, respectively. The corresponding estimated broadband aliasing intensity map is obtained through the actual broadband physical propagation model. A loss function is constructed based on the estimated broadband aliasing intensity map and the experimentally measured broadband aliasing intensity map; Based on the corresponding loss function, the improved residual neural network and the fully connected neural network are iteratively optimized alternately. When the network converges, the reconstructed diffraction pattern and spectrum are output.

[0017] The above one or more technical solutions have the following beneficial effects: Compared to traditional broadband diffraction imaging methods, the measurement device of this invention has a simple structure and extremely low cost. Unlike other spectral imaging systems, it does not require complex optical paths, expensive spectral cameras, piezoelectric displacement stages, etc. The measurement device of this invention only requires a monochromatic light source, three lenses, and a grayscale camera (only intensity response, no spectral resolution) to complete the acquisition of diffraction patterns. Furthermore, based on a dual-network spectral-image joint measurement method, it simultaneously performs joint reconstruction of images and spectra, greatly simplifying the difficulty of broadband diffraction imaging.

[0018] The measuring device of this invention uses a grayscale camera and achieves joint reconstruction of the target image and the transmission spectrum of the target to the light source under the condition of only a single broadband aliasing diffraction pattern (a broadband aliasing intensity pattern obtained by high dynamic range fusion of broadband diffraction patterns at a series of exposure times measured in experiments). Then, the image can be color synthesized based on the spectral information.

[0019] Compared to traditional methods of training network models using datasets, this invention optimizes network parameters using physical models, enabling measurements of various targets without the need to collect and construct large datasets. This saves costs and solves the problem of poor generalization in traditional methods. Furthermore, broadband imaging is generally applied in the X-ray band, where there are currently no suitable spectral cameras, making data construction virtually impossible. Even in the visible light range, while hyperspectral cameras exist, their cost is extremely high. The camera used in this invention is a grayscale camera, requiring only intensity response and not spectral resolution, resulting in extremely low cost. This invention provides a new research approach and technical method for broadband coherent diffraction imaging technology and similar dual-mode measurement problems. Besides applications in intensity image and spectral measurement, it also measures the phase of purely phase objects, achieving good results.

[0020] This invention improves the residual network by adding spatial attention and channel attention mechanisms, thereby enhancing the network's fitting performance and enabling the model to converge better and faster.

[0021] When determining the actual diffraction distance, a contrast autofocus algorithm is adopted, which determines a more accurate distance based on the contrast evaluation index and reduces labor costs. An adaptive selection of alternating iteration mode is designed, which adaptively determines whether to switch to another mode update based on the smoothness of the image or spectrum, reducing the consumption of computing resources.

[0022] This invention provides a new solution to problems similar to dual-mode measurements, such as measuring the phase distribution and refractive index at various wavelengths of a purely phase-type object (e.g., a glass etching plate) under known spectral conditions. Attached Figure Description

[0023] The accompanying drawings, which form part of this invention, are used to provide a further understanding of the invention. The illustrative embodiments of the invention and their descriptions are used to explain the invention and do not constitute an improper limitation of the invention.

[0024] Figure 1 This is a schematic diagram of a dual-network-based spectral-image joint measurement device according to an embodiment of the present invention; Figure 2 This is a conceptual diagram of a dual-network-based spectral-image joint measurement device according to an embodiment of the present invention; Figure 3 These are simulation experiment diagrams of embodiments of the present invention, where a is the real target, b is the unreconstructed result, and c is a spectral comparison diagram; Figure 4 These are reconstruction images of an object imaged under known spectral conditions according to an embodiment of the present invention, wherein a is a reconstruction image of a monochromatic diffraction pattern, and b is a reconstruction image of a broadband diffraction pattern under known spectral conditions; Figure 5This is a joint reconstruction image of the transmission spectrum and the image of the imaging object in an embodiment of the present invention, wherein a is a monochrome reconstruction image, b is an image reconstructed with aliasing, and c is a spectral contrast image; Figure 6 These are reconstruction images of an embodiment of the present invention under the condition of unknown probe aperture shape, wherein a is a real-shot image taken by the camera, b is a reconstruction image, and c is a spectral comparison image; Figure 7 This is a reconstruction image of a pure phase-type object under known spectral conditions according to an embodiment of the present invention, and a comparison image of the phase change curve at the corresponding position. Here, a is the phase image reconstruction image, b is the phase change at position A in a, and c is the phase change at position B in a. Figure 8 This is a flowchart of the spectral-image joint reconstruction algorithm based on dual networks in an embodiment of the present invention. Detailed Implementation

[0025] It should be noted that the following detailed descriptions are exemplary and intended to provide further illustration of the invention. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains.

[0026] It should be noted that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the scope of exemplary embodiments according to the invention. As used herein, the singular form is intended to include the plural form as well, unless the context clearly indicates otherwise. Furthermore, it should be understood that when the terms "comprising" and / or "including" are used in this specification, they indicate the presence of features, steps, operations, devices, components, and / or combinations thereof.

[0027] Where there is no conflict, the embodiments and features in the embodiments of the present invention can be combined with each other.

[0028] Example 1 like Figure 1 As shown, this embodiment discloses a spectrum-image joint measurement device based on dual networks, including a light source 1, a first lens 2, an aperture stop 3, a second lens 4, a third lens 5, an imaging object 6, and a CMOS camera 7 arranged along the optical path direction.

[0029] In this embodiment, the center points of the light source 1, the first lens 2, the aperture stop 3, the second lens 4, the third lens 5, the imaging object 6, and the CMOS camera 7 (grayscale camera) on each optical path are all on the same straight line.

[0030] like Figure 2 As shown, side a of the imaging object 6 is the object plane, side b is the detector plane, and the distance between the imaging object 6 and the CMOS camera 7 is the diffraction distance d.

[0031] After the light source 1 is emitted and collimated by the first lens 2 (achromatic lens), it illuminates the aperture stop 3 to form a probe spot. The second lens 4 and the third lens 5 form a 4f system to relay the probe spot to the imaging object 6. The light field carrying the imaging object information diffracts and propagates to generate a diffraction image. The CMOS camera 7 receives the diffraction image and transmits it to the processor. The processor is configured to reconstruct the received diffraction pattern using a dual-network-based spectral-image joint measurement method to obtain the reconstructed diffraction pattern and spectrum.

[0032] Specifically, light source 1 is a broadband light source. After being expanded and collimated by the first lens 2, the broadband light source is emitted onto the aperture stop 3 to form a probe spot. The second lens 4 and the third lens 5 form a 4f system to image the probe spot onto the imaging object 6. The probe spot, carrying the information of the imaging object, diffracts and propagates to the target surface of the camera 7 for acquisition. The acquisition module automatically changes the camera's exposure time. Each time the exposure time is changed, the camera acquires an image. After the diffraction images at each exposure time are acquired, the obtained data are fused using a high dynamic range fusion algorithm. The fused data is then substituted into the reconstruction algorithm for iterative updates. Finally, the transmission spectrum and the reconstructed image of the imaging object are obtained, and an accurate diffraction model is constructed.

[0033] Furthermore, the imaging object 6 is fixed on the mechanical displacement stage, with the direction of the light source as the z-axis. The mechanical displacement stage drives the imaging object to move along the x-axis and y-axis to perform imaging of various parts of the imaging object.

[0034] Furthermore, the high dynamic range fusion algorithm is based on existing algorithms. For example, given three diffraction patterns: the first with an exposure time of 10ms, the second 100ms, and the third 500ms, a threshold is set. Information above this threshold is considered low-frequency, and information below is considered high-frequency. The low-frequency information from the first diffraction pattern is filtered out using the threshold and multiplied by 10 to obtain the low-frequency component of the fusion pattern. Then, the information below the threshold is multiplied by 10 (because 100ms is 10 times 10ms) and added to the information below the threshold from the second diffraction pattern, divided by 2 to obtain the high-frequency component of the fusion pattern. The same operation is then performed: low-frequency information is multiplied by 5 (because 500ms is 10 times 100ms) to obtain the low-frequency component of the fusion pattern, and high-frequency information is multiplied by 5 and added to the high-frequency component of the third diffraction pattern, divided by 2 to obtain the high-frequency component of the fusion pattern. This process is repeated for multiple diffraction patterns.

[0035] In this embodiment, the spectral-image joint measurement method based on dual networks specifically includes the following steps: S1: Collect diffraction patterns at each exposure time, and obtain the fused diffraction pattern after high dynamic range fusion; In this embodiment, the camera exposure time is automatically adjusted and the diffraction images are automatically acquired using program control, thereby enabling the acquisition of a series of diffraction patterns.

[0036] Camera 7 is controlled to acquire diffraction images at the specified exposure time. During this time, the imaging target remains stationary. To ensure that as much information as possible is acquired from the diffraction images, a threshold is set. Information greater than the threshold is considered low-frequency information, and information less than the threshold is considered high-frequency information. When acquiring low-frequency information, the exposure time interval between two adjacent diffraction patterns is appropriately reduced; when acquiring high-frequency information, the exposure time interval between two adjacent diffraction patterns is appropriately increased. The camera records once each time the exposure time is changed, thus enabling the complete acquisition of diffraction image data.

[0037] In some implementations, during diffraction pattern acquisition, the sample is probed using a localized illumination spot (also called an illumination probe), obtained by passing an illumination beam through an aperture stop. The detector acquires the diffraction pattern of the illuminated area of ​​the sample. During algorithm iteration, the diffraction pattern data, obtained through highly dynamic fusion of a series of diffraction patterns, is converged. Because no components need to move during data acquisition, and the number of components is minimal, the device exhibits high compactness and stability. By changing the exposure time, a series of diffraction images with exposure times ranging from short to long can be obtained.

[0038] Preferably, to obtain higher quality reconstruction results, the measuring device should preferably use a regularly shaped aperture stop.

[0039] Preferably, the process of obtaining the aperture stop includes: testing the quality of reconstruction results under simulation conditions with existing equipment and computing power for various probe shapes; determining whether the probe shape is circular or square; performing 3D modeling using Solidworks; and obtaining a stainless steel aperture stop through machining. Known regular-shaped aperture stops are more conducive to network convergence, thus obtaining reconstruction results faster.

[0040] S2: Collect monochromatic diffraction patterns, combine them with the monochromatic diffraction propagation model to obtain the actual diffraction distance, and construct the actual broadband physical propagation model based on the actual diffraction distance using the Fresnel diffraction propagation model and partial coherent mode decomposition. In this embodiment, the PhysenNet algorithm based on physical model constraints is used to reconstruct the acquired monochromatic diffraction pattern, obtaining a reconstructed image of the monochromatic diffraction pattern. The processing procedure of the PhysenNet algorithm based on physical model constraints is as follows: the monochromatic diffraction pattern is input into an improved residual neural network for processing, and its two-dimensional image is output. This two-dimensional image serves as the initial conjecture function (the image to be reconstructed) for the imaging object. Diffraction estimation is obtained through a monochromatic diffraction propagation model. A loss function is constructed based on the diffraction estimation and the experimentally measured monochromatic diffraction pattern. The parameters of the improved residual neural network are optimized based on the loss function, so that the output of the improved residual network gradually approaches the imaging object. As the network model converges, the reconstructed result of the monochromatic diffraction pattern is output, that is, the final output of the imaging object. The improved residual neural network embeds a channel attention mechanism and a spatial attention value on the basis of the residual neural network.

[0041] Furthermore, the actual physical diffraction process involves obtaining the outgoing wave of the imaging plane based on the probe spot and the conjectured function of the imaging plane, and obtaining the incoming wave of the probe plane based on the outgoing wave of the imaging plane and the diffraction propagation model. The obtained incoming wave of the probe plane is the diffraction estimate.

[0042] The specific steps to obtain the actual diffraction distance are as follows: (1) The two-dimensional image output by the improved residual neural network is used as the initial conjecture function of the imaging object. This variable is unknown, while the monochromatic wavelength is known.

[0043] (2) Obtain the monochromatic diffraction pattern through experiments, and set the experimentally measured diffraction pattern as... .

[0044] (3) The monochromatic diffraction propagation model is known. After using the monochromatic diffraction propagation model, the corresponding diffraction estimate is obtained. .

[0045] (4) Construct a loss function using diffraction estimation and experimentally measured diffraction patterns. The parameters of the improved residual neural network are optimized based on this loss function. After several iterations (500 in this embodiment), the reconstructed image can be obtained.

[0046] The PhysenNet algorithm, constrained by a physical model, requires precise wavelength knowledge. While the wavelength of a monochromatic light source is known, the diffraction propagation distance is limited by the system architecture and difficult to obtain directly with accurate values. Therefore, an autofocus algorithm based on contrast is used to evaluate the optimal diffraction distance for reconstructed image quality. Specifically, the autofocus algorithm performs reconstruction at 0.5mm intervals within a 3cm range before and after a roughly measured distance, calculating the Laplacian energy, gradient mean, and global variance of the reconstructed image with equal weights. These Laplacian energy, gradient mean, and global variance are used to determine whether the model outputs the highest quality image at that diffraction distance. If the highest quality is achieved, the diffraction distance is considered accurate, meaning the image contrast is evaluated, and the distance with the highest evaluation metric is retained as the precise diffraction distance.

[0047] The specific steps for constructing a practical broadband physical propagation model using a USAF 1951 resolution target as the imaging object data are as follows: (1) The DFFT algorithm model is obtained by transforming the angular spectrum wave generation principle of Fresnel diffraction; The Huygens-Fresnel principle states that the light field emitted from any point outside the wavefront should be the result of the coherent superposition of all wavelets on the wavefront. According to the description of Kirchhoff's diffraction integral, in the case of normal incidence of a monochromatic plane wave, the diffraction formula of the diffraction plane can be expressed as: (1) in, incident surface The light field function on, diffraction surface The light field function on, Let be the angle between the direction of incident light propagation and the normal to the diffraction plane. z The transmission distance between the two sides. Represents the imaginary unit. Indicates wavelength. Represents the wave vector. represents the vector diameter, This represents the tilt factor.

[0048] Under the Fresnel approximation, i.e., near-field diffraction conditions, the diffraction integral formula can be written in the following form: (2) From formula (2), the impulse response of Fresnel diffraction can be obtained as follows: (3) in, This indicates an impulse response.

[0049] Therefore, the Fresnel diffraction integral formula can be written in the following convolution form: (4) To simplify the convolution operation, we choose to transform it to the frequency domain for calculation, resulting in the following formula: (5) in, This represents the Fourier transform of the light field function. This represents the transfer function, and the corresponding impulse response function in the spatial domain in the frequency domain.

[0050] Then transform it to the spatial domain to obtain: (6) Substituting the Fresnel diffraction transfer function into the equation yields the following diffraction formula: (7) in, Represents the imaginary unit. Indicates the diffraction distance. , These represent the horizontal spatial frequency and the vertical spatial frequency, respectively.

[0051] Note that there are still approximation conditions in formula (7), which are corrected to the following formula: (8) Formula (8) is called the DFFT (double Fourier transform) algorithm for Fresnel diffraction. Using this method, it is only necessary to convert the frequency domain... x and y Frequency value in direction u and v Substituting the values ​​into the analytical expression yields the value of the transfer function. Combining this with the light source wavelength and diffraction distance allows for the construction of an accurate diffraction propagation model. In this embodiment, the light source wavelength is known; obtaining the precise diffraction distance is sufficient to construct an accurate monochromatic diffraction model.

[0052] (2) The theory of partial coherent mode decomposition proves that the cross spectral density function of partially coherent light waves can be expressed as a superposition of a series of orthogonal modes. These modes are orthogonal and uncorrelated, and can propagate independently without interfering with each other. In this way, the broadband continuous spectrum can be discretized into a series of separated spectra of different wavelengths.

[0053] According to WOLF's theory, in the case of temporal partial coherence, a broadband light source can be decomposed into multiple temporally coherent modes, each corresponding to a different frequency component in the spectrum. These modes are orthogonal to each other and can be analyzed and imaged independently.

[0054] By inputting the actual diffraction distance and each discrete wavelength into the DFFT algorithm model and superimposing the results, the actual broadband diffraction propagation model (actual broadband physical propagation model) is obtained. The constructed broadband diffraction propagation model is expressed as follows: (9) in, Indicates frequency, Represents the x-axis, Represents the ordinate, Indicates the total number of wavelengths. Indicates wavelength index. This indicates the magnitude of the spectral value at that frequency. This indicates the diffraction pattern distribution at that wavelength.

[0055] S3: Based on the fused diffraction pattern, a dual-network-based spectral-image joint reconstruction algorithm is used to reconstruct the image and spectrum, obtaining the reconstructed diffraction pattern and spectrum; for example... Figure 8 As shown, the processing procedure of the dual-network-based spectral-image joint reconstruction algorithm is as follows: The fused diffraction pattern is input into an improved residual network for processing, and its two-dimensional image is output. The two-dimensional image of the fused diffraction pattern is used as the initial conjecture function of the imaging object. Obtain the Gaussian distribution, input the Gaussian distribution into a fully connected neural network for processing, output the spectrum, and use the spectrum as the initial conjecture function for the transmission spectrum; Based on the initial conjecture function of the imaged object and the actual broadband physical propagation model, the estimated broadband aliasing intensity map of the image is obtained (broadband diffraction estimation 1). Based on the initial conjecture function of the transmission spectrum and the actual broadband physical propagation model, the estimated broadband aliasing intensity map of the spectrum is obtained (broadband diffraction estimation 2). A first loss function is constructed based on the estimated broadband aliasing intensity map of the image and the experimentally measured broadband aliasing intensity map, and the parameters of the improved residual neural network are optimized based on the first loss function; a second loss function is constructed based on the estimated broadband aliasing intensity map of the spectrum and the experimentally measured broadband aliasing intensity map, and the parameters of the fully connected neural network are optimized based on the second loss function. The improved residual neural network and the fully connected neural network are iteratively optimized alternately. When the network converges, the reconstructed diffraction pattern and spectrum are output.

[0056] In this embodiment, when the improved residual neural network and the fully connected neural network are iteratively optimized alternately, the Laplacian energy is used as the standard to determine whether the image is in positive optimization, and the first-order difference mean is used as the standard to determine whether the transmission spectrum is in positive optimization. During iterative optimization, when the evaluation standard decreases, the other network is optimized. The above standards are adaptively selected and iteratively performed to finally obtain the reconstructed image and spectral results.

[0057] The dual-network-based spectral-image joint reconstruction algorithm innovatively adds the variables to be reconstructed in the physical model to be spectrum and image. Based on the different dimensions of these two data types, two types of neural network models are built: a fully connected neural network and an improved residual neural network. During image reconstruction, the parameters of the fully connected neural network are not updated, i.e., the spectrum is fixed; similarly, during spectral reconstruction, the parameters of the improved residual neural network are not updated, i.e., the image is fixed.

[0058] The specific steps of the dual-network-based spectral-image joint reconstruction algorithm are as follows: The accurate diffraction model is determined based on the monochromatic diffraction chart, and the accurate diffraction distance, i.e. the actual diffraction distance, is obtained. After obtaining the accurate diffraction distance, a specific spectrum-image joint measurement experiment is carried out. The experiment is mostly similar to the experiment of calibrating the accurate diffraction model. The differences are: first, the light source is changed to a broadband spectrum, and a broadband diffraction propagation model (i.e., a broadband physical propagation model) needs to be constructed based on the partial coherence mode decomposition theory; second, the transmission spectrum and image need to be reconstructed at the same time. The network model adds a fully connected neural network to the original improved residual neural network model to reconstruct the spectral information.

[0059] (1) Using the PyTorch neural network framework, construct a fully connected neural network model and a residual neural network model with embedded spatial attention mechanism and channel attention mechanism (improved residual neural network), and initialize the model parameters, optimizer and loss function; (2) The fused diffraction pattern and Gaussian distribution are input into the improved residual neural network and the fully connected neural network respectively for processing, and the two-dimensional image and spectrum are output respectively. The outputs of the two networks are set as the initial conjecture functions of the imaging object. Initial conjecture function for transmission spectrum All of the above data is unknown.

[0060] It should be noted that Gaussian distribution is one-dimensional data and can be directly generated using Python, serving as input to a fully connected neural network.

[0061] (3) After the two-dimensional image and spectrum are processed through the broadband physical propagation model, the corresponding estimated broadband aliasing intensity map is obtained (i.e., the diffraction estimate of the forward propagation process of the initial function). When reconstructing the image, the parameters of the improved residual neural network need to be optimized, i.e., its output... As the variable, the corresponding broadband aliasing diagram is: When reconstructing the spectrum, the parameters of the fully connected neural network need to be optimized, i.e., its output. As the variable, the corresponding broadband aliasing diagram is: ,in For monochromatic diffraction intensity, This is a Fresnel diffraction propagation model.

[0062] (4) The broadband aliasing diffraction intensity pattern obtained in the experiment is as follows: Based on the above data, , Construct the first loss function and the second loss function respectively. Based on and Constructing the first loss function The parameters of the improved residual neural network are optimized using the first loss function; based on and Constructing a second loss function The parameters of the fully connected neural network are optimized using a second loss function. The two networks are then iteratively optimized alternately, and after five to six thousand iterations, a satisfactory result is achieved within approximately ten to twenty minutes.

[0063] It should be noted that the broadband aliasing diffraction intensity maps involving network input and the construction of the loss function are all fused diffraction maps.

[0064] Based on the above measurement equipment and methods, a simulation experiment was conducted, and the results are as follows: Figure 3 As shown, a represents the real target, b represents the unreconstructed result, and c represents the spectral comparison diagram. In c, the solid line represents the real spectrum, and the dashed line represents the reconstructed spectrum. Furthermore, a simulation experiment was conducted under known spectral conditions, and the results are as follows... Figure 4 As shown, a is the reconstructed monochromatic diffraction pattern, with the reconstruction result of the monochromatic diffraction pattern serving as a reference, and b is the broadband diffraction reconstruction pattern when the spectrum is known.

[0065] Furthermore, this invention conducted joint reconstruction experiments on the spectrum and image, and the results are as follows: Figure 5 As shown, a is a monochrome reconstructed image, which is used as a control; b is an image reconstructed with an aliasing image; and c is a spectral comparison image, where the solid line represents the actual spectrum and the dashed line represents the reconstructed spectrum.

[0066] Furthermore, the present invention conducted experiments for cases where the probe aperture shape was unknown, and the results are as follows: Figure 6 As shown, a is a real-world image captured by the camera, which serves as a reference; b is the image reconstruction result, i.e., the reconstructed image; and c is a spectral comparison image.

[0067] Furthermore, this invention focuses on reconstruction experiments of pure phase-type objects under known spectral conditions, with results as follows: Figure 7 As shown, a is the phase image reconstruction, b is the phase change corresponding to position A in a, and c is the phase change corresponding to position B in a.

[0068] In summary, this invention possesses extremely high versatility and robustness. Firstly, compared to traditional stacked coherent diffraction imaging methods, this method does not require the acquisition of large amounts of data or knowledge of the relative distance between two diffraction patterns for reconstruction, and it still achieves good convergence even with indoor visible light interference. Secondly, this invention is based on physical model optimization, enabling reconstruction of various targets at various wavelengths without the need for constructing large datasets. Traditional deep learning-based broadband diffraction imaging schemes all utilize data training methods, which have numerous drawbacks. Firstly, they require the acquisition of large amounts of experimental data, while broadband diffraction imaging is generally applied to the X-ray field, where corresponding detectors are still immature, and detection costs are extremely high, with excessively long acquisition cycles. Secondly, they have poor generalization ability; the imaging target is limited by the types of training sets, requiring the construction of new datasets for new imaging targets.

[0069] Example 2 This embodiment discloses a spectral-image joint measurement system based on a dual-network architecture, including: The image acquisition module is configured to acquire diffraction patterns at various exposure times and obtain a fused diffraction pattern after high dynamic range fusion. The model calibration module is configured to: acquire a monochromatic diffraction pattern, combine it with a monochromatic diffraction propagation model to obtain the actual diffraction distance, and construct an actual broadband physical propagation model based on the actual diffraction distance; The algorithm reconstruction module is configured to: reconstruct the image and spectrum based on the fused diffraction pattern using a dual-network-based spectral-image joint reconstruction algorithm, thereby obtaining the reconstructed diffraction pattern and spectrum; the processing procedure of the dual-network-based spectral-image joint reconstruction algorithm is as follows: The fused diffraction pattern and Gaussian distribution are input into the improved residual neural network and fully connected neural network for processing, respectively, and the two-dimensional image and spectrum of the fused diffraction pattern are output. The two-dimensional image and spectrum of the fused diffraction pattern are used as the initial conjecture functions of the imaging object and the transmission spectrum, respectively. The corresponding estimated broadband aliasing intensity map is obtained through the actual broadband physical propagation model. A loss function is constructed based on the estimated broadband aliasing intensity map and the experimentally measured broadband aliasing intensity map; Based on the corresponding loss function, the improved residual neural network and the fully connected neural network are iteratively optimized alternately. When the network converges, the reconstructed diffraction pattern and spectrum are output.

[0070] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

[0071] While the specific embodiments of the present invention have been described above in conjunction with the accompanying drawings, this is not intended to limit the scope of protection of the present invention. Those skilled in the art should understand that various modifications or variations that can be made by those skilled in the art without creative effort based on the technical solutions of the present invention are still within the scope of protection of the present invention.

Claims

1. A spectral-image joint measurement device based on a dual-network, characterized in that, It includes a light source arranged along the optical path, a first lens, an aperture stop, a second lens, a third lens, an imaging object, and a CMOS camera; The light source is collimated after being emitted by the first lens, and then illuminates the aperture stop to form a probe spot. The second and third lenses form a 4f system to relay the probe spot to the imaging object. The light field carrying the information of the imaging object diffracts and propagates to generate a diffraction pattern. The CMOS camera receives the diffraction pattern and transmits it to the processor. The processor is configured to reconstruct the received diffraction pattern using a dual-network-based spectral-image joint measurement method to obtain the reconstructed diffraction pattern and spectrum.

2. The spectral-image joint measurement device based on dual networks as described in claim 1, characterized in that, The specific steps of the dual-network-based spectral-image joint measurement method are as follows: Diffraction patterns were collected at various exposure times and then fused using high dynamic range fusion to obtain a fused diffraction pattern. Collect monochromatic diffraction patterns, combine them with a monochromatic diffraction propagation model to obtain the actual diffraction distance, and construct an actual broadband physical propagation model based on the actual diffraction distance; Based on the fused diffraction pattern, a dual-network-based spectral-image joint reconstruction algorithm is used to reconstruct the image and spectrum, obtaining the reconstructed diffraction pattern and spectrum. The processing procedure of the dual-network-based spectral-image joint reconstruction algorithm is as follows: The fused diffraction pattern and Gaussian distribution are input into the improved residual neural network and fully connected neural network for processing, respectively, and the two-dimensional image and spectrum of the fused diffraction pattern are output. The two-dimensional image and spectrum of the fused diffraction pattern are used as the initial conjecture functions of the imaging object and the transmission spectrum, respectively. The corresponding estimated broadband aliasing intensity map is obtained through the actual broadband physical propagation model. A loss function is constructed based on the estimated broadband aliasing intensity map and the experimentally measured broadband aliasing intensity map; Based on the corresponding loss function, the improved residual neural network and the fully connected neural network are iteratively optimized alternately. When the network converges, the reconstructed diffraction pattern and spectrum are output.

3. The spectral-image joint measurement device based on dual networks as described in claim 2, characterized in that, The actual diffraction distance is obtained by collecting a monochromatic diffraction pattern and combining it with a monochromatic diffraction propagation model. The monochromatic diffraction pattern is input into an improved residual neural network for processing, and its two-dimensional image is output. The two-dimensional image of the monochromatic diffraction pattern is used as the initial conjecture function of the imaging object, and the diffraction estimate is obtained through the monochromatic diffraction propagation model. A loss function is constructed based on diffraction estimation and experimentally measured monochromatic diffraction patterns. The parameters of the improved residual neural network are optimized based on the loss function, and the reconstructed monochromatic diffraction pattern is output. The actual diffraction distance is calculated based on the reconstructed monochromatic diffraction pattern using a contrast autofocus algorithm.

4. The spectral-image joint measurement device based on dual networks as described in claim 2, characterized in that, Based on the actual diffraction distance, a real broadband physical propagation model is constructed using the Fresnel diffraction propagation model and partial coherent mode decomposition.

5. The spectral-image joint measurement device based on dual networks as described in claim 2, characterized in that, The corresponding estimated broadband aliasing intensity map is obtained as follows: The two-dimensional image of the fused diffraction pattern is used as the initial conjecture function for the imaged object, and the spectrum is used as the initial conjecture function for the transmission spectrum. Based on the initial conjecture function of the imaged object combined with the actual broadband physical propagation model, the estimated broadband aliasing intensity map of the image is obtained. Based on the initial conjecture function of the transmission spectrum combined with the actual broadband physical propagation model, the estimated broadband aliasing intensity map of the spectrum is obtained.

6. The spectral-image joint measurement device based on dual networks as described in claim 5, characterized in that, A first loss function is constructed based on the estimated broadband aliasing intensity map of the image and the experimentally measured broadband aliasing intensity map. The parameters of the improved residual neural network are then optimized based on the first loss function.

7. The spectral-image joint measurement device based on dual networks as described in claim 5, characterized in that, A second loss function is constructed based on the estimated broadband aliasing intensity map and the experimentally measured broadband aliasing intensity map, and the parameters of the fully connected neural network are optimized based on the second loss function.

8. The spectral-image joint measurement device based on dual networks as described in claim 2, characterized in that, During the iterative optimization process of the improved residual neural network and the fully connected neural network, the parameters of the fully connected neural network are not optimized when reconstructing the diffraction pattern, and the parameters of the improved residual neural network are not optimized when reconstructing the spectrum.

9. The spectral-image joint measurement device based on dual networks as described in claim 2, characterized in that, In the iterative optimization process of the improved residual neural network and the fully connected neural network, the Laplace energy is used as the standard to determine whether the diffraction pattern is in positive optimization, and the first-order difference mean is used as the standard to determine whether the transmission spectrum is in positive optimization.

10. A spectral-image joint measurement system based on a dual-network, characterized in that, include: The image acquisition module is configured to acquire diffraction patterns at various exposure times and obtain a fused diffraction pattern after high dynamic range fusion. The model calibration module is configured to: acquire a monochromatic diffraction pattern, combine it with a monochromatic diffraction propagation model to obtain the actual diffraction distance, and construct an actual broadband physical propagation model based on the actual diffraction distance; The algorithm reconstruction module is configured to: reconstruct the image and spectrum based on the fused diffraction pattern using a dual-network-based spectral-image joint reconstruction algorithm, thereby obtaining the reconstructed diffraction pattern and spectrum; the processing procedure of the dual-network-based spectral-image joint reconstruction algorithm is as follows: The fused diffraction pattern and Gaussian distribution are input into the improved residual neural network and fully connected neural network for processing, respectively, and the two-dimensional image and spectrum of the fused diffraction pattern are output. The two-dimensional image and spectrum of the fused diffraction pattern are used as the initial conjecture functions of the imaging object and the transmission spectrum, respectively. The corresponding estimated broadband aliasing intensity map is obtained through the actual broadband physical propagation model. A loss function is constructed based on the estimated broadband aliasing intensity map and the experimentally measured broadband aliasing intensity map; Based on the corresponding loss function, the improved residual neural network and the fully connected neural network are iteratively optimized alternately. When the network converges, the reconstructed diffraction pattern and spectrum are output.

Images