Method for extracting high-resolution phase information of depth

Abstract

This application discloses a method, apparatus, device, and storage medium for high-resolution phase information extraction. The method includes: deconstructing and separating overlapping positive and negative phase images of a sample using a deep convolutional neural network model based on an attention mechanism; establishing a differential phase image based on the deconstructed and separated positive and negative phase images; generating network training data using an X-ray phase difference microscopy physical model; and directly using the trained model for experimental data processing to extract the absolute and differential phase information of the sample. The above-mentioned solution provided by this application can quickly and accurately extract the absolute and differential phase information of a sample from overlapping positive and negative X-ray phase microscopy data. Furthermore, this invention uses numerical simulation to generate network training data, eliminating the need to collect training data on an experimental system, greatly reducing experimental difficulty and saving experimental time.

Description

Technical Field

[0001] This invention relates to the field of image processing methods, specifically to a method, apparatus, device, and storage medium for high-resolution depth phase information extraction. Background Technology

[0002] Over the past few decades, advancements in X-ray phase imaging have broadened the applications of X-ray imaging. As an important complement to traditional absorption imaging, phase imaging can observe the internal structure of weakly absorbing objects (such as biological samples) with greater sensitivity because, for light elements, the X-ray phase shift has a larger interaction cross-section than X-ray attenuation.

[0003] Among the various current X-ray phase-contrast imaging methods, the grating-based X-ray Talbot (-Lau) interferometer has attracted increasing attention from researchers due to its greater adaptability to light source conditions. This device can simultaneously acquire the absorption, small-angle scattering, and differential phase information of a sample. Combined with computed tomography (CT) technology, it can also obtain three-dimensional information of samples based on different contrast levels. Combining it with an X-ray microscope with nanoscale spatial resolution enables high-resolution imaging of the internal morphology of tiny objects (such as cells) with multiple contrast levels. Several research groups internationally have explored and studied this research direction, seeking effective experimental schemes to integrate grating interferometers and X-ray transmission microscopes. One significant advancement is the phase difference imaging achieved in 2017 by Takano et al. from Tohoku University, Japan, by combining a grating interferometer with a commercially available X-ray nanomicroscopy system from ZEISS. Unlike the classic Talbot (-Lau) interferometer, this imaging system obtains not differential phase images, but overlapping positive and negative phase images. The existence of this artifact phenomenon makes it difficult to extract sample structural information, thus hindering the further use and promotion of this imaging method.

[0004] To extract phase information from overlapping phase images, the current mainstream solution is to use image post-processing algorithms. For example, Yashiro et al. proposed a phase-shifting summation algorithm and a deconvolution algorithm. The phase-shifting summation algorithm, based on the imaging principles of grating interferometers and X-ray transmission microscopes, sums the differential signals of different interference levels to extract the phase image; the deconvolution algorithm divides the frequency domain by the interference level correlation term using Fourier transform and obtains the phase distribution through inverse Fourier transform. Furthermore, Takano et al. proposed an iterative deconvolution method, which uses deconvolution to repeatedly correct sample images in the spatial and frequency domains to extract phase information; Wolf et al. proposed a maximum likelihood reconstruction scheme based on total variation (TV) regularization and used an iterative optimization algorithm to reconstruct phase images under low photon numbers.

[0005] Currently, the main problem with extracting volumetric phase information using phase-shift summation and deconvolution algorithms is their susceptibility to noise. When the number of photons is low, the noise level in the acquired differential phase image increases, leading to noise amplification in the extracted phase signal. Furthermore, deconvolution algorithms are prone to producing periodic artifacts. Iterative deconvolution methods can suppress some noise and reduce periodic artifacts, but this algorithm can only be used with π / 2 gratings, limiting its application scenarios. Maximum likelihood reconstruction schemes based on TV regularization can effectively suppress noise and artifacts; however, they suffer from high computational cost and slow reconstruction speed. Summary of the Invention

[0006] In view of the above-mentioned defects or deficiencies in the prior art, it is desirable to provide a method, apparatus, device and storage medium for high-resolution phase information extraction.

[0007] In a first aspect, embodiments of this application provide a method for extracting high-resolution phase information, the method comprising:

[0008] The overlapping positive and negative phase images of the samples are deconstructed and separated using a deep convolutional neural network model based on an attention mechanism.

[0009] A differential phase image is established based on the forward and reverse phase images of the deconstructed and separated sample;

[0010] Network training data is generated using an X-ray phase difference microscopy physical model. The trained model is then directly used for experimental data processing to extract the absolute and differential phase information of the object.

[0011] In one embodiment, the attention mechanism network model includes a deep convolutional neural network based on spatial attention and channel attention mechanisms.

[0012] In one embodiment, the degree of separation between the positive and negative phase images of the sample is: Where λ is the X-ray wavelength, d is the distance from the phase grating to the detector, p is the equivalent period of the phase grating, and M represents the magnification factor of the sample through the zone plate.

[0013] In one embodiment, the step of generating network training data using the X-ray phase difference microscopy physical model includes: downloading a preset number of open-source natural image datasets like ImageNet; for a single image, taking one color channel data, resizing the image, and denoting this data as x0; using Matlab code to generate a binary mask image M with randomly varying position and shape, performing a dot product operation with x0, and extracting the region of interest x. ROI =M·x0; for x ROIThe value of each pixel in the image is adjusted, and its value range is constrained to a preset range according to the actual sample material to simulate the absolute phase shift of the object. Based on the physics of X-ray phase microscopy, a noise model is added to generate overlapping phase images Φ and differential phase images Φ′ at different separation scales. The above steps are repeated until all images are processed.

[0014] Secondly, embodiments of this application also provide a depth high-resolution phase information extraction device, the device comprising:

[0015] The deconstruction and separation unit is used to deconstruct and separate overlapping sample front and back phase images using a deep convolutional neural network model based on an attention mechanism;

[0016] A unit is established to create a differential phase image based on the forward and reverse phase images of the deconstructed and separated sample;

[0017] The extraction unit is used to generate network training data through the physical model of X-ray phase difference microscopy, and to directly use the trained model for experimental data processing to extract the absolute phase and differential phase information of the object.

[0018] In one embodiment, the attention mechanism network model includes a deep convolutional neural network based on spatial attention and channel attention mechanisms.

[0019] In one embodiment, the degree of separation between the positive and negative phase images of the sample is: Where λ is the X-ray wavelength, d is the distance from the phase grating to the detector, p is the equivalent period of the phase grating, and M represents the magnification factor of the sample through the zone plate.

[0020] In one embodiment, the step of generating network training data using the X-ray phase difference microscopy physical model includes: downloading a preset number of open-source natural image datasets like ImageNet; for a single image, taking one color channel data, resizing the image, and denoting this data as x0; using Matlab code to generate a binary mask image M with randomly varying position and shape, performing a dot product operation with x0, and extracting the region of interest x. ROI =M·x0; for x ROI The value of each pixel in the image is adjusted, and its value range is constrained to a preset range according to the actual sample material to simulate the absolute phase shift of the object. Based on the physics of X-ray phase microscopy, a noise model is added to generate overlapping phase images Φ and differential phase images Φ′ at different separation scales. The above steps are repeated until all images are processed.

[0021] Thirdly, embodiments of this application also provide a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement any of the methods described in the embodiments of this application.

[0022] Fourthly, embodiments of this application also provide a computer device and a computer-readable storage medium having a computer program stored thereon, the computer program being used to: implement any of the methods described in the embodiments of this application when the computer program is executed by a processor.

[0023] The beneficial effects of this invention are:

[0024] The depth-high resolution phase information extraction method provided by this invention can quickly and accurately extract the absolute and differential phase information of a sample from X-ray phase microscopy data with overlapping positive and negative phases. At the same time, this invention uses numerical simulation to generate network training data, eliminating the need to collect training data on the experimental system, which greatly reduces the experimental difficulty and saves experimental time. Attached Figure Description

[0025] Other features, objects, and advantages of this application will become more apparent from the following detailed description of non-limiting embodiments with reference to the accompanying drawings:

[0026] Figure 1 A schematic flowchart of the depth high-resolution phase information extraction method provided in an embodiment of this application is shown;

[0027] Figure 2 An exemplary structural block diagram of a depth high-resolution phase information extraction apparatus 200 according to an embodiment of this application is shown;

[0028] Figure 3 A schematic diagram of the structure of a computer system suitable for implementing the terminal device of the present application is shown;

[0029] Figure 4 The flowchart of the method for extracting absolute and differential phase information of X-ray phase microscopy based on deep learning provided in the embodiments of this application is shown.

[0030] Figure 5 A schematic diagram of the U-Net network architecture based on spatial attention and channel attention mechanisms provided in this application embodiment is shown. Detailed Implementation

[0031] To make the above-mentioned objects, features, and advantages of the present invention more apparent and understandable, specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings. Many specific details are set forth in the following description to provide a thorough understanding of the present invention. However, the present invention can be practiced in many other ways different from those described herein, and those skilled in the art can make similar modifications without departing from the spirit of the present invention. Therefore, the present invention is not limited to the specific embodiments disclosed below.

[0032] In the description of this invention, it should be understood that the terms "center," "longitudinal," "lateral," "length," "width," "thickness," "upper," "lower," "front," "rear," "left," "right," "vertical," "horizontal," "top," "bottom," "inner," "outer," "clockwise," "counterclockwise," "axial," "radial," and "circumferential" indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. They are used only for the convenience of describing this invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on this invention.

[0033] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this invention, "a plurality of" means at least two, such as two, three, etc., unless otherwise explicitly specified.

[0034] In this invention, unless otherwise explicitly specified and limited, the terms "installation," "connection," "linking," and "fixing," etc., should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral part; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; they can refer to the internal communication of two components or the interaction between two components, unless otherwise explicitly limited. Those skilled in the art can understand the specific meaning of the above terms in this invention according to the specific circumstances.

[0035] In this invention, unless otherwise explicitly specified and limited, "above" or "below" the second feature can mean that the first feature is in direct contact with the second feature, or that the first feature is in indirect contact with the second feature through an intermediate medium. Furthermore, "above," "over," and "on top" of the second feature can mean that the first feature is directly above or diagonally above the second feature, or simply that the first feature is at a higher horizontal level than the second feature. "Below," "below," and "under" the second feature can mean that the first feature is directly below or diagonally below the second feature, or simply that the first feature is at a lower horizontal level than the second feature.

[0036] It should be noted that when an element is referred to as being "fixed to" or "set on" another element, it can be directly on the other element or there may be an intervening element. When an element is considered to be "connected to" another element, it can be directly connected to the other element or there may be an intervening element. The terms "vertical," "horizontal," "upper," "lower," "left," "right," and similar expressions used herein are for illustrative purposes only and do not represent the only possible implementation.

[0037] Please refer to Figure 1 , Figure 1 A schematic flowchart of the depth high-resolution phase information extraction method provided in the embodiments of this application is shown.

[0038] like Figure 1 As shown, the method includes:

[0039] Step 110: Deconstruct and separate the overlapping positive and negative phase images of the samples using a deep convolutional neural network model based on an attention mechanism;

[0040] Step 120: Establish a differential phase image based on the forward and reverse phase images of the deconstructed and separated sample;

[0041] Step 130: Generate network training data through the physical model of X-ray phase difference microscopy, and use the trained model directly for experimental data processing to extract the absolute phase and differential phase information of the object.

[0042] By adopting the above technical solution, the absolute phase and differential phase information of the sample can be extracted quickly and accurately from the X-ray phase microscopy data with overlapping positive and negative phases. At the same time, the present invention uses numerical simulation to generate network training data, eliminating the need to collect training data on the experimental system, which greatly reduces the experimental difficulty and saves experimental time.

[0043] In some embodiments, the attention mechanism network model in this application includes: a deep convolutional neural network based on spatial attention and channel attention mechanisms.

[0044] Specifically, the network model proposed in this invention is a deep convolutional neural network based on spatial attention and channel attention mechanisms. Taking the U-Net network as an example, refer to... Figure 5As shown, the network input image consists of overlapping positive and negative phase information, and the output is an absolute phase image (positive in this example, but negative values ​​are also possible) and a differential phase image. The network extracts deep image features through multiple convolutional and downsampling operations. The extracted image features generate spatial and channel weight maps through an attention mechanism, and the weight maps are multiplied with features subsequently upsampled to the same dimension. This operation allows the network to focus on the real image phase information at specific spatial locations and strengthens the weights of specific channel features. Finally, symmetrical upsampling of the feature images to the same dimension achieves image phase information extraction. The network is trained using the numerical simulation dataset generated in the previous step. The mean squared error (MSE) between the network output and the ground truth is used as the loss function to initialize the network parameters using Xavier, and the Adam optimizer is used to train the network to obtain the updated network parameter model. The trained network model can be directly used for experimental data processing. First, X-ray phase microscopy imaging data is collected from laboratory equipment, and then this data is input into the trained network model to obtain the separated sample phase information.

[0045] In some embodiments, the degree of separation between the forward and reverse phase images of the sample in this application is as follows:

[0046]

[0047] Where λ is the X-ray wavelength, d is the distance from the phase grating to the detector, p is the equivalent period of the phase grating, and M represents the magnification factor of the sample through the zone plate.

[0048] Specifically, the experimental apparatus referenced in this invention is an X-ray phase microscopy imaging system based on a grating interferometer. The main components of the system include an X-ray source, an ellipsoidal condenser, a source grating, a zone plate, a phase grating, and a detector. The imaging advantage of this system lies primarily in its ability to provide phase imaging of small-sized, weakly absorbing samples with microstructures, offering higher contrast sample morphology information compared to absorption imaging. This imaging system can simultaneously perform microscopic and phase imaging of the sample. Its working principle is as follows: First, an X-ray beam emitted from the source is focused by the ellipsoidal condenser and, after passing through the source grating, forms structured light illuminating the sample. Then, the X-ray beam carrying sample characteristic information is focused by the zone plate, forming an image of the source grating near its back focal plane, while a magnified sample image is formed at the detector end further away. Introducing a phase grating at a suitable position between these two images creates a structure similar to a Talbot (-Lau) interferometer, along with the image of the source grating. By stepping the phase grating along a direction perpendicular to the optical axis, the detector can record raw imaging data containing sample phase information.

[0049] Consider a pure-phase sample with a refractive index of n = (1-δ), corresponding to a global phase shift along the optical axis z of φ(x) = ∫δ(x, z)dz. Based on the principles of wave optics, a physical model of the imaging process can be performed, theoretically deriving the phase information of the sample extracted through phase-stepping operations. Specifically, the form is as follows:

[0050]

[0051] This includes two separate positive and negative phase images, where λ is the X-ray wavelength, k is the wavenumber, p is the equivalent period of the phase grating, M represents the magnification factor of the sample imaged via the zone plate, and d is the distance from the phase grating to the detector. The degree of separation between the two can be expressed as...

[0052]

[0053] Formula (2) can be used to quickly determine the separation distance between the positive and negative phase images of a sample in an actual phase imaging system.

[0054] In some embodiments, the generation of network training data using the X-ray phase difference microscopy physical model in this application includes: downloading a preset number of open-source natural image datasets like ImageNet; for a single image, taking one color channel data, resizing the image, and denoting this data as x0; using Matlab code to generate a binary mask image M with randomly varying position and shape, performing a dot product operation with x0, and extracting the region of interest x. ROI =M·x0; for x ROI The value of each pixel in the image is adjusted, and its value range is constrained to a preset range according to the actual sample material to simulate the absolute phase shift of the object. Based on the physics of X-ray phase microscopy, a noise model is added to generate overlapping phase images Φ and differential phase images Φ′ at different separation scales. The above steps are repeated until all images are processed.

[0055] Specifically, this invention uses deep learning to extract the absolute and differential phase information of an object from overlapping positive and negative phase images. Figure 4 The diagram shows the specific principle of this method, including: the process of using a deep convolutional neural network based on an attention mechanism to extract phase information and the method for generating training data for X-ray phase microscopy imaging.

[0056] The specific steps for generating the training data are as follows: 1) Download the large-scale open-source natural image dataset ImageNet. 2) For a single image, take one color channel data, resize the image (e.g., 1024*1024), and denote this data as x0. 3) Use Matlab code to generate a binary mask image M with randomly varying position and shape, perform a dot product operation with x0, and extract the region of interest x.ROI =M·x0.4) for x ROI The value of each pixel in the image is adjusted, and its value range is constrained to a certain range according to the actual sample material (such as plexiglass or graphite) to simulate the absolute phase shift of the object. 5) Based on the physics of X-ray phase microscopy, a noise model is added to generate overlapping phase images Φ and differential phase images Φ′ at different separation scales. Repeat steps 2)-5) until all images are processed.

[0057] Further, refer to Figure 2 , Figure 2 An exemplary structural block diagram of a depth high-resolution phase information extraction apparatus 200 according to an embodiment of this application is shown.

[0058] like Figure 2 As shown, the device includes:

[0059] The deconstruction and separation unit 210 is used to deconstruct and separate the front and back phase images of the samples by means of a deep convolutional neural network model based on an attention mechanism;

[0060] Establishment unit 220 is used to establish a differential phase image based on the forward and reverse phase images of the deconstructed and separated sample;

[0061] Extraction unit 230 is used to generate network training data through the physical model of X-ray phase difference microscopy, and to directly use the trained model for experimental data processing to extract the absolute phase and differential phase information of the object.

[0062] It should be understood that the units or modules described in device 200 are related to the reference. Figure 1 The steps in the described method correspond to each other. Therefore, the operations and features described above for the method also apply to the device 200 and the units contained therein, and will not be repeated here. The device 200 can be pre-implemented in the browser or other security applications of an electronic device, or it can be loaded into the browser or its security applications of an electronic device through download or other means. The corresponding units in the device 200 can cooperate with the units in the electronic device to implement the solutions of the embodiments of this application.

[0063] The following is for reference. Figure 3 It shows a schematic diagram of the structure of a computer system 300 suitable for implementing terminal devices or servers in the embodiments of this application.

[0064] like Figure 3As shown, the computer system 300 includes a central processing unit (CPU) 301, which can perform various appropriate actions and processes based on programs stored in read-only memory (ROM) 302 or programs loaded from storage section 308 into random access memory (RAM) 303. The RAM 303 also stores various programs and data required for the operation of the system 300. The CPU 301, ROM 302, and RAM 303 are interconnected via a bus 304. An input / output (I / O) interface 305 is also connected to the bus 304.

[0065] The following components are connected to I / O interface 305: an input section 306 including a keyboard, mouse, etc.; an output section 307 including a cathode ray tube (CRT), liquid crystal display (LCD), etc., and speakers, etc.; a storage section 308 including a hard disk, etc.; and a communication section 309 including a network interface card such as a LAN card, modem, etc. The communication section 309 performs communication processing via a network such as the Internet. A drive 310 is also connected to I / O interface 305 as needed. A removable medium 311, such as a disk, optical disk, magneto-optical disk, semiconductor memory, etc., is installed on drive 310 as needed so that computer programs read from it can be installed into storage section 308 as needed.

[0066] In particular, according to embodiments of this disclosure, the above references Figure 1 The described process can be implemented as a computer software program. For example, embodiments of this disclosure include a method for extracting high-resolution phase information, comprising a computer program tangibly contained on a machine-readable medium, the computer program containing instructions for performing... Figure 1 The program code for the method. In such an embodiment, the computer program can be downloaded and installed from a network via the communication section 309, and / or installed from the removable medium 311.

[0067] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present invention. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0068] The units or modules described in the embodiments of this application can be implemented in software or hardware. The described units or modules can also be housed in a processor; for example, a processor can be described as including a first sub-region generation unit, a second sub-region generation unit, and a display area generation unit. The names of these units or modules do not necessarily limit the specific unit or module itself; for example, the display area generation unit can also be described as "a unit for generating a display area of ​​text based on the first and second sub-regions."

[0069] In another aspect, this application also provides a computer-readable storage medium, which may be the computer-readable storage medium included in the aforementioned apparatus in the above embodiments; or it may be a standalone computer-readable storage medium not assembled into the device. The computer-readable storage medium stores one or more programs, which are used by one or more processors to execute the text generation method described in this application for transparent window envelopes.

[0070] The above description is merely a preferred embodiment of this application and an explanation of the technical principles employed. Those skilled in the art should understand that the scope of the invention involved in this application is not limited to technical solutions formed by specific combinations of the above-described technical features, but should also cover other technical solutions formed by arbitrary combinations of the above-described technical features or their equivalents without departing from the foregoing inventive concept. For example, technical solutions formed by substituting the above features with (but not limited to) technical features with similar functions disclosed in this application.

Claims

1. A method for extracting high-resolution phase information, characterized in that, The method includes: The overlapping front and back phase images of the sample acquired by the X-ray phase microscopy system are deconstructed and separated using a deep convolutional neural network model based on an attention mechanism. A differential phase image is established based on the forward and reverse phase images of the deconstructed and separated sample; The physical model of X-ray phase difference microscopy is used to generate network training data. The trained deep convolutional neural network model based on attention mechanism is directly used for experimental data processing to extract the absolute phase and differential phase information of the object. The network training data generated by the X-ray phase difference microscopy physical model includes: Download a preset number of ImageNet open-source datasets of natural images; For a single picture, take one color channel data, reset the picture size, and record the data as ; Generate binary mask images with randomly varying positions and shapes using Matlab code. M ,and Perform a dot product operation to extract the region of interest. ; right The value of each pixel in the image is adjusted, and its value range is constrained to a preset range based on the actual sample material to simulate the absolute phase shift of the object. Based on the physics of X-ray phase microscopy, a noise model is added to generate overlapping phase images at different separation scales. and differential phase image Repeat the above steps until all images have been processed.

2. The deep high-resolution phase information extraction method of claim 1, wherein, The attention-based deep convolutional neural network model includes: Deep convolutional neural networks based on spatial attention and channel attention mechanisms.

3. The method for extracting high-resolution phase information according to claim 1, characterized in that, The degree of separation between the positive and negative phase images of the sample is: in For X-ray wavelength, d The distance from the phase grating to the detector. p The equivalent period of the phase grating. M This represents the magnification factor of the sample as it is imaged via a zone plate.

4. A depth high-resolution phase information extraction apparatus, the apparatus being used to implement the depth high-resolution phase information extraction method as described in any one of claims 1 to 3, characterized in that, The device includes: The deconstruction and separation unit is used to deconstruct and separate overlapping sample front and back phase images using a deep convolutional neural network model based on an attention mechanism; A unit is established to create a differential phase image based on the forward and reverse phase images of the deconstructed and separated sample; The extraction unit is used to generate network training data through the physical model of X-ray phase difference microscopy imaging. The trained deep convolutional neural network model based on the attention mechanism is directly used for experimental data processing to extract the absolute phase and differential phase information of the object.

5. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the method as described in any one of claims 1-3.

6. A computer-readable storage medium having a computer program stored thereon, the computer program being configured to: when executed by a processor, implement the method as described in any one of claims 1-3.

Images