Image processing method with cross-modal transfer point spread function decoupling and related components

Abstract

This invention discloses an image processing method and related components for decoupling the cross-modal migration point spread function. The method includes: selecting calibration samples and recording a first modal image stack and a second modal image stack; preprocessing the first modal image stack to obtain an emission field estimation result; spatially registering the emission field estimation result with the second modal image stack; using the registered emission field estimation result and the second modal image stack as physical constraints to establish an inverse problem model; optimizing and solving the inverse problem model to obtain the migration point spread function characterizing the mapping relationship from the first imaging modality to the second imaging modality; acquiring the first modal image stack of the tested sample, and performing inversion reconstruction based on the migration point spread function to obtain an equivalent image of the second imaging modality. This invention is based on a common emission field formation mechanism and achieves computational migration reconstruction from a low-performance imaging modality to a high-performance imaging modality by constructing sample priors and decoupling the migration point spread function.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and in particular to an image processing method and related components for decoupling cross-modal migration point spread functions. Background Technology

[0002] In the field of fluorescence microscopy, different imaging modalities often exhibit mutually restrictive trade-offs in terms of imaging speed, resolution, optical slicing capability, system complexity, and imaging cost. Low-performance imaging modalities typically offer advantages such as fast acquisition speed, simple system structure, and low cost, but their image quality is relatively limited. Conversely, while high-performance imaging modalities can achieve superior image quality through more complex optical modulation, they generally suffer from lower acquisition efficiency, more complex system implementation, and higher imaging costs, making it difficult to simultaneously balance imaging depth, imaging speed, image quality, and system cost. Therefore, how to achieve the imaging effects of high-performance imaging modalities while retaining the acquisition advantages of low-performance imaging modalities has become an important research direction in computational fluorescence microscopy. Summary of the Invention

[0003] This invention provides an image processing method, apparatus, computer device, and storage medium for cross-modal migration point spread function decoupling, aiming to achieve computational migration reconstruction from low-performance imaging modal to high-performance imaging modal.

[0004] In a first aspect, embodiments of the present invention provide an image processing method for decoupling cross-modal migration point spread functions, including: Select calibration samples, and record the first modal image stack and the second modal image stack of the calibration samples under the low-performance first imaging modal system and the high-performance second imaging modal system, respectively; Preprocess the first modal image stack to obtain the emission field estimation result of the calibration sample in the first imaging modality; Spatial registration is performed between the emission field estimation result and the second modality image stack; The registered emission field estimation results and the second modality image stack are used as the pairing body data, and the inverse problem model is established using the pairing body data as physical constraints. The inverse problem model is optimized and solved to obtain the migration point spread function that characterizes the mapping relationship from the first imaging mode to the second imaging mode; For the tested sample, the first modal image stack of the tested sample is obtained, and inversion reconstruction is performed based on the migration point spread function to obtain the second imaging modal equivalent image corresponding to the tested sample.

[0005] Secondly, embodiments of the present invention provide an image processing apparatus for cross-modal migration point spread function decoupling, comprising: A stack recording unit is used to select calibration samples and record the first modal image stack and the second modal image stack of the calibration samples in a low-performance first imaging modal system and a high-performance second imaging modal system, respectively. The stack preprocessing unit is used to preprocess the first modal image stack to obtain the emission field estimation result of the calibration sample in the first imaging modality; A spatial registration unit is used to spatially register the emission field estimation result with the second modality image stack. The model building unit is used to take the registered emission field estimation results and the second modality image stack as the pairing body data, and use the pairing body data as physical constraints to build an inverse problem model. An optimization and solution unit is used to optimize and solve the inverse problem model to obtain the migration point spread function that characterizes the mapping relationship from the first imaging mode to the second imaging mode; The inversion and reconstruction unit is used to acquire the first modal image stack of the test sample and perform inversion and reconstruction based on the migration point diffusion function to obtain the second imaging modal equivalent image corresponding to the test sample.

[0006] Thirdly, embodiments of the present invention provide a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the image processing method for decoupling the cross-modal migration point spread function as described in the first aspect.

[0007] Fourthly, embodiments of the present invention provide a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the image processing method for decoupling cross-modal migration point spread function as described in the first aspect.

[0008] This invention provides an image processing method, apparatus, computer device, and storage medium for cross-modal migration point spread function decoupling. The method includes: selecting a calibration sample and recording a first-modal image stack and a second-modal image stack of the calibration sample under a low-performance first imaging modal system and a high-performance second imaging modal system, respectively; preprocessing the first-modal image stack to obtain the emission field estimation result of the calibration sample under the first imaging modality; spatially registering the emission field estimation result with the second-modal image stack; using the registered emission field estimation result and the second-modal image stack as paired data and establishing an inverse problem model using the paired data as physical constraints; optimizing and solving the inverse problem model to obtain a migration point spread function characterizing the mapping relationship from the first imaging modality to the second imaging modality; and for a test sample, acquiring the first-modal image stack of the test sample and performing inversion reconstruction based on the migration point spread function to obtain the equivalent image of the test sample in the second imaging modality. Based on a common emission field formation mechanism, this invention achieves computational migration and reconstruction from a low-performance imaging modality to a high-performance imaging modality by constructing sample priors and decoupling the migration point spread function. This invention not only overcomes the technical problem of difficulty in connecting traditional imaging modalities but also integrates the signal formation characteristics and imaging advantages of different imaging modalities, providing strong technical support for high-performance computational reconstruction. Attached Figure Description

[0009] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the following description of the embodiments will be briefly introduced. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0010] Figure 1 A flowchart illustrating the image processing method for decoupling cross-modal migration point spread function provided in an embodiment of the present invention; Figure 2 This is a schematic diagram illustrating the principle architecture of the image processing method for decoupling cross-modal migration point spread function provided in an embodiment of the present invention. Figure 3 This is a schematic block diagram of an image processing apparatus for decoupling cross-modal migration point spread function provided in an embodiment of the present invention. Detailed Implementation

[0011] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0012] It should be understood that, when used in this specification and the appended claims, the terms "comprising" and "including" indicate the presence of the described features, integrals, steps, operations, elements and / or components, but do not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components and / or collections thereof.

[0013] It should also be understood that the terminology used in this specification is for the purpose of describing particular embodiments only and is not intended to limit the invention. As used in this specification and the appended claims, the singular forms “a,” “an,” and “the” are intended to include the plural forms unless the context clearly indicates otherwise.

[0014] It should also be further understood that the term "and / or" as used in this specification and the appended claims refers to any combination of one or more of the associated listed items and all possible combinations, and includes such combinations.

[0015] Please see below. Figure 1 The present invention provides an image processing method for decoupling cross-modal migration point spread function, which specifically includes steps S101 to S106.

[0016] Step S101: Select a calibration sample and record the first modal image stack and the second modal image stack of the calibration sample under the low-performance first imaging modal system and the high-performance second imaging modal system, respectively. Step S102: Preprocess the first modal image stack to obtain the emission field estimation result of the calibration sample in the first imaging modality; Step S103: Spatial registration is performed between the emission field estimation result and the second modality image stack; Step S104: Use the registered emission field estimation results and the second modality image stack as the pairing body data, and use the pairing body data as physical constraints to establish an inverse problem model; Step S105: Optimize and solve the inverse problem model to obtain the migration point spread function that characterizes the mapping relationship from the first imaging mode to the second imaging mode; Step S106: For the test sample, obtain the first modal image stack of the test sample, and perform inversion reconstruction based on the migration point diffusion function to obtain the second imaging modal equivalent image corresponding to the test sample.

[0017] In this embodiment, combined with Figure 2 First, for the selected calibration samples, corresponding image stacks are acquired under the first and second imaging modal systems, respectively. Next, the first modal image stack is preprocessed to obtain the emission field estimation result. Then, this emission field estimation result is spatially registered with the second modal image stack. The registered data are then used as paired volume data, and an inverse problem model is established based on this as a physical constraint. By optimizing and solving this inverse problem model, a migration point spread function characterizing the mapping relationship from the first to the second imaging modality is obtained. Finally, for the tested sample, its corresponding first modal image stack is acquired, and inversion reconstruction is performed using the aforementioned migration point spread function to obtain the equivalent image of the tested sample in the second imaging modality.

[0018] This embodiment, based on a common emission field formation mechanism, achieves computational migration and reconstruction from a low-performance imaging modality to a high-performance imaging modality by constructing sample priors and decoupling the migration point spread function. This invention not only overcomes the technical problem of difficulty in connecting traditional imaging modalities but also integrates the signal formation characteristics and imaging advantages of different imaging modalities, providing strong technical support for high-performance computational reconstruction.

[0019] Here, the first imaging mode described in this embodiment is a low-performance imaging mode, and the second imaging mode is a high-performance imaging mode. The low-performance imaging mode has at least one of the following characteristics: fast acquisition speed, simple system structure, or low imaging cost. The high-performance imaging mode has at least one of the following characteristics: higher resolution, stronger optical slicing capability, or better background suppression capability. Correspondingly, the first mode image stack is a low-performance mode, such as ordinary fluorescence wide-field, which has the characteristics of image stack blur, a large amount of defocus blur crosstalk, and poor imaging effect. The second mode image stack is a high-performance mode such as confocal microscopy, light sheet microscopy, and structured illumination microscopy (SIM), which has the characteristics of complex structure but good imaging effect. It eliminates defocus blur and obtains high-quality images by improving the structure or imaging mechanism.

[0020] Furthermore, when selecting calibration samples, conventional fluorescent samples are specifically used as calibration samples; wherein, the feature size of the conventional fluorescent sample is larger than the diffraction limit of the microscopic imaging system. These conventional fluorescent samples are used to acquire the first imaging modality image stack and the second imaging modality image stack under actual imaging conditions, and serve as the paired samples for subsequent migration point spread function calibration.

[0021] In one embodiment, the preprocessing of the first modality image stack to obtain the emission field estimation result of the calibrated sample in the first imaging modality includes: Obtain the system point spread function of the first imaging modality system; The system point spread function is used to perform a non-blind deconvolution operation on the first modal image stack to obtain the emission field estimation result of the calibration sample in the first imaging modality.

[0022] Specifically, obtaining the system point spread function of the first imaging modality system includes: A theoretical point spread function model is constructed based on the system parameters of the first imaging modality system to obtain the system point spread function; Alternatively, the point spread function of the first imaging modality system can be obtained by experimentally measuring the three-dimensional point spread function of the first imaging modality system using imaging subresolution fluorescent microspheres.

[0023] In this embodiment, the system point spread function of the first imaging modality is obtained. For example, a theoretical point spread function model is constructed based on the optical parameters of the system (such as the numerical aperture of the objective lens, the excitation wavelength, and the refractive index of the imaging medium), or a three-dimensional system point spread function is obtained by experimentally measuring and fitting using sub-resolution fluorescent microspheres as a point light source. Then, this system point spread function is used to perform a non-blind deconvolution operation on the image stack of the first modality, for example, by using the Richardson-Lucy algorithm or Wiener filtering, thereby effectively removing the inherent defocus blur and optical aberrations of the first imaging modality, compensating for the diffraction response of the imaging system, and obtaining an estimate of the true emission field of the calibration sample.

[0024] In one embodiment, spatial registration of the emission field estimation result with the second modality image stack includes: The emission field estimation result is resampled to make the emission field estimation result consistent with the sampling grid of the second modality image stack; Using the second modality image stack as a geometric reference, a coarse-to-fine rigid registration strategy is adopted to spatially align the emission field estimation results with the second modality image stack; wherein, the coarse-to-fine rigid registration strategy includes two-dimensional projection coarse registration and three-dimensional volume data fine registration.

[0025] In this embodiment, during spatial registration, the emission field estimation result is first resampled to ensure that its sampling grid is completely consistent with that of the second modality image stack. Next, using the second modality image stack as a geometric reference standard, a coarse-to-fine rigid registration strategy is employed to achieve spatial alignment between the two. Specifically, coarse two-dimensional projection registration can be performed by projecting the emission field estimation result and the second modality image stack with maximum intensity or average intensity, respectively, to obtain two-dimensional projected images. Then, preliminary rigid transformations such as translation and rotation are performed based on these two-dimensional projected images to quickly reduce spatial position differences. After completing the coarse two-dimensional registration, fine three-dimensional volume data registration is performed. Using three-dimensional mutual information or feature-point-based registration algorithms, the three-dimensional volume data undergoes more refined spatial adjustments to eliminate any possible minor translation, rotation, or scaling differences, ensuring high-precision alignment between the emission field estimation result and the second modality image stack in three-dimensional space. This guarantees the accuracy and reliability of subsequently establishing an inverse problem model using both as paired volume data.

[0026] In one embodiment, optimizing the inverse problem model to obtain the migration point spread function characterizing the mapping relationship from the first imaging mode to the second imaging mode includes: The inverse problem model is solved using an iterative optimization approach, and nonnegativity constraints and normalization constraints are applied during the solution process to obtain the migration point diffusion function.

[0027] The migration point spread function described in this embodiment is a migration operator used to characterize the mapping process from the emission field estimation result in the first imaging mode to the image in the second imaging mode. The migration point spread function uses the paired body data obtained by the calibration sample in the first and second imaging modes as physical constraints, establishes an inverse problem model based on the emission field estimation result of the calibration sample and the image in the second imaging mode, and obtains it through iterative optimization.

[0028] Furthermore, this embodiment ensures the rationality of the physical meaning of the migration point spread function by applying nonnegativity and normalization constraints during the solution process. The nonnegativity constraint requires that all element values ​​of the migration point spread function be greater than or equal to zero, because the point spread function, as the impulse response of an optical system, cannot physically have negative values. The normalization constraint normalizes the energy of the migration point spread function so that its integral in the spatial domain is 1, ensuring energy conservation and preventing unreasonable gain or attenuation of signal energy during the mapping process.

[0029] It should also be noted that the migration point diffusion function is used repeatedly as a fixed operator under the same imaging conditions, so that subsequent tested samples do not need to be repeatedly acquired in the second imaging modality, and the equivalent image of the second imaging modality can be obtained based solely on the image stack of the first modality.

[0030] In one embodiment, the step of inverting and reconstructing based on the migration point spread function to obtain the equivalent image of the second imaging modality corresponding to the tested sample includes: The first modality image stack of the test sample is subjected to non-blind deconvolution preprocessing to obtain the emission field estimation result of the test sample; The emission field estimation results of the tested sample are inverted using a pre-calibrated migration point diffusion function to obtain the equivalent image of the second imaging mode.

[0031] In this embodiment, after obtaining the migration point spread function, the first step is to acquire the first modal image stack of the tested sample under the first imaging modality system. Next, using the same method as the preprocessing of the calibration samples, the system point spread function of the first imaging modality system is obtained, and a non-blind deconvolution operation is performed on the first modal image stack of the tested sample using this system point spread function to obtain the emission field estimation result of the tested sample under the first imaging modality. Subsequently, this emission field estimation result is convolved with the migration point spread function obtained in the pre-calibration process. Through this inversion calculation step, the emission field estimation result of the first imaging modality of the tested sample can be mapped to the image space of the second imaging modality, ultimately reconstructing a high-quality image equivalent to the second imaging modality. This process fully utilizes the inter-modal mapping relationship represented by the migration point spread function, enabling imaging effects comparable to the high-performance second imaging modality to be obtained using only the low-performance first imaging modality acquisition, effectively reducing dependence on expensive high-performance imaging equipment and improving imaging efficiency.

[0032] In one specific embodiment, taking a conventional fluorescent sample as the calibration sample, the image processing method provided in this embodiment may specifically include the following steps: S1: Select a conventional fluorescence sample as the calibration sample, and acquire the wide-field image stack (first mode image stack) and confocal image stack (second mode image stack) of the calibration sample under a wide-field fluorescence microscopy system (first imaging mode system) and a confocal fluorescence microscopy system (second imaging mode system), respectively.

[0033] Here, let the sample distribution of the calibration sample be... ,in, Represents three-dimensional spatial coordinates; the wide-field image stack obtained by the calibration sample in the first imaging mode is denoted as... The confocal image stack obtained in the second imaging mode is denoted as .

[0034] S2: Preprocess the wide-field image stack of the calibration samples to remove the influence of the point spread function of the wide-field imaging system on the imaging results and obtain the emission field estimation results of the calibration samples, which can then be used as sample priors for subsequent point spread function decoupling.

[0035] Here, the wide-field image stack can be represented as: ; in, This represents the luminescence field distribution under wide-field conditions. The system point spread function represents the system point spread function of a wide-field fluorescence microscopy system. This represents the convolution operation.

[0036] Furthermore, by performing non-blind deconvolution processing on the wide-field image stack, the emission field estimation results of the calibrated samples can be obtained: ; in, This represents the emission field estimation results of the calibration sample. This indicates a non-blind deconvolution operation.

[0037] Preferably, the system point spread function of the wide-field fluorescence microscopy system It can be obtained through theoretical modeling or through sub-resolution fluorescent microsphere experimental measurements.

[0038] S3: Since the wide-field image stack and the confocal image stack may differ in sampling interval and spatial coordinate system, it is necessary to spatially register the emission field estimation results with the confocal image stack before performing migration point diffusion function decoupling in order to establish a unified coordinate relationship.

[0039] First, the emission field estimation results are resampled to match the sampling grid of the confocal image stack, which can be expressed as: ; in, and These represent the axial sampling intervals of the wide-field image stack and the confocal image stack, respectively. This indicates a cubic spline interpolation operation. This represents the emission field estimation result after resampling using cubic spline interpolation, i.e., the axial sampling interval is adjusted to Δz. cf Estimation results of the emission field at (confocal sampling interval). The axial sampling interval is Δz wf The emission field estimation results for (wide field sampling interval).

[0040] Furthermore, using the wide-field emission field estimation result as a reference, rigid registration is performed between the resampled emission field estimation result and the confocal image stack. Preferably, a two-stage registration strategy from coarse to fine can be adopted, including coarse registration of two-dimensional projection and fine registration of three-dimensional volume data, ultimately obtaining the registered confocal image stack. .

[0041] S4: The migration point spread function is used to characterize the mapping relationship between the wide-field emission field estimation result and the confocal image. After registration, the emission field estimation result of the calibrated sample and the confocal image satisfy the following: ; in, This represents the migration point diffusion function in three-dimensional spatial coordinates.

[0042] Furthermore, using the registered emission field estimation results and the confocal image stack as paired data, and using them as physical constraints, an inverse problem model is established to optimize the migration point spread function. The migration point spread function can be solved using the following optimization objective: ; in, Represents the square of the L2 norm. h represents the final optimized result of the migration point spread function. t This represents the migration point diffusion function.

[0043] The migration point diffusion function can be solved using an iterative optimization approach. Preferably, its update form can be expressed as: ; Where k represents the k-th iteration, C represents reg The adjoint operator.

[0044] Preferably, during the solution process, nonnegativity constraints and normalization constraints are applied to the migration point diffusion function to ensure the physical rationality of the solution results.

[0045] S5: Acquire the wide-field image stack of the sample under test in a wide-field fluorescence microscope system, denoted as... First, using the same preprocessing method as in the calibration stage, non-blind deconvolution processing is performed on the wide-field image stack of the test sample to obtain the emission field estimation result of the test sample: ; Then, based on the migration point diffusion function obtained in the calibration stage, the emission field estimation results of the tested sample are inverted and reconstructed to obtain the confocal equivalent image corresponding to the tested sample: ; in, This represents the confocal equivalent image corresponding to the tested sample.

[0046] Combination Figure 2 It can be seen that, Figure 2 The upper dashed box represents the calibration stage, corresponding to steps S1 to S4 above. The calibration stage is used to decouple the migration point diffusion function based on the calibration sample, and usually only needs to be performed once under the same imaging conditions. Figure 2 The lower dashed box represents the reconstruction stage, corresponding to step S5. The reconstruction stage is used to perform cross-modal migration reconstruction on the test sample based on the migration point diffusion function obtained by decoupling. Step S5 is executed once each time the test sample is changed.

[0047] This embodiment establishes a mapping relationship between wide-field fluorescence microscopy and confocal fluorescence microscopy based on sample priors, and achieves cross-modal inversion reconstruction through the migration point spread function. This method can achieve imaging results close to those of confocal fluorescence microscopy while retaining the advantages of wide-field fluorescence microscopy, such as fast acquisition speed and relatively simple system structure, providing a new approach to computational imaging.

[0048] Figure 3 A schematic block diagram of an image processing apparatus 300 for cross-modal migration point spread function decoupling provided in an embodiment of the present invention. The apparatus 300 includes: The stack recording unit 301 is used to select calibration samples and record the first modal image stack and the second modal image stack of the calibration samples in a low-performance first imaging modal system and a high-performance second imaging modal system, respectively. The stack preprocessing unit 302 is used to preprocess the first modal image stack to obtain the emission field estimation result of the calibration sample in the first imaging modality. Spatial registration unit 303 is used to spatially register the emission field estimation result with the second modality image stack; The model building unit 304 is used to take the registered emission field estimation result and the second modality image stack as the pairing body data, and use the pairing body data as physical constraints to build an inverse problem model. The optimization and solution unit 305 is used to optimize and solve the inverse problem model to obtain the migration point spread function that characterizes the mapping relationship from the first imaging mode to the second imaging mode; The inversion and reconstruction unit 306 is used to acquire the first modal image stack of the test sample and perform inversion and reconstruction based on the migration point diffusion function to obtain the second imaging modal equivalent image corresponding to the test sample.

[0049] In one embodiment, the stack preprocessing unit 302 includes: The point spread function acquisition unit is used to acquire the system point spread function of the first imaging mode system; The point spread function operation unit is used to perform non-blind deconvolution operation on the first modal image stack using the system point spread function to obtain the emission field estimation result of the calibration sample in the first imaging modality.

[0050] In one embodiment, the point spread function acquisition unit includes: The model building unit is used to construct a theoretical point spread function model based on the system parameters of the first imaging modality system, so as to obtain the system point spread function; An experimental measurement unit is used to, or to, experimentally measure the three-dimensional point spread function of the first imaging modality system using imaging subresolution fluorescent microspheres, in order to obtain the point spread function of the system.

[0051] In one embodiment, the spatial registration unit 303 includes: The result resampling unit is used to resample the emission field estimation result so that the emission field estimation result is consistent with the sampling grid of the second modality image stack; A spatial alignment unit is used to spatially align the emission field estimation result with the second modality image stack using a coarse-to-fine rigid registration strategy, with the second modality image stack as a geometric reference; wherein the coarse-to-fine rigid registration strategy includes two-dimensional projection coarse registration and three-dimensional volume data fine registration.

[0052] In one embodiment, the optimization solution unit 305 includes: The constraint-applying unit is used to solve the inverse problem model using an iterative optimization method, and during the solution process, applies non-negativity constraints and normalization constraints to obtain the migration point diffusion function.

[0053] In one embodiment, the inversion reconstruction unit 306 includes: The sample preprocessing unit is used to perform non-blind deconvolution preprocessing on the first modality image stack of the test sample to obtain the emission field estimation result of the test sample. The inversion calculation unit is used to perform inversion calculation on the emission field estimation result of the measured sample using a pre-calibrated migration point diffusion function to obtain the equivalent image of the second imaging mode.

[0054] In one embodiment, the stack recording unit 301 includes: The sample adopts a unit for using a conventional fluorescent sample as the calibration sample; wherein the feature size of the conventional fluorescent sample is greater than the diffraction limit of the microscopic imaging system.

[0055] Since the embodiments of the apparatus and the embodiments of the method correspond to each other, please refer to the description of the embodiments of the method for the embodiments of the apparatus, which will not be repeated here.

[0056] This invention also provides a computer-readable storage medium storing a computer program thereon, which, when executed, can perform the steps provided in the above embodiments. The storage medium may include various media capable of storing program code, such as a USB flash drive, a portable hard drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0057] This invention also provides a computer device, which may include a memory and a processor. The memory stores a computer program, and when the processor calls the computer program in the memory, it can implement the steps provided in the above embodiments. Of course, the computer device may also include various network interfaces, power supplies, and other components.

[0058] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the systems disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the descriptions are relatively simple; relevant parts can be referred to in the method section. It should be noted that those skilled in the art can make various improvements and modifications to this application without departing from the principles of this application, and these improvements and modifications also fall within the protection scope of the claims of this application.

[0059] It should also be noted that, in this specification, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

Claims

1. An image processing method for decoupling cross-modal migration point spread functions, characterized in that, include: Select calibration samples, and record the first modal image stack and the second modal image stack of the calibration samples under the low-performance first imaging modal system and the high-performance second imaging modal system, respectively; Preprocess the first modal image stack to obtain the emission field estimation result of the calibration sample in the first imaging modality; Spatial registration is performed between the emission field estimation result and the second modality image stack; The registered emission field estimation results and the second modality image stack are used as the pairing body data, and the inverse problem model is established using the pairing body data as physical constraints. The inverse problem model is optimized and solved to obtain the migration point spread function that characterizes the mapping relationship from the first imaging mode to the second imaging mode; For the tested sample, the first modal image stack of the tested sample is obtained, and inversion reconstruction is performed based on the migration point spread function to obtain the second imaging modal equivalent image corresponding to the tested sample.

2. The image processing method for decoupling cross-modal migration point spread function according to claim 1, characterized in that, The preprocessing of the first modality image stack to obtain the emission field estimation result of the calibrated sample in the first imaging modality includes: Obtain the system point spread function of the first imaging modality system; The system point spread function is used to perform a non-blind deconvolution operation on the first modal image stack to obtain the emission field estimation result of the calibration sample in the first imaging modality.

3. The image processing method for decoupling cross-modal migration point spread function according to claim 2, characterized in that, The acquisition of the system point spread function of the first imaging modality system includes: A theoretical point spread function model is constructed based on the system parameters of the first imaging modality system to obtain the system point spread function; Alternatively, the point spread function of the first imaging modality system can be obtained by experimentally measuring the three-dimensional point spread function of the first imaging modality system using imaging subresolution fluorescent microspheres.

4. The image processing method for decoupling cross-modal migration point spread function according to claim 1, characterized in that, The spatial registration of the emission field estimation result with the second modality image stack includes: The emission field estimation result is resampled to make the emission field estimation result consistent with the sampling grid of the second modality image stack; Using the second modality image stack as a geometric reference, a coarse-to-fine rigid registration strategy is adopted to spatially align the emission field estimation results with the second modality image stack; wherein, the coarse-to-fine rigid registration strategy includes two-dimensional projection coarse registration and three-dimensional volume data fine registration.

5. The image processing method for decoupling cross-modal migration point spread function according to claim 1, characterized in that, The optimization solution of the inverse problem model yields the migration point spread function, which characterizes the mapping relationship from the first imaging mode to the second imaging mode, including: The inverse problem model is solved using an iterative optimization approach, and nonnegativity constraints and normalization constraints are applied during the solution process to obtain the migration point diffusion function.

6. The image processing method for decoupling cross-modal migration point spread function according to claim 1, characterized in that, The inversion and reconstruction based on the migration point spread function to obtain the equivalent image of the second imaging modality corresponding to the tested sample includes: The first modality image stack of the test sample is subjected to non-blind deconvolution preprocessing to obtain the emission field estimation result of the test sample; The emission field estimation results of the tested sample are inverted using a pre-calibrated migration point diffusion function to obtain the equivalent image of the second imaging mode.

7. The image processing method for decoupling cross-modal migration point spread function according to claim 1, characterized in that, The selection of calibration samples includes: A conventional fluorescent sample is used as the calibration sample; wherein the feature size of the conventional fluorescent sample is greater than the diffraction limit of the microscopic imaging system.

8. An image processing apparatus for cross-modal migration point spread function decoupling, characterized in that, include: A stack recording unit is used to select calibration samples and record the first modal image stack and the second modal image stack of the calibration samples in a low-performance first imaging modal system and a high-performance second imaging modal system, respectively. The stack preprocessing unit is used to preprocess the first modal image stack to obtain the emission field estimation result of the calibration sample in the first imaging modality; A spatial registration unit is used to spatially register the emission field estimation result with the second modality image stack. The model building unit is used to take the registered emission field estimation results and the second modality image stack as the pairing body data, and use the pairing body data as physical constraints to build an inverse problem model. An optimization and solution unit is used to optimize and solve the inverse problem model to obtain the migration point spread function that characterizes the mapping relationship from the first imaging mode to the second imaging mode; The inversion and reconstruction unit is used to acquire the first modal image stack of the test sample and perform inversion and reconstruction based on the migration point diffusion function to obtain the second imaging modal equivalent image corresponding to the test sample.

9. A computer device, characterized in that, The method includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor, when executing the computer program, implements the image processing method for decoupling the cross-modal migration point spread function as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the image processing method for decoupling cross-modal migration point spread function as described in any one of claims 1 to 7.

Images