Lung ventilation analysis method and system based on CT and MRI

Abstract

The invention discloses a lung ventilation analysis method and system based on CT (computed tomography) and MRI (magnetic resonance imaging), which are characterized in that comprehensive acquisition of anatomical structure and functional information is realized by fusing CT and MRI images, that is, accurate anatomical structure information in the CT is mapped to an MRI space to obtain an MRI lung lobe segmentation image; then, the MRI UTE inspiration phase image and the MRI UTE expiration phase image are used for obtaining accurate function information, namely, the breathing deformation field of the lung of the person is determined; and finally, generating a lung ventilation graph of the person by using the MRI lung lobe segmentation image and the breathing deformation field. Therefore, the problem that modal information of a single image is insufficient in the traditional technology is solved, the accuracy of ventilation graph generation is improved, meanwhile, the limitation that an exogenous contrast agent is additionally needed for generation of the ventilation graph through the MRI technology is also solved, and therefore the use flexibility is improved; on the basis, the invention provides the lung ventilation graph generation method which is flexible to use and high in accuracy, and is very suitable for large-scale application and popularization.

Description

Technical Field

[0001] The present invention belongs to the technical field of medical image processing, and in particular relates to a lung ventilation analysis method and system based on CT and MRI. Background Art

[0002] At present, lung medical image processing technology mainly relies on CT or X-ray imaging to generate lung ventilation maps and display them to medical staff to help them evaluate lung ventilation function. Among them, CT is radioactive and not suitable for repeated follow-up, while magnetic resonance imaging is radiation-free. Therefore, studies have applied MRI to lung ventilation assessment, such as hyperpolarized gas MRI and oxygen-enhanced MRI, which have shown its potential in non-invasive functional assessment. In actual application, the generation method of CT-based lung ventilation maps is mainly based on the volume change method, that is, by aligning 4D-CT images to obtain local volume changes of the lungs, thereby generating lung ventilation maps.

[0003] However, the aforementioned methods have the following shortcomings: CT mainly provides anatomical information, and the acquisition of functional information is relatively limited, and it is impossible to obtain accurate anatomical and functional information at the same time; this will affect the accuracy of ventilation map generation; and although MRI has the advantage of being radiation-free, it usually requires exogenous contrast agents when generating ventilation maps (such as hyperpolarized gas MRI and oxygen-enhanced MRI), which has certain limitations; therefore, based on the aforementioned shortcomings, how to provide a CT- and MRI-based lung ventilation analysis method that is flexible in use and can generate accurate lung ventilation maps has become an urgent problem to be solved. Summary of the Invention

[0004] The purpose of the present invention is to provide a lung ventilation analysis method and system based on CT and MRI to solve the problems of poor accuracy in the existing technology of using CT images to generate ventilation maps and the limitations of using MRI images to generate ventilation maps.

[0005] In order to achieve the above object, the present invention adopts the following technical solutions:

[0006] In a first aspect, a lung ventilation analysis method based on CT and MRI is provided, comprising:

[0007] Acquiring lung image data of a target person, wherein the lung image data includes a lung CT image and a lung MRI image, and the lung MRI image includes an MRI UTE inspiratory phase image and an MRI UTE expiratory phase image;

[0008] performing image segmentation processing on the lung CT image to obtain a CT lung lobe segmentation image of the target person;

[0009] Based on the lung CT image and the lung MRI image, converting the CT lung lobe segmentation image into MRI space to obtain an MRI lung lobe segmentation image;

[0010] Determining a respiratory deformation field based on the MRI UTE inspiratory phase image and the MRI UTE expiratory phase image, wherein the respiratory deformation field is used to represent pixel position transformation information between the MRI UTE inspiratory phase image and the MRI UTE expiratory phase image;

[0011] A lung ventilation map of the target person is generated using the MRI lung lobe segmentation image and the respiratory deformation field, and the lung ventilation map is visualized.

[0012] Based on the above-disclosed content, the present invention first obtains a lung CT image of a target person and a lung MRI image including an MRI UTE inspiratory phase image and an MRI UTE expiratory phase image; then, performs image segmentation on the lung CT image to obtain a CT lung lobe segmentation image of the target person; then, based on the lung CT image and the lung MRI image, the CT lung lobe segmentation image is converted to MRI space to obtain an MRI lung lobe segmentation image; then, the MRI UTE inspiratory phase image and the MRI UTE expiratory phase image are used to derive a respiratory deformation field of the target person, thereby obtaining pixel position transformation information of the target person's lungs caused by breathing; finally, based on the aforementioned MRI lung lobe segmentation image and the respiratory deformation field, a lung ventilation map of the target person can be generated.

[0013] Through the above design, the present invention realizes the comprehensive acquisition of anatomical structure and functional information by fusing CT and MRI images, that is, mapping the precise anatomical structure information in CT to MRI space to obtain MRI lung lobe segmentation images; then, using MRI UTE inspiratory phase images and MRI UTE expiratory phase images, to obtain precise functional information, that is, to determine the respiratory deformation field of the target person's lungs; finally, using MRI lung lobe segmentation images and respiratory deformation fields, a lung ventilation map of the target person can be generated; thus, the present invention solves the problem of insufficient single image modality information in traditional technology, improves the accuracy of ventilation map generation, and at the same time, solves the limitation of existing MRI technology that additional exogenous contrast agents are required to generate ventilation maps, thereby improving the flexibility of use; based on this, the present invention provides a flexible and highly accurate lung ventilation map generation method, and is therefore very suitable for large-scale application and promotion.

[0014] In one possible design, based on the lung CT image and the lung MRI image, converting the CT lung lobe segmentation image into MRI space to obtain an MRI lung lobe segmentation image includes:

[0015] performing global rigid registration on the lung CT image and a target image to obtain a registered lung CT image, wherein the target image is the MRI UTE inspiratory phase image or the MRI UTE expiratory phase image;

[0016] performing affine registration processing on the registered lung CT image and the target image to obtain a coordinate transformation matrix between the lung CT image and the target image after the affine registration processing;

[0017] The CT lung lobe segmentation image is subjected to coordinate transformation processing according to the coordinate transformation matrix, so as to obtain the MRI lung lobe segmentation image after the coordinate transformation processing.

[0018] In a possible design, a respiratory deformation field is determined based on an MRI UTE inspiratory phase image and an MRI UTE expiratory phase image, including:

[0019] performing non-rigid registration processing on the MRI UTE inspiratory phase image and the MRI UTE expiratory phase image, so as to obtain a pixel transformation matrix between the MRI UTE inspiratory phase image and the MRI UTE expiratory phase image after the non-rigid registration processing, and using the pixel transformation matrix as the respiratory deformation field;

[0020] Each element in the pixel transformation matrix corresponds to a pixel point in the MRI UTE inspiratory phase image, each element corresponds to a three-dimensional matrix, and the three-dimensional matrix is ​​used to represent the displacement information of the corresponding pixel point in three-dimensional space.

[0021] In one possible design, generating a lung ventilation map of the target person using the MRI lung lobe segmentation image and the respiratory deformation field includes:

[0022] Calculating a local lung ventilation matrix of the target person based on the respiratory deformation field;

[0023] A lung ventilation map of the target person is generated according to the lung local ventilation matrix and the MRI lung lobe segmentation image.

[0024] In one possible design, the respiratory deformation field is a pixel transformation matrix between the MRI UTE inspiratory phase image and the MRI UTE expiratory phase image, wherein each element in the pixel transformation matrix corresponds to a pixel point in the MRI UTE inspiratory phase image, each element corresponds to a three-dimensional matrix, and the three-dimensional matrix is ​​used to represent the displacement information of the corresponding pixel point in three-dimensional space;

[0025] The method of calculating the target person's lung local ventilation matrix based on the respiratory deformation field includes:

[0026] According to the following formula (1), the local lung ventilation matrix of the target person is calculated;

[0027]

[0028] In the above formula (1), J(x) represents the target person's lung local ventilation matrix, T represents the respiratory deformation field, and x represents an element in the respiratory deformation field. represents the Jacobian matrix of the breathing deformation field, and det() represents the determinant calculation function.

[0029] In one possible design, generating a lung ventilation map of a target person according to the lung local ventilation matrix and the MRI lung lobe segmentation image includes:

[0030] Generate the lung ventilation map of the target person according to the following formula (2);

[0031] V=mask*(J(x)-1) (2)

[0032] In the above formula (2), V represents the lung ventilation map of the target person, mask represents the MRI lung lobe segmentation image, and J(x) represents the local lung ventilation matrix.

[0033] In one possible design, performing image segmentation processing on the lung CT image to obtain a CT lung lobe segmentation image of the target person includes:

[0034] Acquire a lung lobe segmentation model, wherein the lung lobe segmentation model is trained by taking a plurality of lung CT sample images as input and outputting lung lobe segmentation images corresponding to each lung CT sample image;

[0035] The lung CT image is input into the lung lobe segmentation model to obtain a CT lung lobe segmentation image of the target person.

[0036] In a possible design, the lung lobe segmentation model is trained in the following manner:

[0037] Acquire an initial training data set, wherein the initial training data set includes a plurality of initial lung CT sample images;

[0038] resampling each initial lung CT sample image in the initial training data set to obtain a plurality of resampled lung CT sample images;

[0039] performing intensity normalization processing on a number of resampled lung CT sample images to obtain a number of normalized lung CT sample images;

[0040] Performing data enhancement processing on a number of normalized lung CT sample images to obtain a number of lung CT sample images, and using the number of lung CT sample images to form a training data set;

[0041] The neural network model is trained using each lung CT sample image in the training data set as input and the CT lung lobe segmentation image corresponding to each lung CT sample image as output, so as to obtain the lung lobe segmentation model after the training is completed.

[0042] In one possible design, the loss function of the lung lobe segmentation model is:

[0043]

[0044] In formula (3), L represents the loss function, p i represents the segmentation category of the i-th pixel in each lung CT sample image, g i Represents the segmentation label value of the i-th pixel in each lung CT sample image, and the segmentation category is 1 or 0.

[0045] Secondly, a CT and MRI-based lung ventilation analysis system is provided, comprising:

[0046] An image acquisition device is used to acquire lung image data of a target person, wherein the lung image data includes a lung CT image and a lung MRI image, and the lung MRI image includes an MRI UTE inspiratory phase image and an MRI UTE expiratory phase image;

[0047] an image processing device, configured to perform image segmentation processing on the lung CT image to obtain a CT lung lobe segmentation image of the target person;

[0048] An image processing device is used to convert the CT lung lobe segmentation image into MRI space based on the lung CT image and the lung MRI image to obtain an MRI lung lobe segmentation image;

[0049] An image processing device is configured to determine a respiratory deformation field based on the MRI UTE inspiratory phase image and the MRI UTE expiratory phase image, wherein the respiratory deformation field is used to represent pixel position transformation information between the MRI UTE inspiratory phase image and the MRI UTE expiratory phase image;

[0050] The image processing device is also used to generate a lung ventilation map of the target person using the MRI lung lobe segmentation image and the respiratory deformation field, and to visualize the lung ventilation map.

[0051] In a third aspect, a pulmonary ventilation analysis device based on CT and MRI is provided. Taking the device as an electronic device as an example, it includes a memory, a processor and a transceiver that are communicatively connected in sequence, wherein the memory is used to store computer programs, the transceiver is used to send and receive messages, and the processor is used to read the computer program and execute the pulmonary ventilation analysis method based on CT and MRI as described in the first aspect or any possible design of the first aspect.

[0052] In a fourth aspect, a storage medium is provided, on which instructions are stored. When the instructions are run on a computer, the CT and MRI-based lung ventilation analysis method as described in the first aspect or any possible design of the first aspect is executed.

[0053] In a fifth aspect, a computer program product comprising instructions is provided, which, when executed on a computer, causes the computer to execute the CT and MRI-based lung ventilation analysis method as described in the first aspect or any possible design of the first aspect.

[0054] Beneficial effects:

[0055] (1) The present invention realizes the comprehensive acquisition of anatomical structure and functional information by fusing CT and MRI images, that is, mapping the precise anatomical structure information in CT to MRI space to obtain MRI lung lobe segmentation images; then, using MRI UTE inspiratory phase images and MRI UTE expiratory phase images, accurate functional information is obtained, that is, the respiratory deformation field of the target person's lungs is determined; finally, using MRI lung lobe segmentation images and respiratory deformation fields, a lung ventilation map of the target person can be generated; thus, the present invention solves the problem of insufficient single image modality information in traditional technology, improves the accuracy of ventilation map generation, and at the same time, solves the limitation of existing MRI technology that additional exogenous contrast agents are required to generate ventilation maps, thereby improving the flexibility of use; based on this, the present invention provides a method for generating lung ventilation maps that is flexible and accurate, and is therefore very suitable for large-scale application and promotion.

[0056] (2) The present invention solves the problem of radiation dose accumulation in CT follow-up assessment and establishes a repeatable, low-radiation ventilation map construction system, thereby ensuring the safety of personnel. BRIEF DESCRIPTION OF THE DRAWINGS

[0057] Figure 1 A schematic flow chart of the steps of a CT and MRI-based lung ventilation analysis method provided in an embodiment of the present invention;

[0058] Figure 2 A schematic diagram of the structure of a CT and MRI-based lung ventilation analysis system provided in an embodiment of the present invention;

[0059] Figure 3 A schematic structural diagram of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0060] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the present invention will be briefly introduced below in conjunction with the drawings and the description of the embodiments or the prior art. Obviously, the following description of the structure of the drawings is only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative work. It should be noted that the description of these embodiments is used to help understand the present invention, but does not constitute a limitation of the present invention.

[0061] It should be understood that although the terms "first," "second," etc. may be used herein to describe various elements, these elements should not be limited by these terms. These terms are only used to distinguish one element from another. For example, a first element can be referred to as a second element, and similarly, a second element can be referred to as a first element without departing from the scope of the exemplary embodiments of the present invention.

[0062] It should be understood that the term "and / or" that may appear in this document is merely a description of the association relationship between associated objects, indicating that three relationships may exist. For example, A and / or B may indicate three situations: A exists alone, B exists alone, and A and B exist at the same time. The term " / and" that may appear in this document describes another type of association object relationship, indicating that two relationships may exist. For example, A / and B may indicate two situations: A exists alone, and A and B exist alone. In addition, the character " / " that may appear in this document generally indicates that the previous and subsequent associated objects are in an "or" relationship.

[0063] Example:

[0064] See also Figure 1As shown, the CT and MRI-based lung ventilation analysis method provided in this embodiment achieves comprehensive acquisition of anatomical and functional information by fusing CT and MRI images. Specifically, the precise anatomical information in CT is mapped to MRI space to obtain MRI lung lobe segmentation images. Subsequently, the MRI UTE inspiratory phase image and the MRI UTE expiratory phase image are used to obtain precise functional information, that is, to determine the respiratory deformation field of the target person's lungs. Finally, the MRI lung lobe segmentation image and the respiratory deformation field are used to generate a lung ventilation map of the target person. Thus, the method solves the problem of insufficient single image modality information in traditional technologies, improves the accuracy of ventilation map generation, and also solves the limitation of existing MRI technology that requires additional exogenous contrast agents to generate ventilation maps, thereby improving the flexibility of use. Therefore, the method is very suitable for large-scale application and promotion. For example, the method can be, but is not limited to, running on an image processing device. It is understandable that the image processing device can be, but is not limited to, a personal computer. Of course, the aforementioned execution subject does not constitute a limitation on the embodiments of the present application. Accordingly, the operation steps of the method can be, but are not limited to, the following steps S1 to S5.

[0065] S1. Acquire lung image data of a target person, wherein the lung image data includes a lung CT image and a lung MRI image, and the lung MRI image includes an MRI UTE inspiratory phase image and an MRI UTE expiratory phase image. In this embodiment, for example, but not limited to, the target person's lung CT image can be acquired by a CT device, and the target person's lung MRI image can be acquired by a nuclear magnetic resonance device. In this way, the CT device and the nuclear magnetic resonance device can send the acquired images to an image processing device, so that the subsequent image processing device can generate a lung ventilation map of the target person based on the received lung image data.

[0066] After obtaining the lung image data of the target person, this embodiment first uses the lung CT image to perform lung lobe segmentation, thereby obtaining the anatomical structure information of the target person's lungs, that is, the CT lung lobe segmentation image; wherein, the image segmentation process can be, but is not limited to, as shown in the following step S2.

[0067] S2. Perform image segmentation processing on the lung CT image to obtain a CT lung lobe segmentation image of the target person; in a specific application, for example, but not limited to, first obtaining a lung lobe segmentation model, wherein the lung lobe segmentation model is trained with a plurality of lung CT sample images as input and lung lobe segmentation images corresponding to each lung CT sample image as output; then, the lung CT image is input into the lung lobe segmentation model to obtain a CT lung lobe segmentation image of the target person.

[0068] Optionally, for example, the lung lobe segmentation model can be, but is not limited to, a trained nnUNet (no-new-Net) model, which is an automated framework based on the U-Net structure that can automatically adjust hyperparameters and optimize data preprocessing. Its structure includes an encoder and a decoder, and uses skip connections for feature fusion; therefore, the model can automatically adapt to different medical image segmentation tasks without manually adjusting parameters.

[0069] Furthermore, one of the training methods for the model is provided below, as shown in the first to fifth steps below.

[0070] Step 1: Obtain an initial training dataset, wherein the initial training dataset includes several initial lung CT sample images. In this embodiment, lung CT images of several historical users can be collected in advance to serve as the initial lung CT sample images. At the same time, because medical imaging data comes from different scanners and may have different resolutions, this embodiment also provides a resampling process to ensure the consistency of all image data in spatial scale, thereby facilitating subsequent processing and analysis. The resampling process is shown in the second step below.

[0071] Step 2: Resample each initial lung CT sample image in the initial training data set to obtain a number of resampled lung CT sample images; in a specific implementation, for example, but not limited to, first obtaining the voxel spacing of each initial lung CT sample image; then, counting the number of occurrences of each voxel spacing; then, screening out the voxel spacing with the largest number of occurrences, and taking the voxel spacing with the largest number of occurrences as the mode spacing; finally, resampling all the initial lung CT sample images based on the mode spacing to obtain a number of resampled lung CT sample images.

[0072] The above resampling operation not only ensures the consistency of all image data in spatial scale, but also eliminates noise and redundant information in the image, which helps to extract more effective features. At the same time, through resampling, the model can learn the features of images of different resolutions, improve the generalization ability of the model, and make it better adapt to different data sets.

[0073] After completing the resampling process of the initial lung CT sample image, data normalization is required, and the process is shown in the third step below.

[0074] Step 3: Perform intensity normalization processing on the plurality of resampled lung CT sample images to obtain a plurality of normalized lung CT sample images. In this embodiment, for example, but not limited to, Z-score normalization processing can be performed on each resampled lung CT sample image. The normalization processing formula is: Among them, I * is the normalized lung CT sample image, I is the resampled lung CT sample image, μ is the mean of the resampled lung CT sample image, and σ is the standard deviation of the resampled lung CT sample image.

[0075] After completing the intensity normalization of the image, in order to ensure the diversity of the data, this embodiment also performs data enhancement processing, the process of which is shown in the fourth step below.

[0076] Step 4: Perform data augmentation processing on the normalized lung CT sample images to obtain a number of lung CT sample images, and use the number of lung CT sample images to form a training data set. In specific implementation, for example, but not limited to, performing operations such as flipping, rotating, elastic deformation, gamma correction, Gaussian noise, and / or random cropping on the normalized lung CT sample images. In this way, after the aforementioned operations, the data set can be expanded, thereby improving the robustness and generalization ability of the model, so that it can better adapt to different scenarios and data distributions. At the same time, due to the enhanced diversity of the training data, overfitting of the model can also be avoided.

[0077] After completing the data enhancement, several lung CT sample images can be obtained to form a training data set; then, the training data set can be used to train the model, and the process is shown in the fifth step below.

[0078] Step 5: Using each lung CT sample image in the training data set as input and the CT lung lobe segmentation image corresponding to each lung CT sample image as output, the neural network model is trained to obtain the lung lobe segmentation model after the training is completed; in this embodiment, the aforementioned neural network model is the nnUNet model; at the same time, for example, during training, Adam can be used for optimization but is not limited to it, and a dynamic learning rate adjustment strategy is adopted to gradually reduce the learning rate in the later stage of training to prevent model overfitting.

[0079] Furthermore, the loss function of the lung lobe segmentation model can be, but is not limited to:

[0080]

[0081] In formula (3), L represents the loss function, p i represents the segmentation category of the i-th pixel in each lung CT sample image, g irepresents the segmentation label value of the i-th pixel in each lung CT sample image, and the segmentation category is 1 or 0. In this embodiment, a corresponding label image is set for each lung CT sample image, wherein the label image is annotated with the label value of each pixel. Specifically, if the label value is 1, it means that the pixel is a pixel in the lung lobe area, otherwise, it is a pixel in the remaining area (which can be regarded as the background), and i = 1, 2, ..., N, where N represents the total number of samples.

[0082] In this way, through the aforementioned combined loss function, both category imbalance and boundary optimization can be taken into account.

[0083] Based on the first to fifth steps above, after completing the training of the nnUNet model, a lung lobe segmentation model can be obtained; thus, in actual use, it is only necessary to input the target person's lung CT image into the model to obtain a CT lung lobe segmentation image.

[0084] After the anatomical structure information of the target person's lungs is acquired, it needs to be converted to MRI space so that it can be combined with the lung function information to generate a lung ventilation map of the target person. The image conversion process is shown in step S3 below.

[0085] S3. Based on the lung CT image and the lung MRI image, convert the CT lung lobe segmentation image into the MRI space to obtain the MRI lung lobe segmentation image; in specific applications, for example, but not limited to, the following steps S31 to S33 can be used to convert the CT lung lobe segmentation image into the MRI space.

[0086] S31. Perform global rigid registration on the lung CT image and the target image to obtain a registered lung CT image, wherein the target image is the MRI UTE inspiratory phase image or the MRI UTE expiratory phase image; in a specific application, for example, but not limited to, mutual information (MI) can be used as a similarity measurement indicator to perform global rigid registration. registration) to align the lung CT image and the target image; in this embodiment, mutual information measures the statistical correlation between the intensity distributions of the two images, and its calculation formula is: MI(A,B)=H(A)+H(B)-H(A,B), where H(A) and H(B) are the entropies of individual images (i.e., the entropies of the lung CT image and the target image), reflecting the randomness of their respective grayscale distributions; and H(A,B) is the joint entropy, reflecting the uncertainty of the joint grayscale distribution; at the same time, since the mutual information reaches its maximum value when the joint entropy is minimum and the image alignment is optimal, this embodiment can achieve registration between multimodal images, that is, registration between the lung CT image and the target image, by optimizing the mutual information value and maximizing it.

[0087] After completing the global rigid registration of the lung CT image and the target image, an affine transformation may be performed to obtain a coordinate transformation matrix between the lung CT image and the target image; wherein the affine transformation process is shown in the following step S32.

[0088] S32. Perform affine registration on the registered lung CT image and the target image to obtain a coordinate transformation matrix between the lung CT image and the target image after the affine registration. In specific applications, after completing the initial rigid alignment, affine registration can be further used to achieve more flexible deformation alignment to capture geometric transformations including rotation, scaling, translation, and shearing. The affine transformation can be expressed as: * =Ax+t; where x is the pixel coordinate in the original image, x * are the new coordinates after transformation, A is a 3*3 transformation matrix used to represent rotation, scaling and shearing, and t is the translation vector; therefore, the entire affine registration process has 7 degrees of freedom (3 rotations, 3 scaling / shearing, and 1 translation), which can align anatomical structures and further improve the registration accuracy.

[0089] After completing the affine transformation, the coordinate transformation matrix between the lung CT image and the target image is obtained; then, based on this coordinate transformation matrix, the CT lung lobe segmentation image can be projected into the MRI space to obtain the MRI lung lobe segmentation image; wherein, the projection process is shown in the following step S33.

[0090] S33. Perform coordinate transformation processing on the CT lung lobe segmentation image according to the coordinate transformation matrix, so as to obtain the MRI lung lobe segmentation image after the coordinate transformation processing.

[0091] In this way, through the aforementioned steps S31 to S33, the projection of the CT image to the MRI space can be completed, thereby projecting the CT lung lobe segmentation image into the MRI space; based on this, an accurate data foundation can be provided for the subsequent generation of ventilation maps.

[0092] After the anatomical structure is acquired, its functional information needs to be acquired. The functional information refers to the respiratory deformation field of the target person, that is, the pixel position transformation information generated by each pixel point in the lung image due to breathing. In this embodiment, the respiratory deformation field of the target person is generated through a lung MRI image, and the process is shown in the following step S4.

[0093] S4. Determine a respiratory deformation field based on the MRI UTE inspiratory phase image and the MRI UTE expiratory phase image, wherein the respiratory deformation field is used to characterize pixel position transformation information between the MRI UTE inspiratory phase image and the MRI UTE expiratory phase image. In a specific implementation, for example, but not limited to, non-rigid registration processing can be performed on the MRI UTE inspiratory phase image and the MRI UTE expiratory phase image to obtain a pixel transformation matrix between the MRI UTE inspiratory phase image and the MRI UTE expiratory phase image after the non-rigid registration processing, and the pixel transformation matrix is ​​used as the respiratory deformation field.

[0094] Among them, each element in the pixel transformation matrix corresponds to a pixel point in the MRI UTE inspiratory phase image, each element corresponds to a three-dimensional matrix, and the three-dimensional matrix is ​​used to represent the displacement information of the corresponding pixel point in three-dimensional space; thus, the pixel transformation matrix represents the transformation information of each position in the lungs of the target person when breathing in three directions, that is, the aforementioned pixel position transformation information; based on this, the functional information of the lungs can be obtained.

[0095] Furthermore, for example, but not limited to, B-Spline free-shape transformation can be used for local deformation, that is, B-Spline free-shape transformation can be used to perform non-rigid registration processing of MRI UTE inspiratory phase images and MRI UTE expiratory phase images; wherein, this method allows the image to be flexibly deformed in the local area by defining a regular control point grid in the image domain to more accurately capture the subtle geometric differences between tissues; specifically, B-Spline FFD models the spatial transformation as a continuous deformation field guided by control points, and its three-dimensional transformation function can be expressed as: φ(x) = ∑ i,j,k β i (x)β j (y)β k (z)P ijk , where β(x) is the B-spline basis function, β i (x), β j (y) and β k (z) are cubic B-Spline basis functions in three directions, which are used to ensure the smoothness and locality of the transformation field; P ijk is the deformation control point vector located on the control point grid, indicating the displacement magnitude and direction of the corresponding node, and i, j, k respectively represent the grid node coordinates; at the same time, for example, but not limited to, the squared difference can be used as a similarity measure to perform non-rigid registration of the two images.

[0096] Thus, through the aforementioned B-Spline free shape transformation, the respiratory deformation field generated by the target person when breathing can be obtained; of course, B-Spline free shape transformation is a commonly used technology for non-rigid registration, and its detailed registration process will not be repeated here.

[0097] After obtaining the respiratory deformation field of the target person, the aforementioned anatomical structure information, that is, the MRI lung lobe segmentation image, can be combined to generate a lung ventilation map of the target person. The generation process can be, but is not limited to, as shown in the following step S5.

[0098] S5. Generate a lung ventilation map of the target person using the MRI lung lobe segmentation image and the respiratory deformation field, and visualize the lung ventilation map. In this embodiment, for example, but not limited to, first calculate the local lung ventilation matrix of the target person based on the respiratory deformation field; then, generate a lung ventilation map of the target person based on the local lung ventilation matrix and the MRI lung lobe segmentation image.

[0099] In this embodiment, it has been explained above that the respiratory deformation field is a pixel transformation matrix between the MRI UTE inspiratory phase image and the MRI UTE expiratory phase image, wherein each element in the pixel transformation matrix corresponds to a pixel point in the MRI UTE inspiratory phase image, each element corresponds to a three-dimensional matrix, and the three-dimensional matrix is ​​used to characterize the displacement information of the corresponding pixel point in the three-dimensional space; based on this, the respiratory deformation field can describe the change in voxel position caused by respiratory movement, and thus, the local ventilation can be calculated by the Jacobian determinant, that is, for example, but not limited to, the following formula (1) can be used to calculate the local ventilation matrix of the lungs of the target person.

[0100]

[0101] In the above formula (1), J(x) represents the target person's lung local ventilation matrix, T represents the respiratory deformation field, and x represents an element in the respiratory deformation field. represents the Jacobian matrix of the breathing deformation field, and det() represents the determinant calculation function.

[0102] After calculating the target person's lung local ventilation matrix using the above formula (1), the target person's lung ventilation map can be generated by combining it with the aforementioned MRI lung lobe segmentation image.

[0103] For example, but not limited to, the lung ventilation diagram of the target person can be generated according to the following formula (2).

[0104] V=mask*(J(x)-1) (2)

[0105] In the above formula (2), V represents the lung ventilation map of the target person, mask represents the MRI lung lobe segmentation image, and J(x) represents the local lung ventilation matrix.

[0106] In this way, through the above formulas (1) and (2), a lung ventilation map of the target person can be generated; and then, it can be visualized to provide auxiliary images for medical staff to evaluate the lung ventilation function.

[0107] Therefore, through the lung ventilation analysis method based on CT and MRI described in detail in the aforementioned steps S1 to S5, the present invention realizes the comprehensive acquisition of anatomical structure and functional information by fusing CT and MRI images, that is, mapping the precise anatomical structure information in CT to MRI space to obtain MRI lung lobe segmentation images; then, using MRI UTE inspiratory phase images and MRI UTE expiratory phase images, accurate functional information is obtained, that is, the respiratory deformation field of the target person's lungs is determined; finally, using MRI lung lobe segmentation images and respiratory deformation fields, a lung ventilation map of the target person can be generated; thus, the present invention solves the problem of insufficient single image modality information in traditional technology, improves the accuracy of ventilation map generation, and at the same time, solves the limitation of existing MRI technology that additional exogenous contrast agents are required to generate ventilation maps, thereby improving the flexibility of use; based on this, the present invention provides a method for generating lung ventilation maps that is flexible and accurate, and is therefore very suitable for large-scale application and promotion.

[0108] like Figure 2 As shown, the second aspect of this embodiment provides a hardware system for implementing the lung ventilation analysis method based on CT and MRI described in the first aspect of the embodiment, including an image acquisition device and an image processing device, wherein the image acquisition device may include, but is not limited to, a CT device and a nuclear magnetic resonance device, and the working process of the system is as follows:

[0109] An image acquisition device is used to acquire lung image data of a target person, wherein the lung image data includes a lung CT image and a lung MRI image, and the lung MRI image includes an MRI UTE inspiratory phase image and an MRI UTE expiratory phase image.

[0110] The image processing device is used to perform image segmentation processing on the lung CT image to obtain a CT lung lobe segmentation image of the target person.

[0111] An image processing device is used to convert the CT lung lobe segmentation image into MRI space based on the lung CT image and the lung MRI image to obtain an MRI lung lobe segmentation image.

[0112] An image processing device is used to determine a respiratory deformation field based on an MRI UTE inspiratory phase image and an MRI UTE expiratory phase image, wherein the respiratory deformation field is used to represent pixel position transformation information between the MRI UTE inspiratory phase image and the MRI UTE expiratory phase image.

[0113] The image processing device is also used to generate a lung ventilation map of the target person using the MRI lung lobe segmentation image and the respiratory deformation field, and to visualize the lung ventilation map.

[0114] The working process, working details and technical effects of the system provided in this embodiment can be found in the first aspect of the embodiment and will not be described in detail here.

[0115] like Figure 3 As shown, the third aspect of this embodiment provides a lung ventilation analysis device based on CT and MRI. Taking the device as an electronic device as an example, it includes: a memory, a processor and a transceiver that are communicatively connected in sequence, wherein the memory is used to store computer programs, the transceiver is used to send and receive messages, and the processor is used to read the computer program and execute the lung ventilation analysis method based on CT and MRI as described in the first aspect of the embodiment.

[0116] For example, the memory may include, but is not limited to, random access memory (RAM), read-only memory (ROM), flash memory, first-in first-out memory (FIFO), and / or first-in last-out memory (FILO); specifically, the processor may include one or more processing cores, such as a quad-core processor, an octa-core processor, etc. The processor may be implemented in at least one hardware form selected from DSP (Digital Signal Processing), FPGA (Field-Programmable Gate Array), and PLA (Programmable Logic Array). Furthermore, the processor may include a main processor and a coprocessor. The main processor is a processor for processing data in an awake state, also known as a CPU (Central Processing Unit); and the coprocessor is a low-power processor for processing data in a standby state.

[0117] In some embodiments, the processor may be integrated with a GPU (Graphics Processing Unit), which is responsible for rendering and drawing the content to be displayed on the display screen. For example, the processor may be, but is not limited to, a microprocessor of the STM32F105 series, a reduced instruction set computer (RISC) microprocessor, an X86 architecture processor, or a processor with an integrated embedded neural network processing unit (NPU); the transceiver may be, but is not limited to, a wireless fidelity (WIFI) wireless transceiver, a Bluetooth wireless transceiver, a general packet radio service technology (GPRS) wireless transceiver, a ZigBee protocol (a low-power local area network protocol based on the IEEE802.15.4 standard, ZigBee) wireless transceiver, a 3G transceiver, a 4G transceiver, and / or a 5G transceiver. In addition, the device may also include, but is not limited to, a power module, a display screen, and other necessary components.

[0118] The working process, working details and technical effects of the electronic device provided in this embodiment can be found in the first aspect of the embodiment and will not be described in detail here.

[0119] The fourth aspect of this embodiment provides a storage medium storing instructions for the CT and MRI-based lung ventilation analysis method described in the first aspect of the embodiment, that is, the storage medium stores instructions, and when the instructions are run on a computer, the CT and MRI-based lung ventilation analysis method described in the first aspect of the embodiment is executed.

[0120] The storage medium refers to a carrier for storing data, which may include but is not limited to a floppy disk, an optical disk, a hard disk, a flash memory, a USB flash drive and / or a memory stick, and the computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device.

[0121] The working process, working details and technical effects of the storage medium provided in this embodiment can be found in the first aspect of the embodiment and will not be described in detail here.

[0122] A fifth aspect of this embodiment provides a computer program product comprising instructions, which, when executed on a computer, causes the computer to execute the CT and MRI-based lung ventilation analysis method as described in the first aspect of the embodiment, wherein the computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device.

[0123] Finally, it should be noted that the above description is only a preferred embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention shall be included in the scope of protection of the present invention.

Claims

1. A lung ventilation analysis method based on CT and MRI, characterized in that: include: Acquiring lung image data of a target person, wherein the lung image data includes a lung CT image and a lung MRI image, and the lung MRI image includes an MRI UTE inspiratory phase image and an MRI UTE expiratory phase image; performing image segmentation processing on the lung CT image to obtain a CT lung lobe segmentation image of the target person; Based on the lung CT image and the lung MRI image, converting the CT lung lobe segmentation image into MRI space to obtain an MRI lung lobe segmentation image; Determining a respiratory deformation field based on the MRI UTE inspiratory phase image and the MRI UTE expiratory phase image, wherein the respiratory deformation field is used to represent pixel position transformation information between the MRI UTE inspiratory phase image and the MRI UTE expiratory phase image; A lung ventilation map of the target person is generated using the MRI lung lobe segmentation image and the respiratory deformation field, and the lung ventilation map is visualized.

2. The method according to claim 1, characterized in that Based on the lung CT image and the lung MRI image, converting the CT lung lobe segmentation image into MRI space to obtain an MRI lung lobe segmentation image, comprising: performing global rigid registration on the lung CT image and a target image to obtain a registered lung CT image, wherein the target image is the MRI UTE inspiratory phase image or the MRI UTE expiratory phase image; performing affine registration processing on the registered lung CT image and the target image to obtain a coordinate transformation matrix between the lung CT image and the target image after the affine registration processing; The CT lung lobe segmentation image is subjected to coordinate transformation processing according to the coordinate transformation matrix, so as to obtain the MRI lung lobe segmentation image after the coordinate transformation processing.

3. The method according to claim 1, characterized in that Based on the MRI UTE inspiratory phase image and the MRI UTE expiratory phase image, the respiratory deformation field is determined, including: performing non-rigid registration processing on the MRI UTE inspiratory phase image and the MRI UTE expiratory phase image, so as to obtain a pixel transformation matrix between the MRI UTE inspiratory phase image and the MRI UTE expiratory phase image after the non-rigid registration processing, and using the pixel transformation matrix as the respiratory deformation field; Each element in the pixel transformation matrix corresponds to a pixel point in the MRI UTE inspiratory phase image, each element corresponds to a three-dimensional matrix, and the three-dimensional matrix is ​​used to represent the displacement information of the corresponding pixel point in three-dimensional space.

4. The method according to claim 1, wherein Generating a lung ventilation map of the target person using the MRI lung lobe segmentation image and the respiratory deformation field, including: Calculating a local lung ventilation matrix of the target person based on the respiratory deformation field; A lung ventilation map of the target person is generated according to the lung local ventilation matrix and the MRI lung lobe segmentation image.

5. The method according to claim 4, characterized in that The respiratory deformation field is a pixel transformation matrix between the MRI UTE inspiratory phase image and the MRI UTE expiratory phase image, wherein each element in the pixel transformation matrix corresponds to a pixel point in the MRI UTE inspiratory phase image, each element corresponds to a three-dimensional matrix, and the three-dimensional matrix is ​​used to represent the displacement information of the corresponding pixel point in three-dimensional space; The method of calculating the target person's lung local ventilation matrix based on the respiratory deformation field includes: According to the following formula (1), the local lung ventilation matrix of the target person is calculated; In the above formula (1), J(x) represents the target person's lung local ventilation matrix, T represents the respiratory deformation field, and x represents an element in the respiratory deformation field. represents the Jacobian matrix of the breathing deformation field, and det() represents the determinant calculation function.

6. The method according to claim 4, characterized in that Generating a lung ventilation map of a target person according to the lung local ventilation matrix and the MRI lung lobe segmentation image, including: Generate the lung ventilation map of the target person according to the following formula (2); V=mask*(J(x)-1) (2) In the above formula (2), V represents the lung ventilation map of the target person, mask represents the MRI lung lobe segmentation image, and J(x) represents the local lung ventilation matrix.

7. The method according to claim 1, characterized in that Performing image segmentation processing on the lung CT image to obtain a CT lung lobe segmentation image of the target person includes: Acquire a lung lobe segmentation model, wherein the lung lobe segmentation model is trained by taking a plurality of lung CT sample images as input and outputting lung lobe segmentation images corresponding to each lung CT sample image; The lung CT image is input into the lung lobe segmentation model to obtain a CT lung lobe segmentation image of the target person.

8. The method according to claim 7, characterized in that The lung lobe segmentation model is trained in the following way: Acquire an initial training data set, wherein the initial training data set includes a plurality of initial lung CT sample images; resampling each initial lung CT sample image in the initial training data set to obtain a plurality of resampled lung CT sample images; performing intensity normalization processing on a number of resampled lung CT sample images to obtain a number of normalized lung CT sample images; Performing data enhancement processing on a number of normalized lung CT sample images to obtain a number of lung CT sample images, and using the number of lung CT sample images to form a training data set; The neural network model is trained using each lung CT sample image in the training data set as input and the CT lung lobe segmentation image corresponding to each lung CT sample image as output, so as to obtain the lung lobe segmentation model after the training is completed.

9. The method according to claim 7, characterized in that The loss function of the lung lobe segmentation model is: In formula (3), L represents the loss function, p i represents the segmentation category of the i-th pixel in each lung CT sample image, g i Represents the segmentation label value of the i-th pixel in each lung CT sample image, and the segmentation category is 1 or 0.

10. A pulmonary ventilation analysis system based on CT and MRI, characterized in that: include: An image acquisition device is used to acquire lung image data of a target person, wherein the lung image data includes a lung CT image and a lung MRI image, and the lung MRI image includes an MRI UTE inspiratory phase image and an MRI UTE expiratory phase image; an image processing device, configured to perform image segmentation processing on the lung CT image to obtain a CT lung lobe segmentation image of the target person; An image processing device is used to convert the CT lung lobe segmentation image into MRI space based on the lung CT image and the lung MRI image to obtain an MRI lung lobe segmentation image; An image processing device is configured to determine a respiratory deformation field based on the MRI UTE inspiratory phase image and the MRI UTE expiratory phase image, wherein the respiratory deformation field is used to represent pixel position transformation information between the MRI UTE inspiratory phase image and the MRI UTE expiratory phase image; The image processing device is also used to generate a lung ventilation map of the target person using the MRI lung lobe segmentation image and the respiratory deformation field, and to visualize the lung ventilation map.

Images