Method and system for identifying multifunctional lung regions based on image registration and segmentation

Abstract

The present invention relates to the technical field of medical image processing, and in particular to a method and system for identifying a multifunctional lung region based on image registration and segmentation; the method comprises: performing lung segmentation and annotation on a lung CT / MRI image, using a spine segmentation model to perform spine detection on the lung CT / MRI image with lung segment segmentation and annotation, obtaining a spine sequence label, intercepting an image of a corresponding layer as a lung region image with lung segment segmentation and annotation, generating a standard coordinate system Template, and registering the lung region image to the coordinate system template to obtain a registered image training set, forming a lung multifunctional region segmentation model, segmenting the image, and reversely registering the segmentation result to the original coordinate system to obtain a final segmentation result; by detecting the spine and performing registration, model training and recognition only on the lung region image, the accuracy of the lung multifunctional region recognition can be effectively improved.

Description

Technical Field

[0001] The present invention relates to the technical field of medical image processing, and in particular to a method and system for identifying multifunctional lung regions based on image registration and segmentation. Background Art

[0002] Lung segmentation can divide the lungs into different functional areas, which plays an important role in helping to identify the pulmonary artery and its branches during surgery. As a complex organ, the lungs have different blood supply and anatomical characteristics in different areas. With the help of lung segmentation, these functional areas can be identified and distinguished more accurately.

[0003] In the anatomical structure of the pulmonary artery, the pulmonary artery is the main source of blood supply. Pulmonary segmentation divides the lungs into different functional areas based on the distribution pattern of the pulmonary artery, each of which has its own unique blood supply and anatomical characteristics. For surgical operations, accurate identification of the pulmonary artery and its branches is crucial. The location and distribution of the pulmonary artery directly affect surgical planning and execution. Through pulmonary segmentation, a detailed analysis of the lung structure can be performed before surgery to clarify the direction and branches of the pulmonary artery, providing surgeons with precise intraoperative navigation to avoid accidental damage to the pulmonary artery. In addition, pulmonary segmentation can also provide anatomical information of the lung functional areas, helping doctors assess the extent and severity of lesions and develop more personalized treatment strategies.

[0004] From a computer vision perspective, lung segmentation faces many challenges. The lung structure is complex, and the lung morphology and anatomical features of different individuals may vary significantly, especially in lung size, shape, and boundaries, which makes the development of segmentation algorithms more difficult. In addition, changes in image quality, such as noise, artifacts, or motion blur, can also affect the accuracy of segmentation. At the same time, the visibility of the pulmonary artery and its complex branching structure in the image is often limited, which increases the difficulty of the algorithm to extract the pulmonary artery from the image.

[0005] Recently, relevant research institutions have developed lung segmentation models, but due to limitations of imaging equipment and individual differences, the versatility of the models is generally low and difficult to be widely used in clinical practice.

[0006] Image registration is a technique used to align and match different images, and plays a vital role in improving the robustness of image segmentation models. In the image segmentation process, it is necessary to accurately extract the target area from the image. However, due to various factors such as image acquisition conditions, shooting angles, or image deformation, there may be differences or distortions between different images, which brings challenges to the segmentation task.

[0007] The core goal of image registration is to eliminate these differences and distortions so that images can be accurately aligned geometrically. Specifically, image registration is applied to multiple images in the same scene but with different shooting times, angles or devices. Given a fixed image and a moving image, the task of image registration is to find a spatial transformation method to match the image to be registered with the fixed image, so as to achieve one-to-one correspondence of spatial points and complete information fusion. By transforming and aligning the images, the registration technology can keep the feature points or structures of the images consistent in space. This technology is crucial in optimizing image segmentation models. By accurately registering the input images, the inconsistency caused by image differences can be eliminated, thereby improving the comparability and consistency of image data, making the established image segmentation model more accurate and robust. In addition, image registration also provides more information support for the image segmentation process. For example, the superposition of multiple registered images can provide richer feature information and clearer target boundaries. This additional information helps the model better understand the image content, thereby improving the accuracy and stability of the segmentation results.

[0008] However, there are still great challenges in achieving robust image segmentation in the field of computer vision. In the image registration process, the coordinate system of the image space is actually processed, so if the shape, size or appearance of the target changes significantly, the registration result may be inaccurate. For example, when a large-size image is registered to a small-size coordinate system, image information may be lost; conversely, when a small-size image is registered to a large-size coordinate system, the image will be stretched. Ultimately, the registered image may be significantly different from the original image and cannot maintain its original medical or biological information. This requires the registration algorithm to have strong flexibility and adaptability to cope with changes in various scenarios and targets. Summary of the invention

[0009] In view of the above-mentioned deficiencies of the prior art, the present invention aims to provide a method and system for identifying multifunctional lung zones based on image registration and segmentation, so as to solve the problem of inaccurate identification of multifunctional lung zones caused by image registration errors caused by coordinate system transformation in the existing method for segmenting multifunctional lung zones.

[0010] In order to solve the above problems, the present invention adopts the following technical solutions:

[0011] In one aspect, the present invention provides a method for identifying multifunctional lung regions based on image registration and segmentation, comprising:

[0012] Obtain lung CT / MRI images;

[0013] Performing lung segmentation and annotation on the lung CT / MRI image;

[0014] The spine segmentation model is used to detect the spine in lung CT / MRI images with lung segment segmentation annotations to obtain spine sequence labels;

[0015] According to the spinal column sequence markers, an image of a corresponding layer is intercepted as a lung region image with lung segmentation annotations;

[0016] Selecting a plurality of representative images from the lung region images with lung segmentation annotations to generate a standard coordinate system Template, and registering all the lung region images with lung segmentation annotations to the standard coordinate system Template to obtain a registered image training set;

[0017] Using the registered image training set to train and verify the lung segmentation model to form a lung multifunctional area segmentation model;

[0018] The lung multifunctional area segmentation model is used to segment the lung CT / MRI image to be segmented that is converted into the standard coordinate system Template, and the segmentation result is reversely aligned to the original coordinate system to obtain the final segmentation result.

[0019] As an implementable method, the spine segmentation model uses 3D-UNet as a basic framework, a downsampling residual network composed of 5 layers of residual modules and an upsampling residual network composed of 5 layers of residual modules, and the residual module of the upsampling residual network is jump-connected to the residual module of the downsampling residual network through an attention residual module;

[0020] Each residual module consists of two consecutive convolution modules of normalization function and ReLU function;

[0021] First, the 3D-UNet is trained on the downsampled low-resolution data, and then on the full-resolution data. After training with a 50% discount validation, the model configuration is evaluated by calculating the average Dice. The trained low-resolution 3D-UNet model and the full-resolution 3D-UNet model are combined to obtain the spine segmentation model.

[0022] As an implementable method, data enhancement is performed on lung CT / MRI images before training the lung segmentation model, and the data enhancement includes: spatial enhancement, rotation and scaling, adding Gaussian noise, Gaussian blur, brightness and contrast enhancement, low-resolution enhancement, gamma enhancement, mirror change, binary operator setting and / or connected component removal.

[0023] As an implementable method, the step of selecting a plurality of representative images from the lung region images with lung segmentation annotations to generate a standard coordinate system Template includes:

[0024] The average coordinate system is formed by the average coordinates of the multiple representative images, each representative image is registered to the average coordinate system, the deformed images therein are counted and their deformation matrices are calculated, and the standard coordinate system Template is obtained by the following formula:

[0025] Standard coordinate system Template = coordinate system formed by the average coordinates of all deformed images * average deformation matrix^-1.

[0026] As an implementable embodiment, the plurality of representative images are images having similar features or the same anatomical structure or the same acquisition subject.

[0027] On the other hand, the present invention provides a lung multifunctional region recognition system based on image registration and segmentation, comprising: an acquisition module, a labeling module, a spine sequence labeling module, a lung region image acquisition module, a training set construction module, a lung multifunctional region segmentation model construction module and a segmentation module;

[0028] The acquisition module is used to acquire lung CT / MRI images;

[0029] The labeling module is used to perform lung segmentation labeling on the lung CT / MRI image;

[0030] The spine sequence labeling module is used to perform spine detection on the lung CT / MRI image with lung segmentation annotations using a spine segmentation model to obtain a spine sequence labeling;

[0031] The lung region image acquisition module is used to intercept the image of the corresponding layer according to the spinal sequence mark as the lung region image with lung segmentation annotation;

[0032] The training set construction module is used to select multiple representative images from the lung region images with lung segmentation annotations to generate a standard coordinate system Template, and register all the lung region images with lung segmentation annotations to the standard coordinate system Template to obtain a registered image training set;

[0033] The lung multifunctional area segmentation model construction module is used to train and verify the lung segmentation model using the registered image training set to form a lung multifunctional area segmentation model;

[0034] The segmentation module is used to segment the lung CT / MRI image to be segmented that is converted into the standard coordinate system Template using the lung multifunctional area segmentation model, and reversely align the segmentation result to the original coordinate system to obtain the final segmentation result.

[0035] As an implementable method, the spine segmentation model uses 3D-UNet as a basic framework, a downsampling residual network composed of 5 layers of residual modules and an upsampling residual network composed of 5 layers of residual modules, and the residual module of the upsampling residual network is jump-connected to the residual module of the downsampling residual network through an attention residual module;

[0036] Each residual module consists of two consecutive convolution modules of normalization function and ReLU function;

[0037] First, the 3D-UNet is trained on the downsampled low-resolution data, and then on the full-resolution data. After training with a 50% discount validation, the model configuration is evaluated by calculating the average Dice. The trained low-resolution 3D-UNet model and the full-resolution 3D-UNet model are combined to obtain the spine segmentation model.

[0038] As an implementable method, data enhancement is performed on lung CT / MRI images before training the lung segmentation model, and the data enhancement includes: spatial enhancement, rotation and scaling, adding Gaussian noise, Gaussian blur, brightness and contrast enhancement, low-resolution enhancement, gamma enhancement, mirror change, binary operator setting and / or connected component removal.

[0039] As an implementable method, the step of selecting a plurality of representative images from the lung region images with lung segmentation annotations to generate a standard coordinate system Template includes:

[0040] The average coordinate system is formed by the average coordinates of the multiple representative images, each representative image is registered to the average coordinate system, the deformed images therein are counted and their deformation matrices are calculated, and the standard coordinate system Template is obtained by the following formula:

[0041] Standard coordinate system Template = coordinate system formed by the average coordinates of all deformed images * average deformation matrix^-1.

[0042] As an implementable embodiment, the plurality of representative images are images having similar features or the same anatomical structure or the same acquisition subject.

[0043] The beneficial effects of the present invention are as follows: the present invention can effectively improve the versatility and accuracy of the model by detecting the spine and performing registration, model training and recognition only on images of the lung area; it solves the problem that surgeons have difficulty in performing preoperative anatomical analysis and risk assessment for lung-related surgeries and accurately determining the direction and branches of the pulmonary artery during surgery to avoid accidental damage to the pulmonary artery; the present invention solves the problem of poor versatility of computer vision models caused by large individual differences in imaging instruments and subjects. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] Figure 1 The present invention is a flowchart of a method for identifying multifunctional lung areas based on image registration and segmentation in an embodiment of the present invention.

[0045] Figure 2 Schematic diagram of a system for identifying multifunctional lung areas based on image registration and segmentation in an embodiment of the present invention. DETAILED DESCRIPTION

[0046] The present invention is further described in detail below in conjunction with specific embodiments.

[0047] It should be noted that these embodiments are only used to illustrate the present invention, rather than to limit the present invention. Simple improvements to the method based on the concept of the present invention all fall within the scope of protection claimed by the present invention.

[0048] Spine detection technology is mainly used to identify spinal structures in CT or MRI images, which is of great significance for the positioning of the lung area. In these images, the spine, as a significant anatomical structure, has unique morphological characteristics. Through spine detection, the position and morphological information of the spine can be automatically obtained, thereby helping to accurately locate the lungs. The lungs are located in front of the spine. As a stable reference point, the spine can provide an accurate basis for determining the relative position of the lungs. By detecting the position and morphological information of the spine, auxiliary support can be provided for lung positioning. As a fixed reference object, the spine helps to accurately determine the positional relationship of the lungs. In addition, spine detection can also help eliminate interference from other structures to the lungs. In CT or MRI images, there may be similar structures such as ribs. By accurately identifying the spinal structure, misjudgment can be effectively avoided and accurate positioning of the lungs can be ensured.

[0049] See also Figure 1 , which is a method for identifying multifunctional lung regions based on image registration and segmentation, comprising:

[0050] S100: Acquire a lung CT / MRI image.

[0051] Prepare 100-200 lung CT / MRI slice scan images of subjects to ensure the diversity of subject data, including different instrument models, subject age, gender, and different scanning parameters (such as layer thickness, spacing, resolution, etc.) to reduce the deviation caused by individual differences and instrument differences and ensure the generalization of the model.

[0052] The imaging data must cover both normal people and patients with lung diseases to ensure that the trained segmentation model is not only applicable to healthy lungs, but also works accurately in diseased lungs. The data should be anonymized in advance to ensure patient privacy.

[0053] S200: performing lung segmentation and labeling on the lung CT / MRI image.

[0054] The labeling personnel are the hospital's radiologists. The doctors need to accurately mark the lung segment boundaries according to the international standard lung segment anatomy diagram to ensure that the anatomical characteristics of each lung lobe are accurately reflected.

[0055] S300, using a spine segmentation model to perform spine detection on a lung CT / MRI image with lung segmentation annotations to obtain a spine sequence label.

[0056] Before spine sequence labeling, the images can be preprocessed, including signal normalization and resampling.

[0057] Signal Normalization: Voxel intensities in CT scans are associated with quantitative physical properties of tissues. Therefore, intensities should be as consistent as possible between scanners for the same batch of data. This consistency is exploited by using a global intensity normalization scheme with a Z-score distribution. The mean, standard deviation, and 0.5 and 99.5 percentiles of these values ​​are then calculated. All images are subsequently normalized by clipping them between the 0.5 and 99.5 percentiles, subtracting the global mean, and dividing by the global standard deviation.

[0058] Resampling: The target spacing was determined to be 2.06 × 1.94 × 1.94 mm, corresponding to the median voxel spacing. The resampling strategy was determined on a per-image basis. Third order spline interpolation was used to resample image data and linear interpolation was used to resample segmentations for equidistant cases (maximum interaxial spacing / minimum interaxial spacing < 3). Segmentations were always converted to a one hot representation before resampling and then converted back to segmentations after interpolation.

[0059] After resampling: the median image size is 212×199×199. The pipeline prioritizes larger patch sizes over larger batch sizes to capture as much contextual information as possible. Therefore, the 3D U-Net is configured with a patch size of 128×128×128 voxels and a batch size of 2. Since the input patches have almost equidistant spacing, all convolution kernel sizes and downsampling strides are equidistant (3×3×3 and 2×2×2, respectively).

[0060] The spine segmentation model uses 3D-UNet as the basic framework, a downsampling residual network composed of 5 layers of residual modules and an upsampling residual network composed of 5 layers of residual modules, and the residual module of the upsampling residual network is jump-connected to the residual module of the downsampling residual network through the attention residual module.

[0061] Each residual module consists of two consecutive convolution modules with normalization function and ReLU function.

[0062] Kernel sizes: The kernel sizes for this architecture are [[3, 3, 3], [3, 3, 3], [3, 3, 3], [3, 3,3], [3, 3, 3], [3, 3, 3]]. The list contains 6 elements, matching the 6 resolutions encountered in the encoder. Each element in the list gives the kernel size of the convolutional layer at that resolution (this is a three-digit number since convolutions are three-dimensional). Within a resolution, two blocks use the same kernel size. The convolutions in the decoder are symmetric to those in the encoder. Downsampling stride: The downsampling stride is [[1, 1, 1], [2, 2, 2], [2, 2, 2], [2, 2, 2], [2, 2, 2], [2, 2,2]]. Each downsampling step in the encoder is represented by a single entry. A stride of 2 results in a downsampling factor of 2 along that axis, while a stride of 1 keeps the dimensions unchanged. The downsampling stride only applies to the first convolution at each resolution stage in the encoder. The stride of the second convolution is always [1, 1, 1]. The decoder again mirrors the encoder, but the stride is used as the output stride of the convolutional transpose (thus achieving appropriate upscaling of the feature maps). The output shape of all convolutional transposes has the same shape as the skip connections originating from the encoder.

[0063] First, the 3D-UNet is trained on the downsampled low-resolution data, and then on the full-resolution data. After training with a 50% discount validation, the model configuration is evaluated by calculating the average Dice. The trained low-resolution 3D-UNet model and the full-resolution 3D-UNet model are combined to obtain the spine segmentation model.

[0064] The target spacing for the low-resolution U-Net was determined to be 0.80×0.75×0.75 mm, resulting in a median image shape of 545×512×512 voxels. 3D low-resolution was operated on using 128×128×128 patches with a batch size of 2.

[0065] According to this evaluation scheme, the score of 3D full-resolution U-Net is 0.8044, the score of 3D low-resolution U-Net is 0.7796, and the score of cascade full-resolution 3D U-Net is 0.8017. The best combination of two models was determined to be the combination between low-resolution U-Net and full-resolution U-Net, with a score of 0.8111.

[0066] Data augmentation was performed on lung CT / MRI images during training, including spatial augmentation, rotation and scaling, adding Gaussian noise, Gaussian blur, brightness and contrast enhancement, low-resolution enhancement, gamma enhancement, mirror change, binary operator setting, and / or connected component removal.

[0067] Spatial enhancement: training and post-processing During training, spatial enhancements such as scaling and rotation are performed only within the plane to prevent resampling of imaging information between slices, which could cause interpolation artifacts.

[0068] Rotation and Scaling: Reduces the number of data interpolations required to one. Scaling and rotation are each applied with probability 0.2 (0.16 for scaling only, 0.16 for rotation only, and 0.08 for both scaling and rotation). If dealing with an isotropic 3D block, the rotation angles (in degrees) αx, αy, and αz are each randomly chosen from U(-30, 30). If the block is anisotropic or 2D, the rotation angles are sampled from U(-180, 180). If the 2D block dimensions are anisotropic, the angles are sampled from U(-15, 15). Scaling is done by multiplying the coordinates in the voxel grid by the scaling factor. Thus, scaling factors less than 1 produce a "zooming out" effect, while values ​​greater than 1 produce a "zooming in" effect. For all block types, the scaling factor is sampled from U(0.7, 1.4).

[0069] Add Gaussian Noise: Add additive Gaussian noise centered at zero independently. This augmentation is applied with a probability of 0.15. The variance of the noise is randomly selected from U(0, 0.1) (due to intensity normalization, the voxel intensity in all samples is close to zero mean and unit variance).

[0070] Gaussian Blur: For each sample, blur is applied with probability 0.2. If this enhancement is triggered in a sample, blur is applied with probability 0.5 for each relevant modality (so for samples with only a single modality, the combined probability is only 0.1). The width of the Gaussian kernel (in voxels) is sampled from U(0.5, 1.5) for each modality independently.

[0071] Brightness and contrast enhancement: For brightness, with a probability of 0.15, the voxel intensity will be multiplied by x∼U(0.7, 1.3) for enhancement. This means that the voxel intensity in each sample will be scaled by a random factor between 0.7 and 1.3. This scaling can change the overall brightness of the image and increase the diversity of the data;

[0072] Contrast, with probability 0.15, the voxel intensity will be multiplied by x∼U(0.65, 1.5) for enhancement. This means that the voxel intensity in each sample will be scaled by a random factor between 0.65 and 1.5. After the multiplication, the values ​​are clipped to their original value range to ensure that they do not exceed the original range. This enhancement can change the contrast and brightness of the image, introducing a certain degree of variation and diversity.

[0073] Low-resolution enhancement: This enhancement is applied per sample with probability 0.25 and per relevant modality with probability 0.5. The triggered modality is downsampled by a factor of U(1, 2) via nearest neighbor interpolation and then resampled to the original size using cubic spline interpolation. For 2D blocks or anisotropic 3D blocks, this enhancement is applied only in 2D, preserving the original state along the out-of-plane axis (if applicable).

[0074] Gamma augmentation: This augmentation is applied with probability 0.15. Voxel intensities are first scaled to a factor of [0, 1] according to their respective value range. Then, a nonlinear intensity transformation is applied to each voxel: inew = iγold, where γ∼U(0.7, 1.5) is a gamma factor randomly selected from a uniform distribution. Subsequently, the voxel intensities are scaled back to their original value range. With probability 0.15, this augmentation inverts the voxel intensity before the transformation: (1 - inew) = (1 - iold)γ. This augmentation can change the contrast and brightness of an image or voxels and introduce more variation and diversity.

[0075] Mirroring: All patches have a probability of 0.5 to be mirrored along all axes. This means that each patch has a 50% chance of being mirrored on each axis. Mirroring can change the orientation of an image or voxel, increasing the diversity and invariance of the data.

[0076] Binary operator settings: With probability 0.4, a binary operator is applied to all labels in the prediction mask. The operator is randomly selected from [dilation, erosion, opening, closing]. The structure element is a sphere with radius r∼U(1, 8). The operators are applied to the labels in a random order. In this process, the one-hot encoding property is preserved. For example, applying a dilation operation to one label will result in removing all other labels within the dilated region. This augmentation technique can change the shape and distribution of labels in the prediction mask, thereby increasing the diversity of the data. By using different binary operators and structure element parameters, different morphological operations can be simulated, allowing the model to better handle different shapes and structures.

[0077] Connected component removal: With probability 0.2, connected components smaller than 15% of the block size are removed from the one-hot encoding.

[0078] Based on the obtained spine segmentation model, the (combined) configuration to be used for inference is automatically determined based on the average foreground Dice coefficient on the training data. The selected model can be a single U-Net (2D, 3D full resolution, 3D low resolution, or cascaded full resolution U-Net) or an ensemble of any two configurations. Models are integrated by averaging Softmax probabilities.

[0079] Post-processing: Post-processing is configured for this set of segmentation maps. Removing all but the largest connected component is beneficial for both the merged foreground region and for only the lung label, as both methods lead to small performance gains when empirically tested on the training data. In organ segmentation, it is often possible to remove spurious false positive detections by removing all but the largest connected component. This model follows this assumption and automatically evaluates the impact of suppressing smaller components on the cross-validation results.

[0080] First, all foreground categories are considered as one component. If the suppression of all regions except the largest region improves the average foreground Dice coefficient and does not reduce the Dice coefficient of any category, then this process will be selected as the first post-processing step. Finally, nnU-Net is based on the results of this step and decides whether to perform the same process for each category. This means that for each category, it is considered whether only the largest connected component should be retained to further improve the segmentation result.

[0081] S400 , according to the spinal column sequence mark, intercepting an image of a corresponding layer as a lung region image with lung segmentation annotations.

[0082] S500, selecting multiple representative images from the lung region images with lung segmentation annotations to generate a standard coordinate system Template, and registering all the lung region images with lung segmentation annotations to the standard coordinate system Template to obtain a registered image training set.

[0083] Generate a standard coordinate system Template, including:

[0084] The average coordinate system is formed by the average coordinates of multiple representative images. Each representative image is registered to the average coordinate system, and the deformed images are counted and their deformation matrices are calculated. The standard coordinate system Template is obtained by the following formula:

[0085] Standard coordinate system Template = coordinate system formed by the average coordinates of all deformed images * average deformation matrix^-1.

[0086] The multiple representative images are images with similar features or the same anatomical structure or the same acquisition subject.

[0087] Before registration, the selected images are preprocessed, such as intensity normalization, noise reduction, or image enhancement, to improve the quality and consistency of the images.

[0088] Align or register the selected images to a common coordinate space. This step involves finding a spatial transformation that brings the images into spatial correspondence. Different registration algorithms can be used, such as intensity-based registration, feature-based registration, or deformable registration.

[0089] S600: Use the registered image training set to train and verify the lung segmentation model to form a lung multifunctional area segmentation model.

[0090] S700, using the lung multifunctional area segmentation model to segment the lung CT / MRI image to be segmented that is converted into the standard coordinate system Template, and reversely aligning the segmentation result to the original coordinate system to obtain the final segmentation result.

[0091] The present invention can realize automatic recognition and extraction of lung region images. The extracted lung region images are registered and unified into a standard spatial coordinate system, and then model training or recognition is performed. This method can effectively reduce individual differences between images, improve the generalization ability of the model, and ensure high applicability and accuracy among different subjects.

[0092] See also Figure 2, which is a lung multifunctional area recognition system based on image registration and segmentation, comprising: an acquisition module 100, a labeling module 200, a spine sequence labeling module 300, a lung area image acquisition module 400, a training set construction module 500, a lung multifunctional area segmentation model construction module 600 and a segmentation module 700;

[0093] The acquisition module 100 is used to acquire lung CT / MRI images;

[0094] The labeling module 200 is used to label lung segments of lung CT / MRI images;

[0095] The spine sequence labeling module 300 is used to perform spine detection on the lung CT / MRI image with lung segmentation annotations using a spine segmentation model to obtain a spine sequence label;

[0096] The lung region image acquisition module 400 is used to intercept the image of the corresponding layer according to the spinal column sequence mark as the lung region image with lung segmentation annotation;

[0097] The training set construction module 500 is used to select multiple representative images from the lung region images with lung segmentation annotations to generate a standard coordinate system Template, and align all the lung region images with lung segmentation annotations to the standard coordinate system Template to obtain the aligned image training set;

[0098] The lung multifunctional area segmentation model construction module 600 is used to train and verify the lung segmentation model using the registered image training set to form a lung multifunctional area segmentation model;

[0099] The segmentation module 700 is used to segment the lung CT / MRI image to be segmented that is converted into the standard coordinate system Template using the lung multifunctional area segmentation model, and reversely align the segmentation result to the original coordinate system to obtain the final segmentation result.

[0100] The spine segmentation model uses 3D-UNet as the basic framework, a downsampling residual network consisting of 5 layers of residual modules and an upsampling residual network consisting of 5 layers of residual modules, and the residual module of the upsampling residual network is jump-connected to the residual module of the downsampling residual network through the attention residual module;

[0101] Each residual module consists of two consecutive convolution modules of normalization function and ReLU function;

[0102] First, the 3D-UNet is trained on the downsampled low-resolution data, and then on the full-resolution data. After training with a 50% discount validation, the model configuration is evaluated by calculating the average Dice. The trained low-resolution 3D-UNet model and the full-resolution 3D-UNet model are combined to obtain the spine segmentation model.

[0103] Among them, before training the lung segmentation model, the lung CT / MRI image is subjected to data enhancement, and the data enhancement includes: spatial enhancement, rotation and scaling, adding Gaussian noise, Gaussian blur, brightness and contrast enhancement, low-resolution enhancement, gamma enhancement, mirror change, binary operator setting and / or connected component removal.

[0104] Among them, multiple representative images are selected from the lung area images with lung segmentation annotations to generate a standard coordinate system Template, including:

[0105] The average coordinate system is formed by the average coordinates of multiple representative images. Each representative image is registered to the average coordinate system, and the deformed images are counted and their deformation matrices are calculated. The standard coordinate system Template is obtained by the following formula:

[0106] Standard coordinate system Template = coordinate system formed by the average coordinates of all deformed images * average deformation matrix^-1.

[0107] The multiple representative images are images with similar features or the same anatomical structure or the same acquisition subject.

[0108] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit it. Although the present invention has been described with reference to the preferred embodiments of the present invention, it should be understood by those skilled in the art that various changes may be made in form and details without departing from the spirit and scope of the present invention as defined in the appended claims.

Claims

1. A method for identifying multifunctional lung regions based on image registration and segmentation, characterized in that: include: Obtain lung CT / MRI images; Performing lung segmentation and annotation on the lung CT / MRI image; The spine segmentation model is used to detect the spine in lung CT / MRI images with lung segment segmentation annotations to obtain spine sequence labels; According to the spinal column sequence markers, an image of a corresponding layer is intercepted as a lung region image with lung segmentation annotations; Selecting a plurality of representative images from the lung region images with lung segmentation annotations to generate a standard coordinate system Template, and registering all the lung region images with lung segmentation annotations to the standard coordinate system Template to obtain a registered image training set; Using the registered image training set to train and verify the lung segmentation model to form a lung multifunctional area segmentation model; The lung multifunctional area segmentation model is used to segment the lung CT / MRI image to be segmented that is converted into the standard coordinate system Template, and the segmentation result is reversely aligned to the original coordinate system to obtain the final segmentation result.

2. The method for identifying multifunctional lung regions based on image registration and segmentation according to claim 1, characterized in that: The spine segmentation model uses 3D-UNet as the basic framework, a downsampling residual network composed of 5 layers of residual modules and an upsampling residual network composed of 5 layers of residual modules, and the residual module of the upsampling residual network is jump-connected to the residual module of the downsampling residual network through an attention residual module; Each residual module consists of two consecutive convolution modules of normalization function and ReLU function; First, the 3D-UNet was trained using low-resolution data after downsampling, and then the 3D-UNet was trained using full-resolution data. After training with five-fold cross-validation, the configuration of the model was evaluated by calculating the average Dice. The trained low-resolution 3D-UNet model and the full-resolution 3D-UNet model were combined to obtain the spine segmentation model.

3. The method for identifying multifunctional lung regions based on image registration and segmentation according to claim 2, characterized in that: Data enhancement is performed on lung CT / MRI images before training the lung segmentation model, and the data enhancement includes: spatial enhancement, rotation and scaling, adding Gaussian noise, Gaussian blur, brightness and contrast enhancement, low-resolution enhancement, gamma enhancement, mirror change, binary operator setting and / or connected component removal.

4. The method for identifying multifunctional lung regions based on image registration and segmentation according to claim 1, characterized in that: The step of selecting a plurality of representative images from the lung region images with lung segmentation annotations to generate a standard coordinate system Template comprises: The average coordinate system is formed by the average coordinates of the multiple representative images, each representative image is registered to the average coordinate system, the deformed images therein are counted and their deformation matrices are calculated, and the standard coordinate system Template is obtained by the following formula: Standard coordinate system Template = coordinate system formed by the average coordinates of all deformed images * average deformation matrix^-1.

5. The method for identifying multifunctional lung regions based on image registration and segmentation according to claim 1, characterized in that: The multiple representative images are images with similar features or the same anatomical structure or the same acquisition subject.

6. A lung multifunctional area recognition system based on image registration and segmentation, characterized in that: include: Acquisition module, annotation module, spine sequence labeling module, lung region image acquisition module, training set construction module, lung multifunctional region segmentation model construction module and segmentation module; The acquisition module is used to acquire lung CT / MRI images; The labeling module is used to perform lung segmentation labeling on the lung CT / MRI image; The spine sequence labeling module is used to perform spine detection on the lung CT / MRI image with lung segmentation annotations using a spine segmentation model to obtain a spine sequence labeling; The lung region image acquisition module is used to intercept the image of the corresponding layer according to the spinal sequence mark as the lung region image with lung segmentation annotation; The training set construction module is used to select multiple representative images from the lung region images with lung segmentation annotations to generate a standard coordinate system Template, and register all the lung region images with lung segmentation annotations to the standard coordinate system Template to obtain a registered image training set; The lung multifunctional area segmentation model construction module is used to train and verify the lung segmentation model using the registered image training set to form a lung multifunctional area segmentation model; The segmentation module is used to segment the lung CT / MRI image to be segmented that is converted into the standard coordinate system Template using the lung multifunctional area segmentation model, and reversely align the segmentation result to the original coordinate system to obtain the final segmentation result.

7. The system for identifying multifunctional lung regions based on image registration and segmentation according to claim 6, characterized in that: The spine segmentation model uses 3D-UNet as the basic framework, a downsampling residual network composed of 5 layers of residual modules and an upsampling residual network composed of 5 layers of residual modules, and the residual module of the upsampling residual network is jump-connected to the residual module of the downsampling residual network through an attention residual module; Each residual module consists of two consecutive convolution modules of normalization function and ReLU function; First, the 3D-UNet was trained using low-resolution data after downsampling, and then the 3D-UNet was trained using full-resolution data. After training with five-fold cross-validation, the configuration of the model was evaluated by calculating the average Dice. The trained low-resolution 3D-UNet model and the full-resolution 3D-UNet model were combined to obtain the spine segmentation model.

8. The system for identifying multifunctional lung regions based on image registration and segmentation according to claim 7, characterized in that: Data enhancement is performed on lung CT / MRI images before training the lung segmentation model, and the data enhancement includes: spatial enhancement, rotation and scaling, adding Gaussian noise, Gaussian blur, brightness and contrast enhancement, low-resolution enhancement, gamma enhancement, mirror change, binary operator setting and / or connected component removal.

9. The system for identifying multifunctional lung regions based on image registration and segmentation according to claim 6, characterized in that: The step of selecting a plurality of representative images from the lung region images with lung segmentation annotations to generate a standard coordinate system Template includes: forming an average coordinate system using the average coordinates of the plurality of representative images, registering each representative image to the average coordinate system, counting the deformed images therein and calculating the deformation matrix thereof, and obtaining the standard coordinate system Template by the following formula: Standard coordinate system Template = coordinate system formed by the average coordinates of all deformed images * average deformation matrix^-1.

10. The system for identifying multifunctional lung regions based on image registration and segmentation according to claim 6, characterized in that: The multiple representative images are images with similar features or the same anatomical structure or the same acquisition subject.

Images