Lung lobe and lung multi-organ segmentation and result presentation method and related products

Abstract

The present disclosure provides a method for segmenting and presenting lung lobes and multiple lung organs and related products. The lung lobe segmentation method provided by the embodiment of the present disclosure assists in understanding the lung anatomical structure information through the original lung three-dimensional image and the tracheal connectivity map generated based on the lung three-dimensional image during the lung lobe segmentation process, thereby increasing the accuracy of lung lobe segmentation. In addition, the lung multi-organ segmentation and result presentation method provided by the embodiment of the present disclosure helps users understand the lung multi-organ segmentation results intuitively and vividly by synchronously presenting the three-dimensional objects of the lung nodule segmentation results, tracheal segmentation results, and lung lobe segmentation results, thereby reducing the difficulty of understanding.

Description

Technical Field

[0001] The embodiments of the present disclosure relate to the field of image recognition technology, and specifically to a method for segmenting lung lobes and multiple lung organs and presenting the results, as well as related products. Background Art

[0002] Currently, in the process of automatic lung lobe segmentation based on lung 3D images, most of them are only based on the lung 3D images themselves, and the accuracy of lung lobe segmentation needs to be improved.

[0003] In addition, the segmentation results of automatic lung nodule segmentation based on three-dimensional lung images are often relatively professional, which makes it difficult for ordinary users to understand the lung nodule segmentation results. Summary of the Invention

[0004] The embodiments of the present disclosure provide a method, apparatus, electronic device, storage medium, and computer program product for segmenting pulmonary lobes and multiple lung organs and presenting the results.

[0005] In a first aspect, an embodiment of the present disclosure provides a lung lobe segmentation method, the method comprising:

[0006] Obtain a three-dimensional image of the lung to be segmented;

[0007] The following four-channel image generation steps are performed using the to-be-segmented three-dimensional lung image as the input three-dimensional lung image, and the obtained four-channel three-dimensional lung image is used as the to-be-segmented four-channel three-dimensional lung image: based on the lung-trachea segmentation model, the input three-dimensional lung image is subjected to lung-trachea segmentation to obtain a trachea segmentation result, wherein the trachea segmentation result includes trachea voxels and non-trachea voxels; based on the trachea segmentation result, a trachea skeleton segmentation result is generated, wherein the trachea skeleton segmentation result includes trachea skeleton voxels and non-trachea skeleton voxels; based on the trachea skeleton segmentation result, a trachea connectivity map is generated, wherein the trachea connectivity map includes a three-dimensional tangent vector from each trachea skeleton voxel along the trachea skeleton to the trachea root skeleton voxel fastest; the trachea connectivity map is merged with the input three-dimensional lung image to obtain a four-channel three-dimensional lung image;

[0008] Based on the lung lobe feature extraction model, feature extraction is performed on the four-channel three-dimensional lung image to be segmented to obtain a three-dimensional lung lobe feature map to be segmented, wherein each voxel in the three-dimensional lung lobe feature map to be segmented has a corresponding lung lobe feature and corresponds one-to-one to a voxel in the three-dimensional lung image to be segmented;

[0009] Calculate the similarity between the lung lobe feature corresponding to each voxel in the three-dimensional lung lobe feature image to be segmented and the prototype feature corresponding to each preset lung lobe category in the preset lung lobe category set;

[0010] For each voxel in the three-dimensional lung lobe feature map to be segmented, determining the maximum similarity lung lobe category as the lung lobe category corresponding to the voxel, wherein the maximum similarity lung lobe category is the preset lung lobe category with the greatest similarity between the corresponding prototype feature in the preset lung lobe category set and the lung lobe feature corresponding to the voxel;

[0011] A lung lobe segmentation result of the three-dimensional lung image to be segmented is generated based on the lung lobe category corresponding to each voxel in the three-dimensional lung lobe feature map to be segmented.

[0012] In some optional embodiments, calculating the similarity between the lung lobe feature corresponding to each voxel in the three-dimensional lung lobe feature map to be segmented and the prototype feature corresponding to each preset lung lobe category in the preset lung lobe category set includes:

[0013] Clustering the lung lobe features corresponding to all voxels in the three-dimensional lung lobe feature map to be segmented to obtain N cluster center lung lobe features and a cluster center lung lobe feature corresponding to each voxel, where N is the number of preset lung lobe categories in the preset lung lobe category set;

[0014] For each voxel in the three-dimensional lung lobe feature map to be segmented, the average feature of the lung lobe feature of the voxel and the lung lobe feature of the cluster center corresponding to the voxel is calculated, and the similarity between the average feature and the prototype feature corresponding to each preset lung lobe category in the preset lung lobe category set is determined as the similarity between the lung lobe feature corresponding to the voxel and the prototype feature corresponding to the corresponding preset lung lobe category.

[0015] In some optional embodiments, the lung lobe feature extraction model and the prototype features corresponding to each preset lung lobe category in the preset lung lobe category set are predetermined by the following training steps:

[0016] Obtain a first sample set and a second sample set, wherein the first sample includes a first lung 3D image and a corresponding labeled lung lobe segmentation result, and the second sample includes a second lung 3D image and a corresponding labeled lung lobe segmentation result;

[0017] Performing the four-channel image generation step using each first three-dimensional lung image and each second three-dimensional lung image as input three-dimensional lung images to obtain corresponding first four-channel three-dimensional lung images and second four-channel three-dimensional lung images;

[0018] Perform the following parameter adjustment operations until the preset parameter adjustment end conditions are met: based on the lung lobe feature extraction model, perform feature extraction on each first four-channel lung three-dimensional image and each second four-channel lung three-dimensional image respectively to obtain a first three-dimensional lung lobe feature map and a second three-dimensional lung lobe feature map; use each first four-channel lung three-dimensional image and the corresponding labeled lung lobe segmentation result as input support samples, use each second four-channel lung three-dimensional image and the corresponding labeled lung lobe segmentation result as input query samples, calculate the loss function of the input query sample relative to the input support sample, and use the obtained loss function value as the first loss function value; use each second four-channel lung three-dimensional image and the corresponding labeled lung lobe segmentation result as input support samples, use each first four-channel lung three-dimensional image and the corresponding labeled lung lobe segmentation result as input query samples, calculate the loss function of the input query sample relative to the input support sample, and use the obtained loss function value as the second loss function value; adjust the model parameters of the lung lobe feature extraction model based on the first loss function value and the second loss function value;

[0019] Determining a prototype four-channel three-dimensional lung image set based on each of the first four-channel three-dimensional lung images and / or each of the second four-channel three-dimensional lung images;

[0020] Based on the lung lobe feature extraction model, feature extraction is performed on each of the prototype four-channel lung three-dimensional images to obtain a set of prototype three-dimensional lung lobe feature maps;

[0021] For each preset lung lobe category in the preset lung lobe categories, the mean feature of the lung lobe features corresponding to each prototype voxel of the preset lung lobe category is calculated, and the mean feature is determined as the prototype feature of the preset lung lobe category, wherein the prototype voxel of the preset lung lobe category is the voxel whose corresponding labeled lung lobe segmentation result in each prototype three-dimensional lung lobe feature map is the preset lung lobe category.

[0022] In some optional implementations, calculating the loss function of the input query sample relative to the input support sample includes:

[0023] For each preset lung lobe category in the preset lung lobe categories, calculating the mean feature of the lung lobe features corresponding to each supporting voxel of the preset lung lobe category, and determining the mean feature as the supporting prototype feature of the preset lung lobe category, wherein the supporting voxels of the preset lung lobe category are the voxels whose corresponding labeled lung lobe segmentation results in each of the input supporting samples are the preset lung lobe category;

[0024] For each voxel in each of the input query samples, calculating the similarity between the lung lobe feature of the voxel and the supporting prototype feature of each of the preset lung lobe categories, and normalizing the calculated similarity to obtain a predicted probability that the voxel belongs to each of the preset lung lobe categories;

[0025] The loss function value is calculated based on the difference between the predicted probability that the voxels in each input query sample are each preset lung lobe category and the labeled lung lobe segmentation result corresponding to the corresponding voxel.

[0026] In some optional embodiments, generating a tracheal connectivity map based on the tracheal skeleton segmentation result includes:

[0027] Determining a trachea root skeleton voxel in each of the trachea skeleton voxels according to the lung trachea distribution direction corresponding to the trachea skeleton segmentation result;

[0028] For each of the tracheal skeleton voxels, a breadth-first search is used to determine the shortest path from the tracheal skeleton voxel to the tracheal root skeleton voxel, wherein two adjacent tracheal skeleton voxels in the shortest path are cubic neighbors of each other, and a three-dimensional vector of the strongest direction point voxel pointing from the tracheal skeleton voxel to the tracheal skeleton voxel is determined as the three-dimensional tangent vector of the tracheal skeleton voxel, wherein the strongest direction point voxel of the tracheal skeleton voxel is the tracheal skeleton voxel reached by advancing M voxels along the corresponding shortest path toward the tracheal root skeleton voxel;

[0029] The tracheal connectivity map is generated based on the three-dimensional tangent vector of each tracheal skeleton voxel.

[0030] In a second aspect, an embodiment of the present disclosure provides a method for presenting lung multi-organ segmentation results, the method comprising:

[0031] Obtaining a lung nodule segmentation result, a trachea segmentation result, and a lung lobe segmentation result of a three-dimensional lung image to be segmented, wherein the lung nodule segmentation result includes nodule voxels and non-nodule voxels, the trachea segmentation result includes tracheal voxels and non-tracheal voxels, and the lung lobe segmentation result is obtained by performing any lung lobe segmentation method described in the first aspect on the three-dimensional lung image to be segmented;

[0032] Converting the lung nodule segmentation result, the trachea segmentation result, and the lung lobe segmentation result into three-dimensional objects respectively to obtain a lung nodule three-dimensional object, a trachea three-dimensional object, and a lung lobe three-dimensional object;

[0033] Rendering the lung nodule three-dimensional object, the trachea three-dimensional object, and the lung lobe three-dimensional object to obtain a rendered lung nodule three-dimensional object, a rendered trachea three-dimensional object, and a rendered lung lobe three-dimensional object;

[0034] The rendered lung nodule three-dimensional object, the rendered trachea three-dimensional object, and the rendered lung lobe three-dimensional object are presented.

[0035] In some optional embodiments, the rendered lung nodule three-dimensional object, the rendered trachea three-dimensional object, and the rendered lung lobe three-dimensional object have different colors.

[0036] In some optional embodiments, the transparency of the rendered lung nodule three-dimensional object and the rendered trachea three-dimensional object is greater than the transparency of the rendered lung lobe three-dimensional object.

[0037] In some optional embodiments, the lung lobe segmentation result includes voxels corresponding to each lung lobe category in a preset lung lobe category set; and

[0038] The converting the lung lobe segmentation result into a lung lobe three-dimensional object comprises:

[0039] For each of the preset lung lobe categories, generating a lung lobe three-dimensional object corresponding to the preset lung lobe category based on the voxels corresponding to the preset lung lobe category in the lung lobe segmentation result; and

[0040] The rendering of the lung lobe three-dimensional object includes:

[0041] The three-dimensional lung lobe objects corresponding to the preset lung lobe categories are rendered according to different rendering colors.

[0042] In some optional embodiments, the method further comprises:

[0043] The following rotation and presentation steps are performed at preset time intervals: the lung nodule three-dimensional object, the trachea three-dimensional object and the lung lobe three-dimensional object are rotated by a preset angle; the lung nodule three-dimensional object, the trachea three-dimensional object and the lung lobe three-dimensional object are rendered to obtain a rendered lung nodule three-dimensional object, a rendered trachea three-dimensional object and a rendered lung lobe three-dimensional object; and the rendered lung nodule three-dimensional object, the rendered trachea three-dimensional object and the rendered lung lobe three-dimensional object are presented.

[0044] In some optional embodiments, presenting the rendered pulmonary nodule three-dimensional object includes:

[0045] If at least two of the rendered pulmonary nodule three-dimensional objects are included, different rendered pulmonary nodule three-dimensional objects are presented in different time periods.

[0046] In some optional embodiments, the pulmonary nodule segmentation result is obtained by the following pulmonary nodule segmentation steps:

[0047] Performing lung nodule segmentation on the three-dimensional lung image to be segmented based on the lung nodule segmentation model to obtain a preliminary lung nodule segmentation result, wherein the preliminary lung nodule segmentation result includes nodule voxels and non-nodule voxels;

[0048] Performing a connected domain analysis based on the preliminary pulmonary nodule segmentation result to determine at least one pulmonary nodule central voxel;

[0049] For each central voxel of a pulmonary nodule, taking the central voxel of the pulmonary nodule as the center, intercepting a voxel region of a preset three-dimensional size in the preliminary pulmonary nodule segmentation result to obtain a pulmonary nodule voxel block corresponding to the central voxel of the pulmonary nodule;

[0050] Based on the false positive identification model, each lung nodule voxel block is subjected to lung nodule false positive identification to determine whether the lung nodule voxel block is a positive lung nodule voxel block or a false positive lung nodule voxel block;

[0051] The lung nodule segmentation result is generated based on the positive lung nodule voxel blocks in each lung nodule voxel block.

[0052] In some optional embodiments, the pulmonary nodule segmentation step further includes:

[0053] For each positive lung nodule voxel block, the positive lung nodule voxel block is input into the lung nodule density prediction model and the lung nodule calcification score prediction model respectively to obtain the lung nodule density value and lung nodule calcification score of the positive lung nodule voxel block;

[0054] In response to determining that the pulmonary nodule calcification score is greater than a preset calcification score threshold, determining the nodule category of the positive pulmonary nodule voxel block as a calcified pulmonary nodule;

[0055] In response to determining that the pulmonary nodule calcification score is not greater than a preset calcification score threshold, a nodule category of the positive pulmonary nodule voxel block is determined according to the pulmonary nodule density value of the positive pulmonary nodule voxel block.

[0056] In some optional embodiments, determining the nodule category of the positive pulmonary nodule voxel block according to the pulmonary nodule density value of the positive pulmonary nodule voxel block includes:

[0057] In response to the lung nodule density value of the positive lung nodule voxel block being less than a preset low-density threshold, determining the nodule category of the positive lung nodule voxel block as a ground glass nodule;

[0058] In response to the lung nodule density value of the positive lung nodule voxel block being not less than a preset low density threshold and less than a preset high density threshold, determining the nodule category of the positive lung nodule voxel block as a semi-solid nodule, wherein the preset low density threshold is less than the preset high density threshold;

[0059] In response to the lung nodule density value of the positive lung nodule voxel block being not less than the preset high-density threshold, the nodule category of the positive lung nodule voxel block is determined to be a solid nodule.

[0060] In some optional embodiments, the method further comprises:

[0061] For each positive lung nodule voxel block, at least one of the following items of the positive lung nodule voxel block is presented: a lung nodule density value, a lung nodule calcification score, and a nodule category.

[0062] In some optional embodiments, the pulmonary nodule density prediction model includes a first three-dimensional convolutional neural network and a first fully connected network connected in sequence, and the pulmonary nodule calcification score prediction model includes a second three-dimensional convolutional neural network and a second fully connected network connected in sequence; and

[0063] The step of inputting the positive lung nodule voxel block into the lung nodule density prediction model and the lung nodule calcification score prediction model to obtain the lung nodule density value and the lung nodule calcification score corresponding to the positive lung nodule voxel block comprises:

[0064] Inputting the positive lung nodule voxel block into the first three-dimensional convolutional neural network to obtain a first voxel block feature map, and inputting the first voxel block feature map into the first fully connected network to obtain a lung nodule density value corresponding to the positive lung nodule voxel block;

[0065] Inputting the positive pulmonary nodule voxel block into the second three-dimensional convolutional neural network to obtain a second voxel block feature map, and inputting the second voxel block feature map into the second fully connected network to obtain a pulmonary nodule calcification score corresponding to the positive pulmonary nodule voxel block;

[0066] The first voxel block feature map and the second voxel block feature map are input into a third fully connected network to obtain a risk level value of a malignant pulmonary nodule corresponding to the positive pulmonary nodule voxel block, and malignant pulmonary nodule risk level information of the positive pulmonary nodule voxel block is determined based on the obtained risk level value.

[0067] In some optional embodiments, the method further comprises:

[0068] For each positive lung nodule voxel block, the risk level information of the malignant lung nodule corresponding to the positive lung nodule voxel block is presented.

[0069] In a third aspect, an embodiment of the present disclosure provides a lung lobe segmentation device, the device comprising:

[0070] an image acquisition module, configured to acquire a three-dimensional image of the lung to be segmented;

[0071] The four-channel image generation module is configured to perform the following four-channel image generation steps using the three-dimensional lung image to be segmented as the input three-dimensional lung image, and use the obtained four-channel three-dimensional lung image as the four-channel three-dimensional lung image to be segmented: based on the lung-trachea segmentation model, perform lung-trachea segmentation on the input three-dimensional lung image to obtain a trachea segmentation result, wherein the trachea segmentation result includes trachea voxels and non-trachea voxels; generate a trachea skeleton segmentation result based on the trachea segmentation result, wherein the trachea skeleton segmentation result includes trachea skeleton voxels and non-trachea skeleton voxels; generate a trachea connectivity map based on the trachea skeleton segmentation result, wherein the trachea connectivity map includes a three-dimensional tangent vector from each trachea skeleton voxel along the trachea skeleton to the trachea root skeleton voxel fastest; merge the trachea connectivity map with the input three-dimensional lung image to obtain a four-channel three-dimensional lung image;

[0072] a 3D lung lobe feature map generation module configured to perform feature extraction on the four-channel 3D lung image to be segmented based on a lung lobe feature extraction model to obtain a 3D lung lobe feature map to be segmented, wherein each voxel in the 3D lung lobe feature map to be segmented has a corresponding lung lobe feature and corresponds one-to-one to a voxel in the 3D lung image to be segmented;

[0073] A similarity calculation module is configured to calculate the similarity between the lung lobe feature corresponding to each voxel in the three-dimensional lung lobe feature map to be segmented and the prototype feature corresponding to each preset lung lobe category in the preset lung lobe category set;

[0074] a lung lobe category determination module configured to, for each voxel in the three-dimensional lung lobe feature map to be segmented, determine the lung lobe category with the greatest similarity as the lung lobe category corresponding to the voxel, wherein the lung lobe category with the greatest similarity between the prototype feature corresponding to the preset lung lobe category set and the lung lobe feature corresponding to the voxel;

[0075] The lung lobe segmentation module is configured to generate a lung lobe segmentation result of the three-dimensional lung image to be segmented based on the lung lobe category corresponding to each voxel in the three-dimensional lung lobe feature map to be segmented.

[0076] In some optional implementations, the similarity calculation module is further configured to:

[0077] Clustering the lung lobe features corresponding to all voxels in the three-dimensional lung lobe feature map to be segmented to obtain N cluster center lung lobe features and a cluster center lung lobe feature corresponding to each voxel, where N is the number of preset lung lobe categories in the preset lung lobe category set;

[0078] For each voxel in the three-dimensional lung lobe feature map to be segmented, the average feature of the lung lobe feature of the voxel and the lung lobe feature of the cluster center corresponding to the voxel is calculated, and the similarity between the average feature and the prototype feature corresponding to each preset lung lobe category in the preset lung lobe category set is determined as the similarity between the lung lobe feature corresponding to the voxel and the prototype feature corresponding to the corresponding preset lung lobe category.

[0079] In some optional embodiments, the lung lobe feature extraction model and the prototype features corresponding to each preset lung lobe category in the preset lung lobe category set are predetermined by the following training steps:

[0080] Obtain a first sample set and a second sample set, wherein the first sample includes a first lung 3D image and a corresponding labeled lung lobe segmentation result, and the second sample includes a second lung 3D image and a corresponding labeled lung lobe segmentation result;

[0081] Performing the four-channel image generation step using each first three-dimensional lung image and each second three-dimensional lung image as input three-dimensional lung images to obtain corresponding first four-channel three-dimensional lung images and second four-channel three-dimensional lung images;

[0082] Perform the following parameter adjustment operations until the preset parameter adjustment end conditions are met: based on the lung lobe feature extraction model, perform feature extraction on each first four-channel lung three-dimensional image and each second four-channel lung three-dimensional image respectively to obtain a first three-dimensional lung lobe feature map and a second three-dimensional lung lobe feature map; use each first four-channel lung three-dimensional image and the corresponding labeled lung lobe segmentation result as input support samples, use each second four-channel lung three-dimensional image and the corresponding labeled lung lobe segmentation result as input query samples, calculate the loss function of the input query sample relative to the input support sample, and use the obtained loss function value as the first loss function value; use each second four-channel lung three-dimensional image and the corresponding labeled lung lobe segmentation result as input support samples, use each first four-channel lung three-dimensional image and the corresponding labeled lung lobe segmentation result as input query samples, calculate the loss function of the input query sample relative to the input support sample, and use the obtained loss function value as the second loss function value; adjust the model parameters of the lung lobe feature extraction model based on the first loss function value and the second loss function value;

[0083] Determining a prototype four-channel three-dimensional lung image set based on each of the first four-channel three-dimensional lung images and / or each of the second four-channel three-dimensional lung images;

[0084] Based on the lung lobe feature extraction model, feature extraction is performed on each of the prototype four-channel lung three-dimensional images to obtain a set of prototype three-dimensional lung lobe feature maps;

[0085] For each preset lung lobe category in the preset lung lobe categories, the mean feature of the lung lobe features corresponding to each prototype voxel of the preset lung lobe category is calculated, and the mean feature is determined as the prototype feature of the preset lung lobe category, wherein the prototype voxel of the preset lung lobe category is the voxel whose corresponding labeled lung lobe segmentation result in each prototype three-dimensional lung lobe feature map is the preset lung lobe category.

[0086] In some optional implementations, calculating the loss function of the input query sample relative to the input support sample includes:

[0087] For each preset lung lobe category in the preset lung lobe categories, calculating the mean feature of the lung lobe features corresponding to each supporting voxel of the preset lung lobe category, and determining the mean feature as the supporting prototype feature of the preset lung lobe category, wherein the supporting voxels of the preset lung lobe category are the voxels whose corresponding labeled lung lobe segmentation results in each of the input supporting samples are the preset lung lobe category;

[0088] For each voxel in each of the input query samples, calculating the similarity between the lung lobe feature of the voxel and the supporting prototype feature of each of the preset lung lobe categories, and normalizing the calculated similarity to obtain a predicted probability that the voxel belongs to each of the preset lung lobe categories;

[0089] The loss function value is calculated based on the difference between the predicted probability that the voxels in each input query sample are each preset lung lobe category and the labeled lung lobe segmentation result corresponding to the corresponding voxel.

[0090] In some optional embodiments, generating a tracheal connectivity map based on the tracheal skeleton segmentation result includes:

[0091] Determining a trachea root skeleton voxel in each of the trachea skeleton voxels according to the lung trachea distribution direction corresponding to the trachea skeleton segmentation result;

[0092] For each of the tracheal skeleton voxels, a breadth-first search is used to determine the shortest path from the tracheal skeleton voxel to the tracheal root skeleton voxel, wherein two adjacent tracheal skeleton voxels in the shortest path are cubic neighbors of each other, and a three-dimensional vector of the strongest direction point voxel pointing from the tracheal skeleton voxel to the tracheal skeleton voxel is determined as the three-dimensional tangent vector of the tracheal skeleton voxel, wherein the strongest direction point voxel of the tracheal skeleton voxel is the tracheal skeleton voxel reached by advancing M voxels along the corresponding shortest path toward the tracheal root skeleton voxel;

[0093] The tracheal connectivity map is generated based on the three-dimensional tangent vector of each tracheal skeleton voxel.

[0094] In a fourth aspect, an embodiment of the present disclosure provides a device for presenting lung multi-organ segmentation results, the device comprising:

[0095] a segmentation result acquisition module configured to acquire a lung nodule segmentation result, a trachea segmentation result, and a lung lobe segmentation result of the three-dimensional lung image to be segmented, wherein the lung nodule segmentation result includes nodule voxels and non-nodule voxels, the trachea segmentation result includes tracheal voxels and non-tracheal voxels, and the lung lobe segmentation result is obtained by performing the lung lobe segmentation method described in any implementation of the first aspect on the three-dimensional lung image to be segmented;

[0096] a conversion module configured to convert the lung nodule segmentation result, the trachea segmentation result, and the lung lobe segmentation result into three-dimensional objects, respectively, to obtain a lung nodule three-dimensional object, a trachea three-dimensional object, and a lung lobe three-dimensional object;

[0097] a rendering module configured to render the pulmonary nodule 3D object, the trachea 3D object, and the lung lobe 3D object to obtain a rendered pulmonary nodule 3D object, a rendered trachea 3D object, and a rendered lung lobe 3D object;

[0098] A rendering module is configured to render the rendered lung nodule three-dimensional object, the rendered trachea three-dimensional object, and the rendered lung lobe three-dimensional object.

[0099] In some optional embodiments, the rendered lung nodule three-dimensional object, the rendered trachea three-dimensional object, and the rendered lung lobe three-dimensional object have different colors.

[0100] In some optional embodiments, the transparency of the rendered lung nodule three-dimensional object and the rendered trachea three-dimensional object is greater than the transparency of the rendered lung lobe three-dimensional object.

[0101] In some optional embodiments, the lung lobe segmentation result includes voxels corresponding to each lung lobe category in a preset lung lobe category set; and

[0102] The conversion module is further configured to:

[0103] For each of the preset lung lobe categories, generating a lung lobe three-dimensional object corresponding to the preset lung lobe category based on the voxels corresponding to the preset lung lobe category in the lung lobe segmentation result; and

[0104] The rendering module is further configured to:

[0105] The three-dimensional lung lobe objects corresponding to the preset lung lobe categories are rendered according to different rendering colors.

[0106] In some optional embodiments, the lung multi-organ segmentation result presentation device further includes:

[0107] The rotation presentation module is configured to perform the following rotation and presentation steps every preset time period: rotating the lung nodule three-dimensional object, the trachea three-dimensional object and the lung lobe three-dimensional object by a preset angle; rendering the lung nodule three-dimensional object, the trachea three-dimensional object and the lung lobe three-dimensional object to obtain a rendered lung nodule three-dimensional object, a rendered trachea three-dimensional object and a rendered lung lobe three-dimensional object; and presenting the rendered lung nodule three-dimensional object, the rendered trachea three-dimensional object and the rendered lung lobe three-dimensional object.

[0108] In some optional implementations, the presentation module is further configured to:

[0109] If at least two of the rendered pulmonary nodule three-dimensional objects are included, different rendered pulmonary nodule three-dimensional objects are presented in different time periods.

[0110] In some optional embodiments, the pulmonary nodule segmentation result is obtained by the following pulmonary nodule segmentation steps:

[0111] Performing lung nodule segmentation on the three-dimensional lung image to be segmented based on the lung nodule segmentation model to obtain a preliminary lung nodule segmentation result, wherein the preliminary lung nodule segmentation result includes nodule voxels and non-nodule voxels;

[0112] Performing a connected domain analysis based on the preliminary pulmonary nodule segmentation result to determine at least one pulmonary nodule central voxel;

[0113] For each central voxel of a pulmonary nodule, taking the central voxel of the pulmonary nodule as the center, intercepting a voxel region of a preset three-dimensional size in the preliminary pulmonary nodule segmentation result to obtain a pulmonary nodule voxel block corresponding to the central voxel of the pulmonary nodule;

[0114] Based on the false positive identification model, each lung nodule voxel block is subjected to lung nodule false positive identification to determine whether the lung nodule voxel block is a positive lung nodule voxel block or a false positive lung nodule voxel block;

[0115] The lung nodule segmentation result is generated based on the positive lung nodule voxel blocks in each lung nodule voxel block.

[0116] In some optional embodiments, the pulmonary nodule segmentation step further includes:

[0117] For each positive lung nodule voxel block, the positive lung nodule voxel block is input into the lung nodule density prediction model and the lung nodule calcification score prediction model respectively to obtain the lung nodule density value and lung nodule calcification score of the positive lung nodule voxel block;

[0118] In response to determining that the pulmonary nodule calcification score is greater than a preset calcification score threshold, determining the nodule category of the positive pulmonary nodule voxel block as a calcified pulmonary nodule;

[0119] In response to determining that the pulmonary nodule calcification score is not greater than a preset calcification score threshold, a nodule category of the positive pulmonary nodule voxel block is determined according to the pulmonary nodule density value of the positive pulmonary nodule voxel block.

[0120] In some optional embodiments, determining the nodule category of the positive pulmonary nodule voxel block according to the pulmonary nodule density value of the positive pulmonary nodule voxel block includes:

[0121] In response to the lung nodule density value of the positive lung nodule voxel block being less than a preset low-density threshold, determining the nodule category of the positive lung nodule voxel block as a ground glass nodule;

[0122] In response to the lung nodule density value of the positive lung nodule voxel block being not less than a preset low density threshold and less than a preset high density threshold, determining the nodule category of the positive lung nodule voxel block as a semi-solid nodule, wherein the preset low density threshold is less than the preset high density threshold;

[0123] In response to the lung nodule density value of the positive lung nodule voxel block being not less than the preset high-density threshold, the nodule category of the positive lung nodule voxel block is determined to be a solid nodule.

[0124] In some optional embodiments, the device further comprises:

[0125] The positive nodule attribute information presentation module is configured to present, for each positive lung nodule voxel block, at least one of the following items of the positive lung nodule voxel block: a lung nodule density value, a lung nodule calcification score, and a nodule category.

[0126] In some optional embodiments, the pulmonary nodule density prediction model includes a first three-dimensional convolutional neural network and a first fully connected network connected in sequence, and the pulmonary nodule calcification score prediction model includes a second three-dimensional convolutional neural network and a second fully connected network connected in sequence; and

[0127] The step of inputting the positive lung nodule voxel block into the lung nodule density prediction model and the lung nodule calcification score prediction model to obtain the lung nodule density value and the lung nodule calcification score corresponding to the positive lung nodule voxel block comprises:

[0128] Inputting the positive lung nodule voxel block into the first three-dimensional convolutional neural network to obtain a first voxel block feature map, and inputting the first voxel block feature map into the first fully connected network to obtain a lung nodule density value corresponding to the positive lung nodule voxel block;

[0129] Inputting the positive pulmonary nodule voxel block into the second three-dimensional convolutional neural network to obtain a second voxel block feature map, and inputting the second voxel block feature map into the second fully connected network to obtain a pulmonary nodule calcification score corresponding to the positive pulmonary nodule voxel block;

[0130] The first voxel block feature map and the second voxel block feature map are input into a third fully connected network to obtain a risk level value of a malignant pulmonary nodule corresponding to the positive pulmonary nodule voxel block, and malignant pulmonary nodule risk level information of the positive pulmonary nodule voxel block is determined based on the obtained risk level value.

[0131] In some optional embodiments, the lung multi-organ segmentation result presentation device further includes:

[0132] The malignant risk information presentation module is configured to present, for each positive lung nodule voxel block, malignant lung nodule risk level information corresponding to the positive lung nodule voxel block.

[0133] In a fifth aspect, an embodiment of the present disclosure provides an electronic device comprising: one or more processors; a storage device on which one or more programs are stored, and when the one or more programs are executed by the one or more processors, the one or more processors implement the method described in any implementation manner in the first aspect and / or the second aspect.

[0134] In a sixth aspect, an embodiment of the present disclosure provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by one or more processors, implements the method described in any one of the implementation modes of the first aspect and / or the second aspect.

[0135] In a seventh aspect, an embodiment of the present disclosure provides a computer program product, including a computer program / instruction, which, when executed by a processor, implements the method described in any implementation manner in the first aspect and / or the second aspect.

[0136] In order to solve the low accuracy of current lung lobe segmentation methods, the embodiments of the present invention provide a lung lobe segmentation method, device, electronic device, storage medium and computer program product, which first perform lung-trachea segmentation on the three-dimensional lung image to be segmented to obtain a trachea segmentation result, and then generate a tracheal skeleton segmentation result based on the tracheal segmentation result; then generate a tracheal connectivity map based on the tracheal skeleton segmentation result; then merge the tracheal connectivity map with the three-dimensional lung image to be segmented to obtain a four-channel three-dimensional lung image to be segmented; then, perform feature extraction on the four-channel three-dimensional lung image to be segmented to obtain a three-dimensional lung lobe feature map to be segmented; then calculate the similarity between the lung lobe feature corresponding to each voxel in the three-dimensional lung lobe feature map to be segmented and the prototype feature corresponding to each preset lung lobe category in a preset lung lobe category set; then, for each voxel in the three-dimensional lung lobe feature map to be segmented, determine the preset lung lobe category with the greatest similarity between the prototype feature corresponding to the preset lung lobe category set and the lung lobe feature corresponding to the voxel as the lung lobe category corresponding to the voxel; finally, generate a lung lobe segmentation result for the three-dimensional lung image to be segmented based on the lung lobe category corresponding to each voxel in the three-dimensional lung lobe feature map. That is, in the process of lung lobe segmentation, the tracheal connectivity map is introduced to understand the anatomical structure information of the lung, thereby increasing the accuracy of lung lobe segmentation.

[0137] In addition, in order to solve the problem that the current lung nodule segmentation results are relatively professional and difficult to understand, the embodiments of the present disclosure provide a lung multi-organ segmentation and result presentation method, device, electronic device, storage medium and computer program product, which simultaneously presents three-dimensional objects of lung nodule segmentation results, trachea segmentation results and lung lobe segmentation results, helping users to understand the lung multi-organ segmentation results intuitively and vividly, thereby reducing the difficulty of understanding. BRIEF DESCRIPTION OF THE DRAWINGS

[0138] Other features, objects, and advantages of the present disclosure will become more apparent upon reading the detailed description of non-limiting embodiments made with reference to the following drawings. The drawings are for illustration purposes only and are not to be considered as limiting the present invention. In the drawings:

[0139] Figure 1 is an exemplary system architecture diagram in which an embodiment of the present disclosure may be applied;

[0140] Figure 2A is a flow chart of an embodiment of a lung lobe segmentation method according to the present disclosure;

[0141] Figure 2B is a decomposed flow chart of one embodiment of step 202 according to the present disclosure;

[0142] Figure 2C is a decomposed flow chart according to one embodiment of step 2023 of the present disclosure;

[0143] Figure 2D is a decomposed flow chart of one embodiment of step 204 according to the present disclosure;

[0144] Figure 3A is a flow chart of one embodiment of the training step 300 according to the present disclosure;

[0145] Figure 3B is a decomposed flow chart of one embodiment of the parameter adjustment operation in step 303 according to the present disclosure;

[0146] Figure 4 is a flowchart of an embodiment of a method for presenting lung multi-organ segmentation results according to the present disclosure;

[0147] Figure 5A is a flow chart of one embodiment of the lung nodule segmentation step 500 according to the present disclosure;

[0148] Figure 5B is a decomposed flow chart of one embodiment of step 508 according to the present disclosure;

[0149] Figure 5C is a decomposed flow chart of one embodiment of step 506 according to the present disclosure;

[0150] Figure 6 is a schematic structural diagram of an embodiment of a lung nodule density prediction model according to the present disclosure;

[0151] Figure 7 FIG. 8 is a schematic diagram showing a presentation result of lung multi-organ segmentation results according to the present disclosure;

[0152] Figure 8 is a schematic structural diagram of an embodiment of a lung lobe segmentation device according to the present disclosure;

[0153] Figure 9 1 is a schematic structural diagram of an embodiment of a device for presenting lung multi-organ segmentation results according to the present disclosure;

[0154] Figure 10 It is a schematic diagram of the structure of a computer system of an electronic device suitable for implementing the embodiments of the present disclosure. DETAILED DESCRIPTION

[0155] The present disclosure will be further described in detail below with reference to the accompanying drawings and examples. It should be understood that the specific embodiments described herein are intended only to illustrate the relevant invention and are not intended to limit the invention. It should also be noted that, for ease of description, only portions relevant to the relevant invention are shown in the accompanying drawings.

[0156] It should be noted that, in the absence of conflict, the embodiments and features of the embodiments in the present disclosure may be combined with each other. The present disclosure will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0157] Figure 1 An exemplary system architecture 100 is shown to which embodiments of the method, apparatus, electronic device, storage medium, and computer program product of the present disclosure can be applied for lung lobe segmentation and presentation of lung multi-organ segmentation results.

[0158] like Figure 1 As shown, system architecture 100 may include terminal devices 101, 102, 103, a network 104, and a server 105. Network 104 is a medium for providing communication links between terminal devices 101, 102, 103 and server 105. Network 104 may include various connection types, such as wired or wireless communication links or fiber optic cables.

[0159] Users can use terminal devices 101, 102, and 103 to interact with server 105 via network 104 to receive or send messages, etc. Various communication client applications can be installed on terminal devices 101, 102, and 103, such as lung lobe segmentation applications, lung multi-organ segmentation result presentation applications, short video social networking applications, audio and video conferencing applications, web browser applications, shopping applications, search applications, instant messaging tools, email clients, social platform software, etc.

[0160] Terminal devices 101, 102, 103 can be hardware or software. When terminal devices 101, 102, 103 are hardware, they can be various electronic devices with information input devices (for example, keyboard, mouse, touch screen, microphone, camera, etc.) and information output devices (for example, display screen, speaker, etc.), including but not limited to smart phones, tablet computers, e-book readers, MP3 players (Moving Picture Experts Group Audio Layer III, Moving Picture Experts Group Audio Layer 3), MP4 (Moving Picture Experts Group Audio Layer IV, Moving Picture Experts Group Audio Layer 4) players, laptop computers and desktop computers, etc. When terminal devices 101, 102, 103 are software, they can be installed in the terminal devices listed above. It can be implemented as multiple software or software modules (for example, used to provide lung lobe segmentation and / or lung multi-organ segmentation result presentation services), or it can be implemented as a single software or software module. No specific limitation is made here.

[0161] In some cases, the lung lobe segmentation method and / or the lung multi-organ segmentation result presentation method provided by the present disclosure can be executed by the terminal devices 101, 102, and 103. Accordingly, the lung lobe segmentation apparatus and / or the lung multi-organ segmentation result presentation apparatus can be provided in the terminal devices 101, 102, and 103. In this case, the system architecture 100 may also not include the server 105.

[0162] In some cases, the lung lobe segmentation method and / or lung multi-organ segmentation result presentation method provided by the present disclosure can be jointly executed by the terminal devices 101, 102, 103 and the server 105. For example, the step of "obtaining a three-dimensional image of the lung to be segmented" can be executed by the terminal devices 101, 102, 103, and the steps of "based on the lung lobe feature extraction model, extracting features of the four-channel three-dimensional lung image to be segmented to obtain a three-dimensional lung lobe feature map to be segmented" can be executed by the server 105. The present disclosure does not limit this. Accordingly, the lung lobe segmentation device and / or the lung multi-organ segmentation result presentation device can also be respectively set in the terminal devices 101, 102, 103 and the server 105.

[0163] In some cases, the lung lobe segmentation method and / or lung multi-organ segmentation result presentation method provided in the present disclosure can be executed by the server 105. Accordingly, the lung lobe segmentation device and / or lung multi-organ segmentation result presentation device can also be set in the server 105. In this case, the system architecture 100 may also not include the terminal devices 101, 102, and 103.

[0164] It should be noted that the server 105 can be hardware or software. When the server 105 is hardware, it can be implemented as a distributed server cluster consisting of multiple servers, or it can be implemented as a single server. When the server 105 is software, it can be implemented as multiple software or software modules (for example, to provide distributed services), or it can be implemented as a single software or software module. No specific limitations are given here.

[0165] It should be understood that Figure 1 The number of terminal devices, networks and servers in the embodiment is merely illustrative. Any number of terminal devices, networks and servers may be provided as required.

[0166] Continue to refer Figure 2A , which shows a process 200 of an embodiment of a lung lobe segmentation method according to the present disclosure, the lung lobe segmentation method includes the following steps:

[0167] Step 201: Acquire a three-dimensional image of the lung to be segmented.

[0168] In this embodiment, the 3D lung image to be segmented can be various 3D medical images obtained by photographing the human lung. As an example, the 3D lung image to be segmented can be a CT image.

[0169] In some optional embodiments, the three-dimensional image of the lungs to be segmented can be a three-dimensional image of the original three-dimensional medical image after the direction is unified, wherein the first direction corresponding to the line connecting the apex and the base of the lungs in the three-dimensional image of the lungs to be segmented, the second direction corresponding to the line connecting the front chest and the back, and the third direction corresponding to the line connecting the left lung and the right lung are perpendicular to each other, and the first direction, the second direction, and the third direction correspond to the three axes in the image of the lungs to be segmented one by one and are parallel. For example, in the three-dimensional image of the lungs to be segmented, the apex of the lungs faces the positive direction of the Z axis in the three-dimensional image (i.e., the base of the lungs faces the direction of the Z axis 0), the chest faces the direction of the Y axis 0 (i.e., the back faces the positive direction of the Y axis), and the right lung is close to the direction of the X axis 0 (i.e., the left lung is close to the positive direction of the X axis). In this way, the three-dimensional image after the direction is unified is convenient for subsequent lung lobe segmentation.

[0170] In some optional embodiments, the three-dimensional image of the lungs to be segmented can also be the data after the above-mentioned direction unification is performed and then resampled in the X-axis and Y-axis directions (i.e., corresponding to the height and width). For example, the original three-dimensional medical image itself and the size after the direction unification are (D, H", W"), where D, H", and W" are all positive integers, respectively used to represent the depth, height, and width of the original three-dimensional medical image. After the original three-dimensional medical image of size (D, H", W") is resampled in the X-axis and Y-axis directions (i.e., corresponding to the height and width) after the direction unification, a three-dimensional image of the lungs to be segmented of size (D, H, W) is obtained, where H and W are fixed values, for example, H and W are both 256, and due to the characteristics of medical three-dimensional images, it can be seen that the depth D is not fixed.

[0171] In step 202 , the four-channel image generation step is performed using the three-dimensional lung image to be segmented as the input three-dimensional lung image, and the obtained four-channel three-dimensional lung image is used as the four-channel three-dimensional lung image to be segmented.

[0172] In this embodiment, the four-channel image generation step specifically includes the following steps: Figure 2B The following steps 2021 to 2024 are shown:

[0173] Step 2021: Based on the lung and trachea segmentation model, perform lung and trachea segmentation on the input three-dimensional lung image to obtain a trachea segmentation result.

[0174] Here, the lung and trachea segmentation model can be various semantic segmentation models currently known or developed in the future, and this disclosure does not specifically limit this. As an example, the lung and trachea segmentation model can be 3D-Unet, MedLSAM (Medical Long Short-Term Attention Mechanism), SegFormer3D, etc.

[0175] Here, the obtained trachea segmentation result may include tracheal voxels and non-tracheal voxels. For example, the voxel value of the tracheal voxel is 1, and the voxel value of the non-tracheal voxel is 0.

[0176] Step 2022: Generate a tracheal skeleton segmentation result based on the tracheal segmentation result.

[0177] Here, various skeleton generation methods currently known or developed in the future can be used to generate the tracheal skeleton segmentation result based on the tracheal segmentation result, and this disclosure does not specifically limit this. As an example, the above-mentioned skeleton generation method may include but is not limited to the following methods:

[0178] Kernel filter method: Iteratively erodes the surface of the trachea until only the skeleton remains.

[0179] Decision tree method: It iteratively processes all possible binary combinations of tracheal voxels and non-tracheal voxels in the 26-neighborhood and finds all the surface points that can be deleted at each iteration.

[0180] 3D skeleton extraction algorithm implemented by ITK function: ITK (Insight Segmentation and Registration Toolkit) provides a 3D skeleton extraction algorithm based on C++ and Python.

[0181] Topological refinement algorithm: After morphological processing, the voxel points in the lung trachea segmentation result are judged as simple points. The simple points are deleted using the Euler characteristic and connectivity conditions to obtain the tracheal skeleton segmentation result.

[0182] Algorithm based on mathematical morphology: Utilizes structural elements to collect information about tracheal segmentation results, defines different morphological operators, and applies them to the tracheal segmentation results to achieve the purpose of emphasizing or weakening specific voxels, thereby extracting tracheal skeleton voxels.

[0183] Tracking-based method: Starting from a voxel on the boundary of a known blood vessel part, adjacent boundary points are searched and connected in sequence, thereby gradually detecting the entire tracheal boundary or central axis.

[0184] Here, the obtained tracheal skeleton segmentation result includes tracheal skeleton voxels and non-tracheal skeleton voxels. For example, the voxel value of the tracheal skeleton voxel is 1, and the voxel value of the non-tracheal skeleton voxel is 0. In addition, optionally, there are other adjacent tracheal skeleton voxels in the cubic neighborhood of any tracheal skeleton voxel. As an example, the above-mentioned cubic neighborhood can be an M*M*M voxel cubic neighborhood, where M is a positive integer greater than or equal to 2, and then the cubic neighborhood of any tracheal skeleton voxel can be an (M3-1) neighborhood. Preferably, M can be 3.

[0185] Step 2023: Generate a tracheal connectivity map based on the tracheal skeleton segmentation result.

[0186] Here, various connectivity graph generation algorithms can be used to generate a tracheal connectivity graph based on the tracheal skeleton segmentation result. The generated tracheal connectivity graph includes a three-dimensional tangent vector from each tracheal skeleton voxel along the tracheal skeleton to the tracheal root skeleton voxel fastest.

[0187] In some optional implementations, step 2023 may include: Figure 2C The following steps 20231 to 20233 are shown:

[0188] Step 20231: Determine the trachea root skeleton voxel in each trachea skeleton voxel according to the lung trachea distribution direction corresponding to the trachea skeleton segmentation result.

[0189] It should be noted that the tracheal skeleton segmentation result can be pre-assigned to the corresponding lung tracheal distribution direction. For example, if the distribution directions of the lung organ display objects in the 3D lung image to be segmented are uniformly processed, then the distribution direction of the lung trachea in the tracheal skeleton segmentation result is also determined. The tracheal root skeleton voxels can then be determined based on the lung tracheal distribution direction.

[0190] As an example, when the apex of the lung in the three-dimensional image to be segmented faces the positive direction of the Z axis in the three-dimensional image (i.e., the base of the lung faces the Z axis 0 direction), the tracheal skeleton voxel with the largest Z axis coordinate in the tracheal skeleton segmentation result can be determined as the tracheal root skeleton voxel.

[0191] Step 20232: For each tracheal skeleton voxel, use breadth-first search to determine the shortest path from the tracheal skeleton voxel to the tracheal root skeleton voxel, and determine the three-dimensional vector of the strongest direction point voxel pointing from the tracheal skeleton voxel to the tracheal skeleton voxel as the three-dimensional tangent vector of the tracheal skeleton voxel.

[0192] Here, since there are other adjacent tracheal voxels within the M*M*M cubic neighborhood of any tracheal skeleton voxel in the tracheal segmentation result, the two adjacent tracheal skeleton voxels in the shortest path from any tracheal skeleton voxel to the tracheal root skeleton voxel are each other's M*M*M cubic neighborhoods. Accordingly, the strongest direction point voxel of the tracheal skeleton voxel is the tracheal skeleton voxel reached by advancing M voxels along the corresponding shortest path from the tracheal skeleton voxel toward the tracheal root skeleton voxel. The strongest direction point voxel determined in this way is used to indicate the fastest direction of travel from the tracheal skeleton voxel to the tracheal root voxel. The vector obtained by subtracting the three-dimensional coordinates of the tracheal skeleton voxel from the three-dimensional coordinates of the strongest direction point voxel is the three-dimensional tangent vector of the tracheal skeleton voxel.

[0193] After step 20232, the three-dimensional tangent vector of each tracheal skeleton voxel can be obtained.

[0194] Step 20233: Generate a tracheal connectivity map based on the three-dimensional tangent vectors of each tracheal skeleton voxel.

[0195] Here, the generated tracheal connectivity map has the same size as the tracheal skeleton segmentation result. The three channel values of the voxels in the tracheal connectivity map corresponding to each tracheal skeleton voxel in the tracheal skeleton segmentation result are the three values in the three-dimensional tangent vector of the corresponding tracheal skeleton voxel, while the three channel values of the voxels corresponding to each non-tracheal skeleton voxel in the tracheal skeleton segmentation result can be preset specified values corresponding to the corresponding channels, for example, they can all be 0.

[0196] The tracheal connectivity map generated in this manner can be used to characterize the distribution, structure, and orientation of the pulmonary airways. This information can be used to assist in understanding the anatomical structure of the lungs. The human lungs have two lobes on the left and three on the right. The distribution of airways within different lobes varies, so the tracheal connectivity map can be used to assist in understanding and distinguishing between different lobes.

[0197] In step 2024, the tracheal connectivity map and the input three-dimensional lung image are combined to obtain a four-channel three-dimensional lung image.

[0198] It should be noted that the image size of the tracheal connectivity map is the same as the image size of the input lung 3D image, for example, (D, H, W), where D, H, and W are all positive integers representing the depth, height, and width of the input lung 3D image, respectively. Here, the input lung 3D image can be data resampled in the X-axis and Y-axis directions (i.e., corresponding to the height and width), that is, H and W are fixed values, for example, H and W are both 256. However, due to the characteristics of medical 3D imaging, the depth D is not fixed.

[0199] Each voxel in the input lung 3D image corresponds to only one value, that is, each voxel is single-channel. However, each voxel in the tracheal connectivity map corresponds to a three-dimensional tangent vector, that is, corresponds to three values, and thus each voxel is three-channel. When merging, a three-dimensional image with the same size as the tracheal connectivity map and the input lung 3D image can be generated, in which each voxel has four channels, that is, corresponds to four values, which are the voxel value of the corresponding voxel in the input lung 3D image and the three voxel values in the tracheal connectivity map, thereby obtaining a four-channel lung 3D image, whose size can be, for example, (4, D, H, W).

[0200] After step 202, the four-channel image generation step is performed using the three-dimensional lung image to be segmented as the input three-dimensional lung image, and a four-channel three-dimensional lung image corresponding to the three-dimensional lung image to be segmented can be obtained, that is, the four-channel three-dimensional lung image to be segmented.

[0201] Step 203 : Based on the lung lobe feature extraction model, feature extraction is performed on the four-channel three-dimensional lung image to be segmented to obtain a feature map of the three-dimensional lung lobe to be segmented.

[0202] In this embodiment, the execution entity may perform feature extraction on the four-channel three-dimensional lung image to be segmented obtained in step 202 based on the lung lobe feature extraction model to obtain a three-dimensional lung lobe feature map to be segmented.

[0203] It should be noted that each voxel in the three-dimensional lung lobe feature map to be segmented has a corresponding lung lobe feature and corresponds one-to-one to the voxel in the three-dimensional lung image to be segmented. As an example, when the size of the four-channel three-dimensional lung image is (4, D, H, W), the size of the three-dimensional lung lobe feature map to be segmented can be (F, D, H, W), where F is a positive integer used to characterize the feature dimension of each lung lobe feature, for example, F can be 256, and H and W are also 256. Furthermore, the three-dimensional lung lobe feature map to be segmented after feature extraction also has (D*H*W) voxels, and each voxel corresponds to a lung lobe feature with a dimension of (F*1*1*1).

[0204] Here, the four-channel three-dimensional lung image to be segmented can be directly input into the lung lobe feature extraction model, and the lung lobe feature extraction model directly outputs a three-dimensional lung lobe feature map to be segmented with a size of (F, D, H, W).

[0205] Alternatively, the four-channel three-dimensional lung image to be segmented can be first input into the lung lobe feature extraction model and a feature map can be output, and then the feature map output by the lung lobe feature extraction model can be resampled to generate a three-dimensional lung lobe feature map to be segmented with a size of (F, D, H, W).

[0206] Here, the lung lobe feature extraction model can be various feature extraction models. As an example, the lung lobe feature extraction model can be the feature extraction part of the resnetX101 model pre-trained based on MS COCO (Microsoft Common Objects in Context, MS COCO is a large image dataset built by Microsoft), and then the output layer is connected to a 1*1*1 convolution layer. The size of the feature map output by the lung lobe feature model is (F, D, H', W'), where H' and W' are both positive integers and are different from H and W respectively. For example, H' can be H / 4, and W' can be W / 4. Then, the feature map of size (F, D, H', W') output by the lung lobe feature model is resampled to obtain a three-dimensional lung lobe feature map to be segmented of size (F, D, H, W).

[0207] Step 204 : Calculate the similarity between the lung lobe feature corresponding to each voxel in the three-dimensional lung lobe feature map to be segmented and the prototype feature corresponding to each preset lung lobe category in the preset lung lobe category set.

[0208] As an example, when the size of the three-dimensional lung lobe feature map to be segmented is (F, D, H, W), it is necessary to calculate the similarity between the lung lobe feature of dimension (F*1*1*1) of each voxel in the (D*H*W) voxels and the prototype feature corresponding to each preset lung lobe category in the preset lung lobe category set. Assuming that the preset lung lobe category set includes 6 preset lung lobe categories, that is, for each voxel in the D*H*W voxels, the similarity between the lung lobe feature of dimension (F*1*1*1) corresponding to the voxel and the prototype feature of dimension (F*1*1*1) corresponding to each preset lung lobe category in the 6 preset lung lobe categories is calculated.

[0209] Optionally, the preset lobe category set may include six lobe categories: "upper left lobe", "lower left lobe", "upper right lobe", "right middle lobe", "lower right lobe" and "background". Each lobe category corresponds to a prototype feature, and the prototype feature of each lobe category may also be a feature with a dimension of (F*1*1*1).

[0210] Here, various vector similarity calculation methods may be used, such as cosine similarity.

[0211] In some optional embodiments, step 204 may include: Figure 2D The following steps 2041 and 2042 are shown:

[0212] Step 2041 , clustering the lung lobe features corresponding to all voxels in the three-dimensional lung lobe feature map to be segmented, to obtain N cluster center lung lobe features and a cluster center lung lobe feature corresponding to each voxel.

[0213] Here, N is the number of preset lung lobe categories in the preset lung lobe category set.

[0214] Assuming that the size of the three-dimensional lung lobe feature map to be segmented is (F, D, H, W), we can cluster the (D*H*W) lung lobe features with a dimension of (F*1*1*1), and obtain N cluster center lung lobe features through clustering. We can also determine the cluster center corresponding to the cluster to which each voxel belongs. That is to say, clustering can determine the cluster center lung lobe feature with a dimension of (F*1*1*1) corresponding to each voxel.

[0215] Here, in practice, three-dimensional medical images taken of different lung lobe regions of the human body have different image features, and thus each voxel in different lung lobe regions corresponding to the three-dimensional lung lobe feature map obtained based on steps 202 and 203 will also have different lung lobe features. By clustering the lung lobe features of all voxels into N categories, the cluster center lung lobe features of each category can characterize the corresponding lung lobe category to a certain extent.

[0216] Here, various clustering algorithms currently known or developed in the future can be used for clustering, and the present disclosure does not make specific limitations on this. For example, the algorithms that can be used include but are not limited to K-Means clustering algorithm, hierarchical clustering algorithm (HierarchicalClustering), DBSCAN (Density-Based Spatial Clustering of Applications withNoise), OPTICS (Ordering Points To Identify the Clustering Structure), MeanShift, BIRCH (Balanced Iterative Reducing and Clustering using Hierarchies), AGNES (Agglomerative Nesting), DENSITY CLUSTERING, CLARA (Clustering LARgeApplications based on randomized search), CLARANS (Clustering LARgeApplications based on randomized search), G-Means, Fuzzy C-Means (FCM), etc.

[0217] Step 2042, for each voxel in the three-dimensional lung lobe feature map to be segmented, calculate the average feature of the lung lobe feature of the voxel and the lung lobe feature of the cluster center corresponding to the voxel, and determine the similarity between the average feature and the prototype feature corresponding to each preset lung lobe category in the preset lung lobe category set as the similarity between the lung lobe feature corresponding to the voxel and the prototype feature corresponding to the corresponding preset lung lobe category.

[0218] Here, assuming that the size of the feature map of the three-dimensional lung lobe to be segmented is (F, D, H, W), each voxel V in the (D*H*W) voxels can be i , i is a natural number between 1 and (D*H*W), the voxel V i The lung lobe feature in the three-dimensional lung lobe feature map to be segmented is P i The cluster center lobe feature corresponding to this voxel is C i , step 2042 may be performed as follows:

[0219] First, calculate P i and C i The average feature E i For easier understanding, please refer to the following formula:

[0220] E i =(P i +C i ) / 2

[0221] Then, calculate the above voxel V i The average feature E i The prototype feature T corresponding to each of the N preset lung lobe categories n The similarity S between i,n , n is a natural number between 1 and N.

[0222] Finally, the similarity S calculated above is i,n Determined as voxel V i The corresponding lung lobe feature and the prototype feature T corresponding to the nth preset lung lobe category n The similarity between them.

[0223] That is to say, the voxel V is not directly calculated here. i The corresponding lung lobe feature P i The prototype feature T corresponding to the nth preset lung lobe category n The similarity between them is obtained by first transforming the lung lobe feature P corresponding to the voxel Vi i The cluster center lobe feature corresponding to the voxel is C i The average feature E is obtained by averaging i , and then calculate the average feature E iThe similarity between the prototype feature Tn corresponding to the nth preset lung lobe category. In this process, the average feature E i The original lung lobe feature P i The cluster center lobe feature is C i The voxels are clustered together, and the average feature E of each voxel after clustering i It will be closer to the corresponding cluster center lobe feature C i , and then the calculated voxel V i The corresponding average feature E i The prototype feature T corresponding to the nth preset lung lobe category n The similarities between them are also clustered accordingly, which is more in line with the actual situation that voxels of different lung lobe categories have more similar lung lobe characteristics, thus helping to improve the accuracy of subsequent determination of the lung lobe category to which each voxel belongs.

[0224] Here, the prototype features of each preset lung lobe category can be obtained as follows: First, a three-dimensional lung feature map sample is obtained based on a three-dimensional lung image sample labeled with the lung lobe category using the same method as steps 202 and 203. Then, for each preset lung lobe category, a prototype feature map of the preset lung lobe category is determined based on the lung lobe features corresponding to the voxels in the three-dimensional lung feature map sample labeled with the preset lung lobe category.

[0225] In some optional embodiments, the lung lobe feature extraction model and the prototype features corresponding to each preset lung lobe category in the preset lung lobe category set are obtained by Figure 3A The training step 300 shown is predetermined and includes the following steps 301 to 306:

[0226] Step 301: Acquire a first sample set and a second sample set.

[0227] Here, the first sample includes a first lung 3D image and a corresponding labeled lung lobe segmentation result, and the second sample includes a second lung 3D image and a corresponding labeled lung lobe segmentation result.

[0228] Optionally, the first lung 3D image and the second lung 3D image may be 3D images obtained by unifying the orientation of the original 3D medical image and resampling the X-axis and Y-axis. For details, please refer to the relevant description in step 201 and will not be repeated here.

[0229] Here, the labeled lung lobe category segmentation result of the first lung three-dimensional image is used to indicate the preset lung lobe category corresponding to each voxel in the first lung three-dimensional image, and the labeled lung lobe category segmentation result of the second lung three-dimensional image is used to indicate the preset lung lobe category corresponding to each voxel in the second lung three-dimensional image.

[0230] In step 302 , a four-channel image generation step is performed using each first three-dimensional lung image and each second three-dimensional lung image as input three-dimensional lung images to obtain corresponding first four-channel three-dimensional lung images and second four-channel three-dimensional lung images.

[0231] Here, the description of the four-channel image generation step can refer to the above description of step 202, which will not be repeated here.

[0232] Through step 302 , a first four-channel three-dimensional lung image corresponding to each first three-dimensional lung image and a second four-channel three-dimensional lung image corresponding to each second three-dimensional lung image can be obtained.

[0233] As an example, assuming that there are 5 first samples in the first sample set and 25 second samples in the second sample set, 5 first four-channel 3D lung images and 25 second four-channel 3D lung images can be obtained after step 302. Assuming that the image dimensions of the first and second 3D lung images are (D, H, W), the image dimensions of the first and second four-channel 3D lung images can be (4, D, H, W).

[0234] Step 303: Execute parameter adjustment operations until the preset parameter adjustment end conditions are met.

[0235] Here, the parameter adjustment operation may specifically include: Figure 3B The following steps are shown:

[0236] Step 3031: Based on the lung lobe feature extraction model, feature extraction is performed on each first four-channel three-dimensional lung image and each second four-channel three-dimensional lung image to obtain a first three-dimensional lung lobe feature map and a second three-dimensional lung lobe feature map.

[0237] Here, the execution process of step 3031 can refer to the relevant description of step 203 above, which will not be repeated here.

[0238] Continuing with the example of step 302, after step 3031, 5 first three-dimensional lung lobe feature maps and 25 second three-dimensional lung lobe feature maps can be obtained. The image size of each first three-dimensional lung lobe feature map and the second three-dimensional lung lobe feature map can be (F, D, H, W). That is, the voxel in each first three-dimensional lung lobe feature map has a corresponding lung lobe feature of dimension (F, 1, 1, 1), and corresponds one-to-one with the voxel in the first four-channel lung three-dimensional image. The voxel in each second three-dimensional lung lobe feature map has a corresponding lung lobe feature of dimension (F, 1, 1, 1), and corresponds one-to-one with the voxel in the second four-channel lung three-dimensional image.

[0239] In step 3032, each first four-channel three-dimensional lung image and the corresponding labeled lung lobe segmentation result are used as input support samples, and each second four-channel three-dimensional lung image and the corresponding labeled lung lobe segmentation result are used as input query samples. The loss function of the input query sample relative to the input support sample is calculated, and the obtained loss function value is used as the first loss function value.

[0240] In step 3033, each second four-channel three-dimensional lung image and the corresponding labeled lung lobe segmentation result are used as input support samples, and each first four-channel three-dimensional lung image and the corresponding labeled lung lobe segmentation result are used as input query samples. The loss function of the input query sample relative to the input support sample is calculated, and the obtained loss function value is used as the second loss function value.

[0241] Here, various loss function calculation methods can be used to calculate the loss function of the query sample relative to the input support sample, which is not specifically limited in this disclosure, for example, cosine similarity loss function (Cosine Similarity Loss), hinge loss (Hinge Loss), etc.

[0242] Alternatively, the loss function of the query sample relative to the input support sample can be calculated as follows:

[0243] First, for each preset lung lobe category in the preset lung lobe categories, the mean feature of the lung lobe features corresponding to each supporting voxel of the preset lung lobe category is calculated, and the above mean feature is determined as the supporting prototype feature of the preset lung lobe category.

[0244] Here, the supporting voxels of the preset lung lobe category are voxels in each input supporting sample whose corresponding labeled lung lobe segmentation results are the preset lung lobe category.

[0245] For example, assuming there are S support samples, the image size of the three-dimensional lobe feature map in each support sample is (F, D, H, W). That is, the voxels in the three-dimensional lobe feature map in each support sample have corresponding lobe features of dimension (F, 1, 1, 1), and each voxel is associated with a corresponding lobe category. Following the above assumptions about the six preset lobe categories, for one of the preset lobe categories, such as the left upper lobe category, the mean feature of the lobe features (F, 1, 1, 1) corresponding to all voxels marked as left upper lobe in the S support samples can be determined as the support prototype feature of the left upper lobe category. That is, the support prototype feature of each preset lobe category is also a feature of dimension (F, 1, 1, 1).

[0246] Then, for each voxel in each input query sample, the similarity between the lung lobe feature of the voxel and the supporting prototype feature of each preset lung lobe category is calculated, and the calculated similarity is normalized to obtain the predicted probability of the voxel being each preset lung lobe category.

[0247] Specifically, assume that there are Q query samples, and the image size of the three-dimensional lung lobe feature map in each query sample is (F, D, H, W), in which there are (D*H*W) voxels, and each voxel has a lung lobe feature of dimension (F, 1, 1, 1). Here, for each query sample in the Q query samples, the similarity between the lung lobe feature of dimension (F, 1, 1, 1) corresponding to the voxel in the three-dimensional lung lobe feature map of dimension (F, D, H, W) in the query sample is calculated and the support prototype feature of dimension (F, 1, 1, 1) of each preset lung lobe category is calculated. The calculated similarity is then normalized by a normalization method (for example, by inputting a Softmax function) to obtain the predicted probability that the voxel belongs to the corresponding preset lung lobe category. Finally, the predicted probability that each voxel in each query sample in the Q query samples corresponds to each of the 6 preset lung lobe categories can be obtained, that is, Q*D*H*W*6 predicted probability values can be obtained in the end.

[0248] Finally, the loss function value is calculated based on the difference between the predicted probability that the voxels in each input query sample are each preset lung lobe category and the labeled lung lobe segmentation result corresponding to the corresponding voxel.

[0249] After steps 3032 and 3033, a first loss function value and a second loss function value can be obtained, respectively. The first loss function value is the loss function value of the second four-channel three-dimensional lung image and the corresponding labeled lung lobe segmentation result relative to each first four-channel three-dimensional lung image and the corresponding labeled lung lobe segmentation result. The second loss function value is the loss function value of the first four-channel three-dimensional lung image and the corresponding labeled lung lobe segmentation result relative to each second four-channel three-dimensional lung image and the corresponding labeled lung lobe segmentation result.

[0250] Step 3034: Adjust the model parameters of the lung lobe feature extraction model based on the first loss function value and the second loss function value.

[0251] Here, the total loss function value can be first determined based on the first loss function value and the second loss function value, and then various model parameter optimization methods can be used to adjust the model parameters of the lung lobe feature extraction model based on the total loss function value.

[0252] Optionally, the sum of the first loss function value and the second loss function value may be determined as the total loss function value. Alternatively, the mean of the first loss function value and the second loss function value may be determined as the total loss function value.

[0253] It should be noted that the cost of pixel-level annotation of three-dimensional lung images is high. Therefore, in practice, it is difficult to obtain a large number of three-dimensional lung images with annotated lung lobe segmentation results (i.e., annotated samples). In order to solve the problem of inaccurate lung lobe segmentation results due to the small number of annotated samples, in the process of optimizing the model parameters of the lung lobe feature extraction model based on the first sample set and the second sample set in steps 301 to 303, the annotated samples are not taken as a whole. Instead, the annotated samples are divided into a first sample set and a second sample set, and the first sample set and the second sample set are used as query samples and support samples respectively. Then, the second sample set and the first sample set are used as query samples and support samples respectively, and the first loss function value and the second loss function value are calculated respectively. Finally, the model parameters of the lung lobe feature extraction model are adjusted based on the above two loss function values, thereby making full use of the annotated samples, realizing few-sample segmentation, and improving the accuracy of lung lobe segmentation.

[0254] Step 304 : Determine a prototype four-channel three-dimensional lung image set based on each first four-channel three-dimensional lung image and / or each second four-channel three-dimensional lung image.

[0255] Here, the first four-channel lung three-dimensional images can be used to generate a prototype four-channel lung three-dimensional image set, the second four-channel lung three-dimensional images can be used to generate a prototype four-channel lung three-dimensional image set, or the first four-channel lung three-dimensional images and the second four-channel lung three-dimensional images can be used to generate a prototype four-channel lung three-dimensional image set.

[0256] For example, assuming that the first sample set includes S first samples and the second sample set includes Q second samples, then if a prototype four-channel three-dimensional lung image set is generated using the first four-channel three-dimensional lung image and each second four-channel three-dimensional lung image in step 304, the prototype four-channel three-dimensional lung image set will include (S+Q) prototype four-channel three-dimensional lung images.

[0257] Step 305 : Based on the lung lobe feature extraction model, feature extraction is performed on each prototype four-channel lung three-dimensional image to obtain a prototype three-dimensional lung lobe feature map set.

[0258] Here, the specific implementation of step 305 and the technical effects it may bring can be referred to the relevant records in step 3031, and will not be repeated here.

[0259] After step 305 , a set of prototype three-dimensional lung lobe feature maps corresponding to each prototype four-channel three-dimensional lung image can be obtained.

[0260] Continuing with the assumption in step 304, the prototype four-channel three-dimensional lung image set includes (S+Q) prototype four-channel three-dimensional lung images, and the prototype three-dimensional lung lobe feature map set includes (S+Q) prototype three-dimensional lung lobe feature maps, and the size of each prototype three-dimensional lung lobe feature map is (F, D, H, W). That is, the voxels in each prototype three-dimensional lung lobe feature map have corresponding lung lobe features of dimensions (F, 1, 1, 1), and correspond one-to-one with the voxels in the prototype four-channel three-dimensional lung image. That is, (S+Q) prototype three-dimensional lung lobe feature maps of dimensions (F, D, H, W) can be obtained.

[0261] Step 306 : For each preset lung lobe category in the preset lung lobe categories, calculate the mean feature of the lung lobe features corresponding to each prototype voxel of the preset lung lobe category, and determine the mean feature as the prototype feature of the preset lung lobe category.

[0262] Here, since the voxels in each prototype three-dimensional lung lobe feature map correspond one-to-one to the voxels in the prototype four-channel lung three-dimensional image, and the voxels in the prototype four-channel lung three-dimensional image correspond one-to-one to the voxels in the first four-channel lung three-dimensional image and / or the second four-channel lung three-dimensional image, the voxels in each prototype three-dimensional lung lobe feature map also have corresponding labeled lung lobe categories.

[0263] Therefore, here, for each preset lobe category in the preset lobe category, the voxel with the corresponding labeled lobe segmentation result of each prototype three-dimensional lobe feature map as the preset lobe category can be first determined as the prototype voxel of the preset lobe category. For example, if the preset lobe category is the upper left lobe, then the voxels labeled as the upper left lobe in each voxel of the prototype three-dimensional lobe feature map with a size of (F, D, H, W) can be first found, and these voxels can be determined as the upper left lobe prototype voxels, assuming that there are L upper left lobe prototype voxels. Since each voxel in the prototype three-dimensional lobe feature map corresponds to a lobe feature, such as a lobe feature with a size of (F, 1, 1, 1), the mean feature of the lobe features corresponding to each prototype voxel of the preset lobe category can then be calculated, and the mean feature can be determined as the prototype feature of the preset lobe category. For example, the mean feature of the lung lobe features of size (F, 1, 1, 1) corresponding to the above L left upper lobe prototype voxels can be calculated, and the mean feature obtained by the above calculation can be used as the prototype feature of the left upper lobe lung category, and the size of the above prototype feature is (F, 1, 1, 1).

[0264] The above-mentioned training step 300 can be used to optimize the model parameters of the lung lobe feature extraction model based on a small number of labeled samples, and obtain the prototype features of each preset lung lobe category based on the lung lobe feature extraction model optimized based on the above-mentioned small number of samples, which are used to characterize each preset lung lobe category.

[0265] Step 205 : For each voxel in the three-dimensional lung lobe feature map to be segmented, the lung lobe category with the maximum similarity is determined as the lung lobe category corresponding to the voxel.

[0266] Here, the maximum similarity lung lobe category is the preset lung lobe category with the greatest similarity between the corresponding prototype feature in the preset lung lobe category set and the lung lobe feature corresponding to the voxel.

[0267] That is to say, assuming that the size of the three-dimensional lung lobe feature map to be segmented is (F, D, H, W), and the preset lung lobe category set includes 6 preset lung lobe categories, for each voxel in the D*H*W voxels, the lung lobe category with the greatest similarity between the prototype feature of size (F*1*1*1) corresponding to the 6 preset lung lobe categories and the lung lobe feature of size (F*1*1*1) corresponding to the voxel can be determined as the lung lobe category corresponding to the voxel.

[0268] Step 206 : Generate a lung lobe segmentation result of the three-dimensional lung image to be segmented based on the lung lobe category corresponding to each voxel in the three-dimensional lung lobe feature map to be segmented.

[0269] Assuming that the size of the three-dimensional lung lobe feature map to be segmented is (F, D, H, W), and the preset lung lobe category set includes 6 preset lung lobe categories, a lung lobe segmentation result of size (1, D, H, W) can be generated here. The lung lobe segmentation result includes D*H*W voxels, and the value corresponding to each voxel is used to characterize the lung lobe category corresponding to the voxel.

[0270] The lung lobe segmentation method provided by the above-mentioned embodiment of the present disclosure first performs lung-trachea segmentation on the three-dimensional lung image to be segmented to obtain a trachea segmentation result, and then generates a trachea skeleton segmentation result based on the trachea segmentation result; then generates a trachea connectivity map based on the trachea skeleton segmentation result; then merges the trachea connectivity map and the three-dimensional lung image to be segmented to obtain a four-channel three-dimensional lung image to be segmented; then, performs feature extraction on the four-channel three-dimensional lung image to be segmented to obtain a three-dimensional lung lobe feature map to be segmented; then calculates the similarity between the lung lobe feature corresponding to each voxel in the three-dimensional lung lobe feature map to be segmented and the prototype feature corresponding to each preset lung lobe category in the preset lung lobe category set; then, for each voxel in the three-dimensional lung lobe feature map to be segmented, the preset lung lobe category with the greatest similarity between the prototype feature corresponding to the preset lung lobe category set and the lung lobe feature corresponding to the voxel is determined as the lung lobe category corresponding to the voxel; finally, the lung lobe segmentation result of the three-dimensional lung image to be segmented is generated based on the lung lobe category corresponding to each voxel in the three-dimensional lung lobe feature map to be segmented. Specifically, during the lung lobe segmentation process, the tracheal connectivity map is introduced to understand the lung anatomical structure, thereby increasing the accuracy of lung lobe segmentation. Optionally, by adjusting the lung lobe feature extraction model and generating prototype features corresponding to each preset lung lobe category, a small number of labeled samples are divided into a first sample set and a second sample set, and the first and second sample sets are interchangeably used as query samples and support samples, thereby reducing the number of labeled samples required and thereby reducing the labor cost of labeling samples.

[0271] Reference below Figure 4 , which shows a process 400 of an embodiment of a method for presenting lung multi-organ segmentation results according to the present disclosure. The method for presenting lung multi-organ segmentation results comprises the following steps:

[0272] Step 401: Obtain lung nodule segmentation results, trachea segmentation results, and lung lobe segmentation results of the three-dimensional image of the lung to be segmented.

[0273] Here, the pulmonary nodule segmentation result may be a segmentation result obtained by performing pulmonary nodule segmentation on the 3D lung image to be segmented using various target segmentation methods. The size of the pulmonary nodule segmentation result may be the same as the size of the 3D lung image to be segmented, the voxels in the pulmonary nodule segmentation result may correspond one-to-one with the voxels in the 3D lung image to be segmented, and the voxels in the pulmonary nodule segmentation result may be nodule voxels used to indicate that the corresponding voxels in the 3D lung image to be segmented are nodules, or non-nodule voxels used to indicate that the corresponding voxels in the 3D lung image to be segmented are not nodules.

[0274] Here, the trachea segmentation result may be a segmentation result obtained by performing trachea segmentation on the 3D lung image to be segmented using various target segmentation methods. The size of the trachea segmentation result may be the same as the size of the 3D lung image to be segmented, and the voxels in the trachea segmentation result may correspond one-to-one with the voxels in the 3D lung image to be segmented. The voxels in the trachea segmentation result may be tracheal voxels indicating that the corresponding voxels in the 3D lung image to be segmented are trachea, or non-tracheal voxels indicating that the corresponding voxels in the 3D lung image to be segmented are not trachea.

[0275] The lung lobe segmentation result can be obtained by performing the following steps on the segmented lung 3D image: Figure 2A The size of the lung lobe segmentation result can be the same as the size of the three-dimensional lung image to be segmented, the voxels in the lung lobe segmentation result correspond one-to-one with the voxels in the three-dimensional lung image to be segmented, and the corresponding values of the voxels in the lung lobe segmentation result are used to represent the lung lobe category corresponding to the corresponding voxels in the three-dimensional lung image to be segmented.

[0276] Assume that the size of the three-dimensional image to be segmented is (D, H, W), and the preset lung lobe category set includes 6 preset lung lobe categories, then the sizes of the lung nodule segmentation result, the trachea segmentation result, and the lung lobe segmentation result are also (D, H, W).

[0277] Step 402 : Convert the lung nodule segmentation result, the trachea segmentation result, and the lung lobe segmentation result into three-dimensional objects, respectively, to obtain a lung nodule three-dimensional object, a trachea three-dimensional object, and a lung lobe three-dimensional object.

[0278] Here, various 3D object conversion methods can be used to convert the lung nodule segmentation results, trachea segmentation results, and lung lobe segmentation results into 3D objects, respectively, to obtain 3D lung nodule objects, 3D trachea objects, and 3D lung lobe objects. As an example, the VTK library in Python can be used to convert the lung nodule segmentation results, trachea segmentation results, and lung lobe segmentation results in Nifti format into VTK polygon data format. Here, marching cues are used to extract isosurface values from the 3D voxel data.

[0279] Optionally, the same three-dimensional object conversion method can be used to convert the lung nodule segmentation results, trachea segmentation results and lung lobe segmentation results into three-dimensional objects respectively, and the converted lung nodule three-dimensional object, trachea three-dimensional object and lung lobe three-dimensional object are based on the same reference coordinate system, and then when the lung nodule three-dimensional object, trachea three-dimensional object and lung lobe three-dimensional object are subsequently rendered and presented, the lung nodules, trachea and lung lobes can be presented simultaneously in the same reference coordinate system.

[0280] Step 403 : Render the lung nodule 3D object, the trachea 3D object, and the lung lobe 3D object to obtain a rendered lung nodule 3D object, a rendered trachea 3D object, and a rendered lung lobe 3D object.

[0281] Here, various rendering methods may be used to render the lung nodule 3D object, the trachea 3D object, and the lung lobe 3D object to obtain a rendered lung nodule 3D object, a rendered trachea 3D object, and a rendered lung lobe 3D object.

[0282] In some optional embodiments, the colors of the rendered lung nodule three-dimensional object, the rendered trachea three-dimensional object, and the rendered lung lobe three-dimensional object are different. This can distinguish and display the rendered lung nodule three-dimensional object, the rendered trachea three-dimensional object, and the rendered lung lobe three-dimensional object, making it easier for ordinary users to understand three-dimensional medical images.

[0283] In some optional embodiments, the transparency of the rendered lung nodule three-dimensional object and the rendered trachea three-dimensional object is greater than the transparency of the rendered lung lobe three-dimensional object. In this way, due to the lower transparency of the lung lobe three-dimensional object, the lung nodules and lung trachea can be presented simultaneously in the same lung lobe, which makes it easier for users to understand the overall distribution of multiple lung organs.

[0284] In some optional embodiments, the lung lobe segmentation result may include voxels corresponding to each lung lobe category in a preset lung lobe category set. In addition, the conversion of the lung lobe segmentation result into a lung lobe three-dimensional object may be performed as follows: for each preset lung lobe category, a lung lobe three-dimensional object corresponding to the preset lung lobe category is generated based on the voxels corresponding to the preset lung lobe category in the lung lobe segmentation result. Accordingly, when rendering the lung lobe three-dimensional object, the lung lobe three-dimensional objects corresponding to each preset lung lobe category may be rendered according to different rendering colors. That is, a lung lobe three-dimensional object is generated for each lung lobe category, and the colors of the lung lobe three-dimensional objects of different lung lobe categories are different, so that the user can distinguish the various lung lobes by different colors and understand the actual situation of multiple lung organs.

[0285] As an example, step 403 may be performed as follows:

[0286] In the first step, the association relationship information among the lung nodule 3D object, the trachea 3D object, the lung lobe 3D object, and the rendering method is stored in a JSON file.

[0287] As an example, the association relationship information between the lung nodule 3D object, the trachea 3D object, and the lung lobe 3D object and the rendering method is set as follows:

[0288] Lung nodule voxels in the lung nodule segmentation result: transparency 100, color white;

[0289] Tracheal voxels in the tracheal segmentation result: transparency 100, color yellow;

[0290] The voxels corresponding to different lung lobe categories in the lung lobe segmentation results: transparency 20, different lung lobe category voxels are distinguished by different colors.

[0291] The second step is to use JavaScript to read the JSON file containing the above relationship.

[0292] The third step is to import the VTK.js library.

[0293] The fourth step is to create a renderer instance and a rendering window instance.

[0294] In the fifth step, VTKXMLPolyDataReader is used to read the file paths of the lung nodule 3D object, trachea 3D object, and lung lobe 3D object in the json file, and the corresponding files are read as the files where the lung nodule 3D object, trachea 3D object, and lung lobe 3D object are located to obtain the lung nodule 3D object, trachea 3D object, and lung lobe 3D object.

[0295] In the sixth step, a mapper is created and the read lung nodule 3D object, trachea 3D object, and lung lobe 3D object are set as input data.

[0296] Step 7. Create an Actor and set the mapper to input data.

[0297] Step 8. Add the Actor to the renderer.

[0298] Step 9. Add the renderer to the rendering window.

[0299] The tenth step is to read the corresponding relationship between the rendering mode and the input data stored in the json file, and render the input data according to the corresponding rendering mode.

[0300] Step 404 : presenting a rendered pulmonary nodule 3D object, a rendered trachea 3D object, and a rendered lung lobe 3D object.

[0301] After steps 401 to 404 , the lung nodules, trachea and lung lobes can be simultaneously presented in the same reference coordinate system, making it easier for ordinary users to intuitively understand the three-dimensional medical imaging images.

[0302] In some optional implementations, the method flow 400 may further include the following steps 405:

[0303] Step 405 , executing the rotation and presentation steps every preset time period.

[0304] The rotation and presentation steps here specifically include:

[0305] First, the lung nodule 3D object, the trachea 3D object, and the lung lobe 3D object are rotated by a preset angle.

[0306] Second, the lung nodule 3D object, the trachea 3D object and the lung lobe 3D object are rendered to obtain a rendered lung nodule 3D object, a rendered trachea 3D object and a rendered lung lobe 3D object.

[0307] Third, a rendered lung nodule three-dimensional object, a rendered trachea three-dimensional object, and a rendered lung lobe three-dimensional object are presented.

[0308] That is to say, through step 405, the distribution of multiple organs in the lungs can be displayed by rotating at a fixed speed, which makes it convenient for users to understand the actual situation of the lungs from multiple angles.

[0309] In some optional embodiments, the rendering of the 3D lung nodule object in step 404 may be performed as follows: if at least two 3D lung nodule objects are included, different 3D lung nodule objects are presented in different time periods. That is, only one lung nodule is presented in each time period, which allows the user to view the details of each lung nodule individually.

[0310] In some optional implementations, the pulmonary nodule segmentation result obtained in step 401 may be obtained through the pulmonary nodule segmentation step 500 shown in FIG5 , which includes the following steps 501 to 505:

[0311] Step 501 : performing lung nodule segmentation on the 3D lung image to be segmented based on the lung nodule segmentation model to obtain a preliminary lung nodule segmentation result.

[0312] Here, the preliminary pulmonary nodule segmentation results may include nodule voxels and non-nodule voxels. The pulmonary nodule segmentation model may be various target segmentation models trained based on a large number of pulmonary nodule 3D images and corresponding pixel-level annotated pulmonary nodule segmentation results, which is not specifically limited in this disclosure.

[0313] Step 502 : Perform connected domain analysis based on the preliminary lung nodule segmentation result to determine at least one central voxel of the lung nodule.

[0314] Here, various connected domain analysis methods can be used to perform connected domain analysis based on the preliminary lung nodule segmentation results to determine at least one lung nodule center voxel. Each lung nodule center voxel is used to represent a lung nodule center, or in other words, a lung nodule voxel is the center or centroid of a lung nodule region composed of at least two lung nodule voxels.

[0315] Step 503 : For each central voxel of the lung nodule, a voxel region of a preset three-dimensional size is intercepted from the preliminary lung nodule segmentation result with the central voxel of the lung nodule as the center, to obtain a lung nodule voxel block corresponding to the central voxel of the lung nodule.

[0316] Here, the preset three-dimensional size can be a cube or a cuboid, which is not specifically limited in the present disclosure. The sizes of the lung nodule voxel blocks intercepted according to the same preset three-dimensional size are also the same.

[0317] Step 504 : Based on the false positive identification model, false positive identification of lung nodules is performed on each lung nodule voxel block to determine whether the lung nodule voxel block is a positive lung nodule voxel block or a false positive lung nodule voxel block.

[0318] Here, the false positive identification model can be obtained by training a binary classification model using the voxel blocks obtained by executing steps 502 and 503 on the pixel-level labeled lung nodule segmentation results corresponding to the three-dimensional lung image as input data and the corresponding false positive labeling results as label data.

[0319] Through steps 501 to 504 , it can be determined whether each lung nodule voxel block is a positive lung nodule voxel block or a false-positive lung nodule voxel block, so that the false-positive lung nodule voxel blocks can be removed subsequently and only the positive lung nodule voxel blocks are retained.

[0320] Step 505 : Generate a lung nodule segmentation result based on the positive lung nodule voxel blocks in each lung nodule voxel block.

[0321] Here, in the generated lung nodule segmentation result, the voxels in the positive lung nodule voxel block are nodule voxels, and the other voxels are non-nodule voxels.

[0322] The false positive lung nodules are deleted from the lung nodule segmentation results generated by the above steps 501 to 505, thereby improving the detection accuracy of positive lung nodules.

[0323] Based on the above optional implementation of removing false positive lung nodule voxel blocks, in some optional implementations, the above lung nodule segmentation step 500 may further include the following steps 506 to 508:

[0324] Step 506 : For each positive lung nodule voxel block, the positive lung nodule voxel block is input into a lung nodule density prediction model and a lung nodule calcification score prediction model respectively to obtain a lung nodule density value and a lung nodule calcification score of the positive lung nodule voxel block.

[0325] Here, the lung nodule density prediction model can be a regression model pre-trained based on a large number of lung nodule voxel blocks and corresponding labeled lung nodule density values.

[0326] Here, the lung nodule calcification score prediction model can be a regression model pre-trained based on a large number of lung nodule voxel blocks and corresponding labeled lung nodule calcification scores.

[0327] In practice, various methods can be used to obtain the density value and calcification score of the labeled lung nodules, which are not specifically limited in this disclosure. For example, the following methods can be used, but are not limited to:

[0328] The density of lung nodules can be assessed by the CT value on CT images. The CT value refers to the degree of X-ray absorption by different tissues on CT images, expressed in Hounsfield units (HU).

[0329] Calcified nodules are round or quasi-circular, clearly defined, complete calcium deposits within the lung parenchyma, typically with a CT value exceeding 100 HU. The calcification score of a pulmonary nodule typically refers to the proportion of calcification within the nodule.

[0330] Step 507 : In response to determining that the pulmonary nodule calcification score is greater than a preset calcification score threshold, the nodule category of the positive pulmonary nodule voxel block is determined to be a calcified pulmonary nodule.

[0331] As an example, the preset calcification score threshold may be 0.5, that is, the proportion of the calcified portion inside the nodule is greater than 50%. In this case, the nodule category of the positive lung nodule voxel block may be determined as a calcified lung nodule.

[0332] Step 508 : In response to determining that the pulmonary nodule calcification score is not greater than a preset calcification score threshold, determining the nodule category of the positive pulmonary nodule voxel block according to the pulmonary nodule density value of the positive pulmonary nodule voxel block.

[0333] If it is determined that the pulmonary nodule calcification score is not greater than the preset calcification score threshold, it indicates that the calcification portion inside the nodule accounts for a small proportion, and it is necessary to further determine the nodule category of the positive pulmonary nodule voxel block based on the pulmonary nodule density value of the positive pulmonary nodule voxel block.

[0334] Specifically, in step 508, determining the nodule category of the positive lung nodule voxel block according to the lung nodule density value of the positive lung nodule voxel block may include: Figure 5B Steps 5081 to 5083 are shown:

[0335] Step 5081: In response to the lung nodule density value of the positive lung nodule voxel block being less than a preset low-density threshold, the nodule category of the positive lung nodule voxel block is determined to be a ground glass nodule.

[0336] Step 5082: In response to the lung nodule density value of the positive lung nodule voxel block being not less than a preset low density threshold and less than a preset high density threshold, the nodule category of the positive lung nodule voxel block is determined to be a semi-solid nodule.

[0337] Here, the preset low-density threshold is smaller than the preset high-density threshold.

[0338] Step 5083: In response to the lung nodule density value of the positive lung nodule voxel block being not less than a preset high density threshold, the nodule category of the positive lung nodule voxel block is determined to be a solid nodule.

[0339] For ease of understanding, the process from step 5081 to step 5083 is expressed as follows:

[0340]

[0341] Where C is the nodule category of the positive lung nodule voxel block, ρ is the lung nodule density value of the positive lung nodule voxel block, ρ1 and ρ2 are the preset low density threshold and the preset high density threshold, respectively, and ρ1<ρ2. As an example, ρ1 and ρ2 can be 0.5 and 1.5, respectively.

[0342] By adopting the optional implementation of steps 506 to 508, the lung nodule density value, lung nodule calcification score and nodule category of each positive lung nodule voxel block can be obtained. Furthermore, optionally, the lung multi-organ segmentation result presentation method 400 can also include the following step 406:

[0343] Step 406 : For each positive lung nodule voxel block, present at least one of the following items of the positive lung nodule voxel block: lung nodule density value, lung nodule calcification score, and nodule category.

[0344] By presenting the lung nodule density value, lung nodule calcification score and nodule category of the positive lung nodule voxel block, users can further understand the growth of the nodule and simplify the difficulty of users' understanding of medical imaging data.

[0345] In some optional implementations, please refer to Figure 6 , Figure 6 FIG. 1 shows a schematic diagram of a structure of an embodiment of a lung nodule density prediction model according to the present disclosure. Figure 6 As shown, the lung nodule density prediction model may include a first three-dimensional convolutional neural network and a first fully connected network connected in sequence, and the lung nodule calcification score prediction model may include a second three-dimensional convolutional neural network and a second fully connected network connected in sequence. Accordingly, in step 506, for each positive lung nodule voxel block, the positive lung nodule voxel block is input into the lung nodule density prediction model and the lung nodule calcification score prediction model respectively to obtain the lung nodule density value and lung nodule calcification score of the positive lung nodule voxel block, which may include the following: Figure 5C Steps 5061, 5062, and 5063 are shown:

[0346] In step 5061, the positive lung nodule voxel block is input into a first three-dimensional convolutional neural network to obtain a first voxel block feature map, and the first voxel block feature map is input into a first fully connected network to obtain a lung nodule density value corresponding to the positive lung nodule voxel block.

[0347] In step 5062, the positive lung nodule voxel block is input into a second three-dimensional convolutional neural network to obtain a second voxel block feature map, and the second voxel block feature map is input into a second fully connected network to obtain a lung nodule calcification score corresponding to the positive lung nodule voxel block.

[0348] Step 5063: Input the first voxel block feature map and the second voxel block feature map into a third fully connected network to obtain the risk level value of the malignant lung nodule corresponding to the positive lung nodule voxel block, and determine the malignant lung nodule risk level information of the positive lung nodule voxel block based on the obtained risk level value.

[0349] Here, various implementations can be used to determine the malignant pulmonary nodule risk level information of the positive pulmonary nodule voxel block based on the obtained risk level value. For example, a correspondence between R risk level value ranges and malignant pulmonary nodule risk level information can be pre-set to map the risk level value to the malignant pulmonary nodule risk level information. Here, R is a positive integer greater than or equal to 2.

[0350] Based on the optional implementation of the above steps including step 5061, step 5062 and step 5063, the lung multi-organ segmentation result presentation method 400 may further include the following step 407:

[0351] Step 407 : For each positive lung nodule voxel block, present the risk level information of the malignant lung nodule corresponding to the positive lung nodule voxel block.

[0352] After step 407, the risk level information of each lung nodule as a malignant lung nodule can be presented, so that the user can intuitively understand the malignant condition of the lung nodule.

[0353] refer to Figure 7 , Figure 7 The left side of the figure shows an embodiment of the presentation result of the lung multi-organ segmentation result according to the present disclosure. Figure 7 As shown on the left side of the image, three white lung nodules with a transparency of 100, a yellow trachea with a transparency of 100, and five different colored lung lobes with a transparency of 20 can be seen. Figure 7 As shown in the right side of the figure, the risk level of malignant pulmonary nodules is given for each pulmonary nodule ( Figure 7 Medium risk probability corresponds to: high risk, medium-high risk, medium risk and low risk), nodule type ( Figure 7 Medium type corresponds to: mixed, calcified).

[0354] The method for presenting lung multi-organ segmentation results provided by the above-mentioned embodiment of the present disclosure helps users understand the lung multi-organ segmentation results intuitively and vividly by synchronously presenting three-dimensional objects of lung nodule segmentation results, trachea segmentation results and lung lobe segmentation results, thereby reducing the difficulty of understanding.

[0355] Further references Figure 8 As an implementation of the methods shown in the above figures, the present disclosure provides an embodiment of a lung lobe segmentation device. Figure 2A Corresponding to the method embodiment shown, the device can be specifically applied to various electronic devices.

[0356] like Figure 8As shown, the lung lobe segmentation device 800 of this embodiment includes: an image acquisition module 801, a four-channel image generation module 802, a three-dimensional lung lobe feature map generation module 803, a similarity calculation module 804, a lung lobe category determination module 805 and a lung lobe segmentation module 806. Among them, the image acquisition module 801 is configured to acquire a three-dimensional image of the lung to be segmented; the four-channel image generation module 802 is configured to perform the following four-channel image generation steps with the three-dimensional image of the lung to be segmented as the input three-dimensional lung image, and use the obtained four-channel three-dimensional lung image as the four-channel three-dimensional lung image to be segmented: based on the lung-trachea segmentation model, perform lung-trachea segmentation on the input three-dimensional lung image to obtain a trachea segmentation result, and the trachea segmentation result includes trachea voxels and non-trachea voxels; based on the trachea segmentation result, a trachea skeleton segmentation result is generated, and the trachea skeleton segmentation result includes trachea skeleton voxels and non-trachea skeleton voxels; based on the trachea skeleton segmentation result, a trachea connectivity map is generated, wherein the trachea connectivity map includes a three-dimensional tangent vector from each trachea skeleton voxel along the trachea skeleton to the trachea root skeleton voxel fastest; the trachea connectivity map is merged with the input three-dimensional lung image to obtain a four-channel three-dimensional lung image; the three-dimensional lobe feature map generation module 803 is configured to generate a trachea skeleton voxel based on the lobe feature map. Take the model and perform feature extraction on the four-channel three-dimensional lung image to be segmented to obtain a three-dimensional lung lobe feature map to be segmented, wherein each voxel in the three-dimensional lung lobe feature map to be segmented has a corresponding lung lobe feature and corresponds one-to-one to the voxel in the three-dimensional lung image to be segmented; a similarity calculation module 804 is configured to calculate the similarity between the lung lobe feature corresponding to each voxel in the three-dimensional lung lobe feature map to be segmented and the prototype feature corresponding to each preset lung lobe category in the preset lung lobe category set; a lung lobe category determination module 805 is configured to determine the maximum similarity lung lobe category as the lung lobe category corresponding to each voxel in the three-dimensional lung lobe feature map to be segmented, wherein the maximum similarity lung lobe category is the preset lung lobe category with the greatest similarity between the prototype feature corresponding to the voxel in the preset lung lobe category set and the lung lobe feature corresponding to the voxel; a lung lobe segmentation module 806 is configured to generate a lung lobe segmentation result of the three-dimensional lung image to be segmented based on the lung lobe category corresponding to each voxel in the three-dimensional lung lobe feature map to be segmented.

[0357] In this embodiment, the specific processing and technical effects of the image acquisition module 801, the four-channel image generation module 802, the three-dimensional lung lobe feature map generation module 803, the similarity calculation module 804, the lung lobe category determination module 805 and the lung lobe segmentation module 806 of the lung lobe segmentation device 800 can be referred to respectively. Figure 2A The relevant descriptions of step 201, step 202, step 203, step 204, step 205 and step 206 in the corresponding embodiment are not repeated here.

[0358] In some optional implementations, the similarity calculation module 804 may be further configured to:

[0359] Clustering the lung lobe features corresponding to all voxels in the three-dimensional lung lobe feature map to be segmented to obtain N cluster center lung lobe features and a cluster center lung lobe feature corresponding to each voxel, where N is the number of preset lung lobe categories in the preset lung lobe category set;

[0360] For each voxel in the three-dimensional lung lobe feature map to be segmented, the average feature of the lung lobe feature of the voxel and the lung lobe feature of the cluster center corresponding to the voxel is calculated, and the similarity between the average feature and the prototype feature corresponding to each preset lung lobe category in the preset lung lobe category set is determined as the similarity between the lung lobe feature corresponding to the voxel and the prototype feature corresponding to the corresponding preset lung lobe category.

[0361] In some optional embodiments, the lung lobe feature extraction model and the prototype features corresponding to each preset lung lobe category in the preset lung lobe category set may be predetermined by the following training steps:

[0362] Obtain a first sample set and a second sample set, wherein the first sample includes a first lung 3D image and a corresponding labeled lung lobe segmentation result, and the second sample includes a second lung 3D image and a corresponding labeled lung lobe segmentation result;

[0363] Performing the four-channel image generation step using each first three-dimensional lung image and each second three-dimensional lung image as input three-dimensional lung images to obtain corresponding first four-channel three-dimensional lung images and second four-channel three-dimensional lung images;

[0364] Perform the following parameter adjustment operations until the preset parameter adjustment end conditions are met: based on the lung lobe feature extraction model, perform feature extraction on each first four-channel lung three-dimensional image and each second four-channel lung three-dimensional image respectively to obtain a first three-dimensional lung lobe feature map and a second three-dimensional lung lobe feature map; use each first four-channel lung three-dimensional image and the corresponding labeled lung lobe segmentation result as input support samples, use each second four-channel lung three-dimensional image and the corresponding labeled lung lobe segmentation result as input query samples, calculate the loss function of the input query sample relative to the input support sample, and use the obtained loss function value as the first loss function value; use each second four-channel lung three-dimensional image and the corresponding labeled lung lobe segmentation result as input support samples, use each first four-channel lung three-dimensional image and the corresponding labeled lung lobe segmentation result as input query samples, calculate the loss function of the input query sample relative to the input support sample, and use the obtained loss function value as the second loss function value; adjust the model parameters of the lung lobe feature extraction model based on the first loss function value and the second loss function value;

[0365] Determining a prototype four-channel three-dimensional lung image set based on each of the first four-channel three-dimensional lung images and / or each of the second four-channel three-dimensional lung images;

[0366] Based on the lung lobe feature extraction model, feature extraction is performed on each of the prototype four-channel lung three-dimensional images to obtain a set of prototype three-dimensional lung lobe feature maps;

[0367] For each preset lung lobe category in the preset lung lobe categories, the mean feature of the lung lobe features corresponding to each prototype voxel of the preset lung lobe category is calculated, and the mean feature is determined as the prototype feature of the preset lung lobe category, wherein the prototype voxel of the preset lung lobe category is the voxel whose corresponding labeled lung lobe segmentation result in each prototype three-dimensional lung lobe feature map is the preset lung lobe category.

[0368] In some optional implementations, calculating the loss function of the input query sample relative to the input support sample may include:

[0369] For each preset lung lobe category in the preset lung lobe categories, calculating the mean feature of the lung lobe features corresponding to each supporting voxel of the preset lung lobe category, and determining the mean feature as the supporting prototype feature of the preset lung lobe category, wherein the supporting voxels of the preset lung lobe category are the voxels whose corresponding labeled lung lobe segmentation results in each of the input supporting samples are the preset lung lobe category;

[0370] For each voxel in each of the input query samples, calculating the similarity between the lung lobe feature of the voxel and the supporting prototype feature of each of the preset lung lobe categories, and normalizing the calculated similarity to obtain a predicted probability that the voxel belongs to each of the preset lung lobe categories;

[0371] The loss function value is calculated based on the difference between the predicted probability that the voxels in each input query sample are each preset lung lobe category and the labeled lung lobe segmentation result corresponding to the corresponding voxel.

[0372] In some optional embodiments, generating a tracheal connectivity map based on the tracheal skeleton segmentation result may include:

[0373] Determining a trachea root skeleton voxel in each of the trachea skeleton voxels according to the lung trachea distribution direction corresponding to the trachea skeleton segmentation result;

[0374] For each of the tracheal skeleton voxels, a breadth-first search is used to determine the shortest path from the tracheal skeleton voxel to the tracheal root skeleton voxel, wherein two adjacent tracheal skeleton voxels in the shortest path are cubic neighbors of each other, and a three-dimensional vector of the strongest direction point voxel pointing from the tracheal skeleton voxel to the tracheal skeleton voxel is determined as the three-dimensional tangent vector of the tracheal skeleton voxel, wherein the strongest direction point voxel of the tracheal skeleton voxel is the tracheal skeleton voxel reached by advancing M voxels along the corresponding shortest path toward the tracheal root skeleton voxel;

[0375] The tracheal connectivity map is generated based on the three-dimensional tangent vector of each tracheal skeleton voxel.

[0376] It should be noted that the implementation details and technical effects of each module in the lung lobe segmentation device provided by the embodiments of the present disclosure can be referred to the description of other embodiments in the present disclosure and will not be repeated here.

[0377] Reference below Figure 9 As an implementation of the methods shown in the above figures, the present disclosure provides an embodiment of a device for presenting lung multi-organ segmentation results. Figure 4 Corresponding to the method embodiment shown, the device can be specifically applied to various electronic devices.

[0378] like Figure 9 As shown, the lung multi-organ segmentation result presentation device 900 of this embodiment includes: a segmentation result acquisition module 901, a conversion module 902, a rendering module 903 and a presentation module 904. The segmentation result acquisition module 901 is configured to obtain the lung nodule segmentation result, trachea segmentation result and lung lobe segmentation result of the three-dimensional lung image to be segmented, wherein the lung nodule segmentation result includes nodule voxels and non-nodule voxels, the trachea segmentation result includes trachea voxels and non-trachea voxels, and the lung lobe segmentation result is obtained by performing the following steps on the three-dimensional lung image to be segmented. Figure 2A The lung lobe segmentation method described in any optional embodiment is obtained; the conversion module 902 is configured to convert the lung nodule segmentation result, the trachea segmentation result and the lung lobe segmentation result into three-dimensional objects respectively, and obtain a lung nodule three-dimensional object, a trachea three-dimensional object and a lung lobe three-dimensional object; the rendering module 903 is configured to render the lung nodule three-dimensional object, the trachea three-dimensional object and the lung lobe three-dimensional object, and obtain a rendered lung nodule three-dimensional object, a rendered trachea three-dimensional object and a rendered lung lobe three-dimensional object; the presentation module 904 is configured to present the rendered lung nodule three-dimensional object, the rendered trachea three-dimensional object and the rendered lung lobe three-dimensional object.

[0379] In this embodiment, the specific processing of the segmentation result acquisition module 901, the conversion module 902, the rendering module 903 and the presentation module 904 of the lung multi-organ segmentation result presentation device 900 and the technical effects thereof can be referred to respectively. Figure 4 The relevant descriptions of step 401, step 402, step 403 and step 404 in the corresponding embodiment are not repeated here.

[0380] In some optional embodiments, the rendered lung nodule three-dimensional object, the rendered trachea three-dimensional object, and the rendered lung lobe three-dimensional object have different colors.

[0381] In some optional embodiments, the transparency of the rendered lung nodule three-dimensional object and the rendered trachea three-dimensional object is greater than the transparency of the rendered lung lobe three-dimensional object.

[0382] In some optional embodiments, the lung lobe segmentation result includes voxels corresponding to each lung lobe category in a preset lung lobe category set; and

[0383] The conversion module 902 may be further configured to:

[0384] For each of the preset lung lobe categories, generating a lung lobe three-dimensional object corresponding to the preset lung lobe category based on the voxels corresponding to the preset lung lobe category in the lung lobe segmentation result; and

[0385] The rendering module 903 may be further configured to:

[0386] The three-dimensional lung lobe objects corresponding to the preset lung lobe categories are rendered according to different rendering colors.

[0387] In some optional embodiments, the lung multi-organ segmentation result presentation device 900 may further include:

[0388] The rotation presentation module 905 is configured to perform the following rotation and presentation steps every preset time period: rotating the lung nodule three-dimensional object, the trachea three-dimensional object and the lung lobe three-dimensional object by a preset angle; rendering the lung nodule three-dimensional object, the trachea three-dimensional object and the lung lobe three-dimensional object to obtain a rendered lung nodule three-dimensional object, a rendered trachea three-dimensional object and a rendered lung lobe three-dimensional object; and presenting the rendered lung nodule three-dimensional object, the rendered trachea three-dimensional object and the rendered lung lobe three-dimensional object.

[0389] In some optional implementations, the presentation module 904 may be further configured to:

[0390] If at least two of the rendered pulmonary nodule three-dimensional objects are included, different rendered pulmonary nodule three-dimensional objects are presented in different time periods.

[0391] In some optional embodiments, the pulmonary nodule segmentation result is obtained by the following pulmonary nodule segmentation steps:

[0392] Performing lung nodule segmentation on the three-dimensional lung image to be segmented based on the lung nodule segmentation model to obtain a preliminary lung nodule segmentation result, wherein the preliminary lung nodule segmentation result includes nodule voxels and non-nodule voxels;

[0393] Performing a connected domain analysis based on the preliminary pulmonary nodule segmentation result to determine at least one pulmonary nodule central voxel;

[0394] For each central voxel of a pulmonary nodule, taking the central voxel of the pulmonary nodule as the center, intercepting a voxel region of a preset three-dimensional size in the preliminary pulmonary nodule segmentation result to obtain a pulmonary nodule voxel block corresponding to the central voxel of the pulmonary nodule;

[0395] Based on the false positive identification model, each lung nodule voxel block is subjected to lung nodule false positive identification to determine whether the lung nodule voxel block is a positive lung nodule voxel block or a false positive lung nodule voxel block;

[0396] The lung nodule segmentation result is generated based on the positive lung nodule voxel blocks in each lung nodule voxel block.

[0397] In some optional embodiments, the pulmonary nodule segmentation step may further include:

[0398] For each positive lung nodule voxel block, the positive lung nodule voxel block is input into the lung nodule density prediction model and the lung nodule calcification score prediction model respectively to obtain the lung nodule density value and lung nodule calcification score of the positive lung nodule voxel block;

[0399] In response to determining that the pulmonary nodule calcification score is greater than a preset calcification score threshold, determining the nodule category of the positive pulmonary nodule voxel block as a calcified pulmonary nodule;

[0400] In response to determining that the pulmonary nodule calcification score is not greater than a preset calcification score threshold, a nodule category of the positive pulmonary nodule voxel block is determined according to the pulmonary nodule density value of the positive pulmonary nodule voxel block.

[0401] In some optional embodiments, determining the nodule category of the positive pulmonary nodule voxel block according to the pulmonary nodule density value of the positive pulmonary nodule voxel block includes:

[0402] In response to the lung nodule density value of the positive lung nodule voxel block being less than a preset low-density threshold, determining the nodule category of the positive lung nodule voxel block as a ground glass nodule;

[0403] In response to the lung nodule density value of the positive lung nodule voxel block being not less than a preset low density threshold and less than a preset high density threshold, determining the nodule category of the positive lung nodule voxel block as a semi-solid nodule, wherein the preset low density threshold is less than the preset high density threshold;

[0404] In response to the lung nodule density value of the positive lung nodule voxel block being not less than the preset high-density threshold, the nodule category of the positive lung nodule voxel block is determined to be a solid nodule.

[0405] In some optional embodiments, the lung multi-organ segmentation result presentation device 900 may further include:

[0406] The positive nodule attribute information presentation module 906 is configured to present, for each positive lung nodule voxel block, at least one of the following items of the positive lung nodule voxel block: a lung nodule density value, a lung nodule calcification score, and a nodule category.

[0407] In some optional embodiments, the pulmonary nodule density prediction model may include a first three-dimensional convolutional neural network and a first fully connected network connected in sequence, and the pulmonary nodule calcification score prediction model includes a second three-dimensional convolutional neural network and a second fully connected network connected in sequence; and

[0408] The step of inputting the positive lung nodule voxel block into the lung nodule density prediction model and the lung nodule calcification score prediction model to obtain the lung nodule density value and the lung nodule calcification score corresponding to the positive lung nodule voxel block may include:

[0409] Inputting the positive lung nodule voxel block into the first three-dimensional convolutional neural network to obtain a first voxel block feature map, and inputting the first voxel block feature map into the first fully connected network to obtain a lung nodule density value corresponding to the positive lung nodule voxel block;

[0410] Inputting the positive pulmonary nodule voxel block into the second three-dimensional convolutional neural network to obtain a second voxel block feature map, and inputting the second voxel block feature map into the second fully connected network to obtain a pulmonary nodule calcification score corresponding to the positive pulmonary nodule voxel block;

[0411] The first voxel block feature map and the second voxel block feature map are input into a third fully connected network to obtain a risk level value of a malignant pulmonary nodule corresponding to the positive pulmonary nodule voxel block, and malignant pulmonary nodule risk level information of the positive pulmonary nodule voxel block is determined based on the obtained risk level value.

[0412] In some optional embodiments, the lung multi-organ segmentation result presentation device 900 may further include:

[0413] The malignant risk information presentation module 907 is configured to present, for each positive lung nodule voxel block, the malignant lung nodule risk level information corresponding to the positive lung nodule voxel block.

[0414] It should be noted that the implementation details and technical effects of each module in the lung lobe segmentation device provided by the embodiments of the present disclosure can be referred to the description of other embodiments in the present disclosure and will not be repeated here.

[0415] Reference below Figure 10 , which shows a schematic structural diagram of a computer system 1000 suitable for implementing the electronic device of the present disclosure. Figure 10 The computer system 1000 shown is only an example and should not limit the functions and scope of use of the embodiments of the present disclosure.

[0416] like Figure 10 As shown, the computer system 1000 may include a processing device (e.g., a central processing unit, a graphics processing unit, etc.) 1001, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 1002 or a program loaded from a storage device 1008 into a random access memory (RAM) 1003. Various programs and data required for the operation of the computer system 1000 are also stored in the RAM 1003. The processing device 1001, the ROM 1002, and the RAM 1003 are connected to each other via a bus 1004. An input / output (I / O) interface 1005 is also connected to the bus 1004.

[0417] Typically, the following devices may be connected to the I / O interface 1005: an input device 1006 including, for example, a touch screen, a touchpad, a keyboard, a mouse, a camera, a microphone, etc.; an output device 1007 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; a storage device 1008 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 1009. The communication device 1009 may allow the computer system 1000 to communicate with other devices wirelessly or by wire to exchange data. Although Figure 10 The computer system 1000 of the electronic device having various devices is shown, but it should be understood that it is not required to implement or possess all the devices shown. More or fewer devices may be implemented or possessed instead.

[0418] In particular, according to an embodiment of the present disclosure, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from the network via the communication device 1009, or installed from the storage device 1008, or installed from the ROM 1002. When the computer program is executed by the processing device 1001, the above-mentioned functions defined in the method of the embodiment of the present disclosure are performed.

[0419] It should be noted that the computer-readable medium mentioned above in the present disclosure may be a computer-readable signal medium or a computer-readable storage medium, or any combination of the two. A computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or component, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present disclosure, a computer-readable storage medium may be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, device, or component. In the present disclosure, a computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, which carries computer-readable program code. Such a propagated data signal may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium that can transmit, propagate, or transport a program for use by or in conjunction with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium may be transmitted using any suitable medium, including but not limited to wires, optical cables, RF (radio frequency), etc., or any suitable combination thereof.

[0420] The computer-readable medium may be included in the electronic device, or may exist independently without being incorporated into the electronic device.

[0421] The computer readable medium carries one or more programs. When the one or more programs are executed by the electronic device, the electronic device can realize the following operation: Figure 2A The lung lobe segmentation method shown and / or Figure 4 The illustrated embodiment and its optional implementations illustrate a method for presenting lung multi-organ segmentation results.

[0422] Computer program code for performing the operations of the present disclosure may be written in one or more programming languages, or a combination thereof, including object-oriented programming languages such as Python, Java, Smalltalk, C++, and conventional procedural programming languages such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a stand-alone software package, partially on the user's computer and partially on a remote computer, or entirely on the remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., through the Internet using an Internet service provider).

[0423] The flowcharts and block diagrams in the accompanying drawings illustrate the possible implementation architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each box in the flowchart or block diagram can represent a module, program segment, or a part of code, and the module, program segment, or a part of code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order than that marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flowchart, and the combination of the boxes in the block diagram and / or flowchart, can be implemented with a dedicated hardware-based system that performs the specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0424] The modules described in the embodiments of the present disclosure may be implemented in software or hardware. In some cases, the name of a module does not limit the module itself. For example, an image acquisition module may also be described as a "module for acquiring a three-dimensional image of the lung to be segmented."

[0425] The above description is merely a preferred embodiment of the present disclosure and an illustration of the technical principles employed. Those skilled in the art should understand that the scope of disclosure involved in the present disclosure is not limited to the technical solutions formed by the specific combination of the above-mentioned technical features, but also includes other technical solutions formed by any combination of the above-mentioned technical features or their equivalents without departing from the above-mentioned disclosed concepts. For example, a technical solution formed by replacing the above-mentioned features with (but not limited to) technical features with similar functions disclosed in this disclosure.

Claims

1. A lung lobe segmentation method, comprising: Obtain a three-dimensional image of the lung to be segmented; The following four-channel image generation steps are performed using the to-be-segmented three-dimensional lung image as the input three-dimensional lung image, and the obtained four-channel three-dimensional lung image is used as the to-be-segmented four-channel three-dimensional lung image: based on the lung-trachea segmentation model, the input three-dimensional lung image is subjected to lung-trachea segmentation to obtain a trachea segmentation result, wherein the trachea segmentation result includes trachea voxels and non-trachea voxels; based on the trachea segmentation result, a trachea skeleton segmentation result is generated, wherein the trachea skeleton segmentation result includes trachea skeleton voxels and non-trachea skeleton voxels; based on the trachea skeleton segmentation result, a trachea connectivity map is generated, wherein the trachea connectivity map includes a three-dimensional tangent vector from each trachea skeleton voxel along the trachea skeleton to the trachea root skeleton voxel fastest; the trachea connectivity map is merged with the input three-dimensional lung image to obtain a four-channel three-dimensional lung image; Based on the lung lobe feature extraction model, feature extraction is performed on the four-channel three-dimensional lung image to be segmented to obtain a three-dimensional lung lobe feature map to be segmented, wherein each voxel in the three-dimensional lung lobe feature map to be segmented has a corresponding lung lobe feature and corresponds one-to-one to a voxel in the three-dimensional lung image to be segmented; Calculate the similarity between the lung lobe feature corresponding to each voxel in the three-dimensional lung lobe feature image to be segmented and the prototype feature corresponding to each preset lung lobe category in the preset lung lobe category set; For each voxel in the three-dimensional lung lobe feature map to be segmented, determining the maximum similarity lung lobe category as the lung lobe category corresponding to the voxel, wherein the maximum similarity lung lobe category is the preset lung lobe category with the greatest similarity between the corresponding prototype feature in the preset lung lobe category set and the lung lobe feature corresponding to the voxel; A lung lobe segmentation result of the three-dimensional lung image to be segmented is generated based on the lung lobe category corresponding to each voxel in the three-dimensional lung lobe feature map to be segmented.

2. The method according to claim 1, wherein Calculating the similarity between the lung lobe feature corresponding to each voxel in the three-dimensional lung lobe feature map to be segmented and the prototype feature corresponding to each preset lung lobe category in the preset lung lobe category set includes: Clustering the lung lobe features corresponding to all voxels in the three-dimensional lung lobe feature map to be segmented to obtain N cluster center lung lobe features and a cluster center lung lobe feature corresponding to each voxel, where N is the number of preset lung lobe categories in the preset lung lobe category set; For each voxel in the three-dimensional lung lobe feature map to be segmented, the average feature of the lung lobe feature of the voxel and the lung lobe feature of the cluster center corresponding to the voxel is calculated, and the similarity between the average feature and the prototype feature corresponding to each preset lung lobe category in the preset lung lobe category set is determined as the similarity between the lung lobe feature corresponding to the voxel and the prototype feature corresponding to the corresponding preset lung lobe category.

3. The method according to claim 1, wherein The lung lobe feature extraction model and the prototype features corresponding to each preset lung lobe category in the preset lung lobe category set are predetermined by the following training steps: Obtain a first sample set and a second sample set, wherein the first sample includes a first lung 3D image and a corresponding labeled lung lobe segmentation result, and the second sample includes a second lung 3D image and a corresponding labeled lung lobe segmentation result; Performing the four-channel image generation step using each first three-dimensional lung image and each second three-dimensional lung image as input three-dimensional lung images to obtain corresponding first four-channel three-dimensional lung images and second four-channel three-dimensional lung images; Perform the following parameter adjustment operations until the preset parameter adjustment end conditions are met: based on the lung lobe feature extraction model, perform feature extraction on each first four-channel lung three-dimensional image and each second four-channel lung three-dimensional image respectively to obtain a first three-dimensional lung lobe feature map and a second three-dimensional lung lobe feature map; use each first four-channel lung three-dimensional image and the corresponding labeled lung lobe segmentation result as input support samples, use each second four-channel lung three-dimensional image and the corresponding labeled lung lobe segmentation result as input query samples, calculate the loss function of the input query sample relative to the input support sample, and use the obtained loss function value as the first loss function value; use each second four-channel lung three-dimensional image and the corresponding labeled lung lobe segmentation result as input support samples, use each first four-channel lung three-dimensional image and the corresponding labeled lung lobe segmentation result as input query samples, calculate the loss function of the input query sample relative to the input support sample, and use the obtained loss function value as the second loss function value; adjust the model parameters of the lung lobe feature extraction model based on the first loss function value and the second loss function value; Determining a prototype four-channel three-dimensional lung image set based on each of the first four-channel three-dimensional lung images and / or each of the second four-channel three-dimensional lung images; Based on the lung lobe feature extraction model, feature extraction is performed on each of the prototype four-channel lung three-dimensional images to obtain a set of prototype three-dimensional lung lobe feature maps; For each preset lung lobe category in the preset lung lobe categories, the mean feature of the lung lobe features corresponding to each prototype voxel of the preset lung lobe category is calculated, and the mean feature is determined as the prototype feature of the preset lung lobe category, wherein the prototype voxel of the preset lung lobe category is the voxel whose corresponding labeled lung lobe segmentation result in each prototype three-dimensional lung lobe feature map is the preset lung lobe category.

4. The method according to claim 3, wherein: The calculating the loss function of the input query sample relative to the input support sample includes: For each preset lung lobe category in the preset lung lobe categories, calculating the mean feature of the lung lobe features corresponding to each supporting voxel of the preset lung lobe category, and determining the mean feature as the supporting prototype feature of the preset lung lobe category, wherein the supporting voxels of the preset lung lobe category are the voxels whose corresponding labeled lung lobe segmentation results in each of the input supporting samples are the preset lung lobe category; For each voxel in each of the input query samples, calculating the similarity between the lung lobe feature of the voxel and the supporting prototype feature of each of the preset lung lobe categories, and normalizing the calculated similarity to obtain a predicted probability that the voxel belongs to each of the preset lung lobe categories; The loss function value is calculated based on the difference between the predicted probability that the voxels in each input query sample are each preset lung lobe category and the labeled lung lobe segmentation result corresponding to the corresponding voxel.

5. The method according to claim 1, wherein Generating a tracheal connectivity map based on the tracheal skeleton segmentation result includes: Determining a trachea root skeleton voxel in each of the trachea skeleton voxels according to the lung trachea distribution direction corresponding to the trachea skeleton segmentation result; For each of the tracheal skeleton voxels, a breadth-first search is used to determine the shortest path from the tracheal skeleton voxel to the tracheal root skeleton voxel, wherein two adjacent tracheal skeleton voxels in the shortest path are cubic neighbors of each other, and a three-dimensional vector of the strongest direction point voxel pointing from the tracheal skeleton voxel to the tracheal skeleton voxel is determined as the three-dimensional tangent vector of the tracheal skeleton voxel, wherein the strongest direction point voxel of the tracheal skeleton voxel is the tracheal skeleton voxel reached by advancing M voxels along the corresponding shortest path toward the tracheal root skeleton voxel; The tracheal connectivity map is generated based on the three-dimensional tangent vector of each tracheal skeleton voxel.

6. A method for presenting lung multi-organ segmentation results, comprising: Obtaining a lung nodule segmentation result, a trachea segmentation result, and a lung lobe segmentation result of a three-dimensional lung image to be segmented, wherein the lung nodule segmentation result includes nodule voxels and non-nodule voxels, the trachea segmentation result includes tracheal voxels and non-tracheal voxels, and the lung lobe segmentation result is obtained by performing the lung lobe segmentation method according to any one of claims 1 to 5 on the three-dimensional lung image to be segmented; Converting the lung nodule segmentation result, the trachea segmentation result, and the lung lobe segmentation result into three-dimensional objects respectively to obtain a lung nodule three-dimensional object, a trachea three-dimensional object, and a lung lobe three-dimensional object; Rendering the lung nodule three-dimensional object, the trachea three-dimensional object, and the lung lobe three-dimensional object to obtain a rendered lung nodule three-dimensional object, a rendered trachea three-dimensional object, and a rendered lung lobe three-dimensional object; The rendered lung nodule three-dimensional object, the rendered trachea three-dimensional object, and the rendered lung lobe three-dimensional object are presented.

7. The method according to claim 6, wherein: The colors of the rendered lung nodule three-dimensional object, the rendered trachea three-dimensional object, and the rendered lung lobe three-dimensional object are different; The transparency of the rendered lung nodule three-dimensional object and the rendered trachea three-dimensional object is greater than the transparency of the rendered lung lobe three-dimensional object; The lung lobe segmentation result includes voxels corresponding to each lung lobe category in a preset lung lobe category set; as well as The converting the lung lobe segmentation result into a lung lobe three-dimensional object comprises: For each of the preset lung lobe categories, generating a lung lobe three-dimensional object corresponding to the preset lung lobe category based on the voxels corresponding to the preset lung lobe category in the lung lobe segmentation result; and The rendering of the lung lobe three-dimensional object includes: Rendering the three-dimensional lung lobe objects corresponding to the preset lung lobe categories according to different rendering colors; The method further comprises: The following rotation and rendering steps are performed at preset time intervals: rotating the lung nodule three-dimensional object, the trachea three-dimensional object, and the lung lobe three-dimensional object by a preset angle; rendering the lung nodule three-dimensional object, the trachea three-dimensional object, and the lung lobe three-dimensional object to obtain a rendered lung nodule three-dimensional object, a rendered trachea three-dimensional object, and a rendered lung lobe three-dimensional object; and presenting the rendered lung nodule three-dimensional object, the rendered trachea three-dimensional object, and the rendered lung lobe three-dimensional object. Presenting the rendered lung nodule three-dimensional object includes: If at least two of the rendered pulmonary nodule three-dimensional objects are included, different rendered pulmonary nodule three-dimensional objects are presented in different time periods.

8. The method according to claim 6, wherein: The pulmonary nodule segmentation result is obtained through the following pulmonary nodule segmentation steps: Performing lung nodule segmentation on the three-dimensional lung image to be segmented based on the lung nodule segmentation model to obtain a preliminary lung nodule segmentation result, wherein the preliminary lung nodule segmentation result includes nodule voxels and non-nodule voxels; Performing a connected domain analysis based on the preliminary pulmonary nodule segmentation result to determine at least one pulmonary nodule central voxel; For each central voxel of a pulmonary nodule, taking the central voxel of the pulmonary nodule as the center, intercepting a voxel region of a preset three-dimensional size in the preliminary pulmonary nodule segmentation result to obtain a pulmonary nodule voxel block corresponding to the central voxel of the pulmonary nodule; Based on the false positive identification model, each lung nodule voxel block is subjected to lung nodule false positive identification to determine whether the lung nodule voxel block is a positive lung nodule voxel block or a false positive lung nodule voxel block; Generating the lung nodule segmentation result based on the positive lung nodule voxel blocks in each lung nodule voxel block; The pulmonary nodule segmentation step further includes: For each positive lung nodule voxel block, the positive lung nodule voxel block is input into the lung nodule density prediction model and the lung nodule calcification score prediction model respectively to obtain the lung nodule density value and lung nodule calcification score of the positive lung nodule voxel block; In response to determining that the pulmonary nodule calcification score is greater than a preset calcification score threshold, determining the nodule category of the positive pulmonary nodule voxel block as a calcified pulmonary nodule; In response to determining that the pulmonary nodule calcification score is not greater than a preset calcification score threshold, determining a nodule category of the positive pulmonary nodule voxel block according to the pulmonary nodule density value of the positive pulmonary nodule voxel block; Determining the nodule category of the positive pulmonary nodule voxel block according to the pulmonary nodule density value of the positive pulmonary nodule voxel block includes: In response to the lung nodule density value of the positive lung nodule voxel block being less than a preset low-density threshold, determining the nodule category of the positive lung nodule voxel block as a ground glass nodule; In response to the lung nodule density value of the positive lung nodule voxel block being not less than a preset low density threshold and less than a preset high density threshold, determining the nodule category of the positive lung nodule voxel block as a semi-solid nodule, wherein the preset low density threshold is less than the preset high density threshold; In response to the lung nodule density value of the positive lung nodule voxel block being not less than the preset high-density threshold, determining the nodule category of the positive lung nodule voxel block as a solid nodule; The method further comprises: For each positive lung nodule voxel block, at least one of the following items of the positive lung nodule voxel block is presented: lung nodule density value, lung nodule calcification score and nodule category; The pulmonary nodule density prediction model includes a first three-dimensional convolutional neural network and a first fully connected network connected in sequence, and the pulmonary nodule calcification score prediction model includes a second three-dimensional convolutional neural network and a second fully connected network connected in sequence; and The step of inputting the positive lung nodule voxel block into the lung nodule density prediction model and the lung nodule calcification score prediction model to obtain the lung nodule density value and the lung nodule calcification score corresponding to the positive lung nodule voxel block comprises: Inputting the positive lung nodule voxel block into the first three-dimensional convolutional neural network to obtain a first voxel block feature map, and inputting the first voxel block feature map into the first fully connected network to obtain a lung nodule density value corresponding to the positive lung nodule voxel block; Inputting the positive pulmonary nodule voxel block into the second three-dimensional convolutional neural network to obtain a second voxel block feature map, and inputting the second voxel block feature map into the second fully connected network to obtain a pulmonary nodule calcification score corresponding to the positive pulmonary nodule voxel block; Inputting the first voxel block feature map and the second voxel block feature map into a third fully connected network to obtain a risk level value of a malignant pulmonary nodule corresponding to the positive pulmonary nodule voxel block, and determining malignant pulmonary nodule risk level information of the positive pulmonary nodule voxel block based on the obtained risk level value; The method further comprises: For each positive lung nodule voxel block, the risk level information of the malignant lung nodule corresponding to the positive lung nodule voxel block is presented.

9. A lung lobe segmentation device, comprising: an image acquisition module configured to acquire a three-dimensional image of the lung to be segmented; The four-channel image generation module is configured to perform the following four-channel image generation steps using the three-dimensional lung image to be segmented as the input three-dimensional lung image, and use the obtained four-channel three-dimensional lung image as the four-channel three-dimensional lung image to be segmented: based on the lung-trachea segmentation model, perform lung-trachea segmentation on the input three-dimensional lung image to obtain a trachea segmentation result, wherein the trachea segmentation result includes trachea voxels and non-trachea voxels; generate a trachea skeleton segmentation result based on the trachea segmentation result, wherein the trachea skeleton segmentation result includes trachea skeleton voxels and non-trachea skeleton voxels; generate a trachea connectivity map based on the trachea skeleton segmentation result, wherein the trachea connectivity map includes a three-dimensional tangent vector from each trachea skeleton voxel along the trachea skeleton to the trachea root skeleton voxel fastest; merge the trachea connectivity map with the input three-dimensional lung image to obtain a four-channel three-dimensional lung image; a 3D lung lobe feature map generation module configured to perform feature extraction on the four-channel 3D lung image to be segmented based on a lung lobe feature extraction model to obtain a 3D lung lobe feature map to be segmented, wherein each voxel in the 3D lung lobe feature map to be segmented has a corresponding lung lobe feature and corresponds one-to-one to a voxel in the 3D lung image to be segmented; A similarity calculation module is configured to calculate the similarity between the lung lobe feature corresponding to each voxel in the three-dimensional lung lobe feature map to be segmented and the prototype feature corresponding to each preset lung lobe category in the preset lung lobe category set; a lung lobe category determination module configured to, for each voxel in the three-dimensional lung lobe feature map to be segmented, determine the lung lobe category with the greatest similarity as the lung lobe category corresponding to the voxel, wherein the lung lobe category with the greatest similarity between the prototype feature corresponding to the preset lung lobe category set and the lung lobe feature corresponding to the voxel; The lung lobe segmentation module is configured to generate a lung lobe segmentation result of the three-dimensional lung image to be segmented based on the lung lobe category corresponding to each voxel in the three-dimensional lung lobe feature map to be segmented.

10. A device for presenting lung multi-organ segmentation results, comprising: a segmentation result acquisition module configured to acquire a lung nodule segmentation result, a trachea segmentation result, and a lung lobe segmentation result of the three-dimensional lung image to be segmented, wherein the lung nodule segmentation result includes nodule voxels and non-nodule voxels, the trachea segmentation result includes tracheal voxels and non-tracheal voxels, and the lung lobe segmentation result is obtained by performing the lung lobe segmentation method according to any one of claims 1 to 5 on the three-dimensional lung image to be segmented; a conversion module configured to convert the lung nodule segmentation result, the trachea segmentation result, and the lung lobe segmentation result into three-dimensional objects, respectively, to obtain a lung nodule three-dimensional object, a trachea three-dimensional object, and a lung lobe three-dimensional object; a rendering module configured to render the pulmonary nodule 3D object, the trachea 3D object, and the lung lobe 3D object to obtain a rendered pulmonary nodule 3D object, a rendered trachea 3D object, and a rendered lung lobe 3D object; A rendering module is configured to render the rendered lung nodule three-dimensional object, the rendered trachea three-dimensional object, and the rendered lung lobe three-dimensional object.

11. An electronic device comprising: one or more processors; a storage device having one or more programs stored thereon, When the one or more programs are executed by the one or more processors, the one or more processors are enabled to implement the method according to any one of claims 1 to 5 and / or claims 6 to 8.

12. A computer-readable storage medium having a computer program stored thereon, wherein: When the computer program is executed by one or more processors, the method according to any one of claims 1 to 5 and / or claims 6 to 8 is implemented.

13. A computer program product comprising a computer program / instruction, wherein when the computer program / instruction is executed by a processor, the computer program / instruction implements the method according to any one of claims 1 to 5 and / or claims 6 to 8.

Images