Pulmonary nodule automatic segmentation method based on deep learning

Abstract

The invention discloses a pulmonary nodule automatic segmentation method based on deep learning, and belongs to the field of medical image segmentation, and the method comprises the steps: obtaining a lung medical image data set; performing information labeling on the lung medical image data set to obtain a pulmonary nodule image data training set; constructing a pulmonary nodule segmentation model; training a pulmonary nodule segmentation model by using the pulmonary nodule image data training set to obtain a trained pulmonary nodule segmentation model; and collecting a to-be-detected lung medical image, and carrying out pulmonary nodule identification and segmentation on the to-be-detected lung medical image by using the trained pulmonary nodule segmentation model. The problems that an existing method depends on manual intervention and is insufficient in accuracy are solved.

Description

Technical Field

[0001] This invention belongs to the field of medical image segmentation, and in particular relates to an automatic lung nodule segmentation method based on deep learning. Background Technology

[0002] Lung cancer, a malignant tumor with extremely high incidence and mortality rates, requires early detection and treatment to improve patient survival rates. Pulmonary nodules are an important early manifestation of lung cancer. Timely and accurate detection and segmentation of pulmonary nodules help doctors identify potential lung cancer lesions early, develop personalized treatment plans, and thus significantly improve patients' cure rates and quality of life.

[0003] In lung cancer treatment, information such as the size, location, and morphology of lung nodules is crucial for determining surgical plans, radiotherapy target areas, and chemotherapy drug selection. Accurate lung nodule segmentation provides doctors with detailed nodule characteristics, aiding in the development of more precise and effective treatment plans. Furthermore, dynamic segmentation and monitoring of lung nodules during treatment allows for the assessment of treatment effectiveness and timely adjustments to treatment strategies.

[0004] Existing technical solutions include threshold-based segmentation methods, a simple and direct image segmentation technique. Its basic principle is to classify pixels in an image into different categories based on the image's grayscale characteristics and one or more thresholds. In lung nodule segmentation, by selecting an appropriate threshold, lung nodules can be separated from the background and surrounding tissue. However, due to the complexity of lung images, such as the heterogeneity of lung tissue and noise interference, a single threshold is often insufficient to accurately segment lung nodules, easily leading to incomplete or incorrect segmentation.

[0005] Existing technical solutions include region growing algorithms, which start from one or more seed points and gradually merge adjacent pixels with similar features (such as grayscale values, colors, textures, etc.) into the same region until a stopping condition is met. In lung nodule segmentation, seed points need to be selected manually or automatically first, and then region growing is performed according to a set similarity criterion. The advantage of this method is that it can handle continuous regions with similar features well, but the disadvantage is that the selection of seed points has a significant impact on the segmentation results, and the segmentation effect is often unsatisfactory for lung nodules with unclear boundaries or complex shapes.

[0006] Existing technical solutions include edge detection methods, which determine the boundaries of objects by detecting abrupt changes in grayscale values ​​in an image. Commonly used edge detection operators include the Canny operator and the Sobel operator. In lung nodule segmentation, edge detection can extract the boundary information of lung nodules, thereby achieving segmentation. However, edge detection methods are sensitive to noise, prone to generating false edges, and struggle to accurately extract the complete boundaries of lung nodules with blurred boundaries.

[0007] Traditional lung nodule segmentation methods often rely on the grayscale features of images, requiring high image quality. In actual clinical applications, due to factors such as patient respiratory movements and differences in the performance of scanning equipment, lung images may contain noise, artifacts, and uneven grayscale, all of which affect the accuracy and stability of traditional segmentation methods.

[0008] Furthermore, traditional methods primarily rely on local image features for segmentation, lacking effective utilization of global image context information. The morphology, size, and location of pulmonary nodules are often closely related to the structure and lesion status of the surrounding lung tissue. Traditional methods struggle to fully consider this contextual information, resulting in poor segmentation performance in complex situations.

[0009] In addition, many traditional lung nodule segmentation methods require manual selection of seed points and adjustment of parameters such as thresholds, which is cumbersome and the segmentation results are easily affected by human factors, lacking objectivity and consistency. Summary of the Invention

[0010] To address the aforementioned shortcomings in existing technologies, this invention provides an automatic lung nodule segmentation method based on deep learning, which solves the problems of existing methods relying on manual intervention and lacking accuracy.

[0011] To achieve the aforementioned objectives, the present invention employs the following technical solution: an automatic lung nodule segmentation method based on deep learning, comprising: Obtain a dataset of lung medical images; Information was annotated on the lung medical image dataset to obtain a lung nodule image dataset; A lung nodule segmentation model is constructed. The lung nodule segmentation model includes a lung parenchyma contour feature extraction module for obtaining the contour features of lung parenchyma without nodules, a lung parenchyma contour nodule confirmation module for confirming the nodule status of lung parenchyma based on the contour features of lung parenchyma without nodules, a lung parenchyma extraction module for confirming the lung parenchyma cutting contour based on the nodule status of lung parenchyma and extracting the lung parenchyma image, a lung nodule recognition module for obtaining the lung nodule recognition result based on the lung parenchyma image, a lung nodule center point confirmation module for confirming the coordinates of the lung nodule center point based on the lung nodule recognition result, and a lung nodule segmentation contour extraction module for segmenting lung nodules based on the coordinates of the lung nodule center point. A lung nodule segmentation model was trained using a lung nodule image dataset to obtain the trained lung nodule segmentation model. Medical images of the lungs to be tested are acquired, and a trained lung nodule segmentation model is used to identify and segment lung nodules in the medical images of the lungs to be tested.

[0012] The beneficial effects of this invention are as follows: lung nodule identification is divided into two tasks: linear structure identification and block structure identification, namely, nodules existing on the contour of the lung parenchyma and nodules within the lung parenchyma; the aggregated features of the lung parenchyma contour without nodules are obtained and used as the benchmark features for confirming the nodule status of the lung parenchyma contour. Since lung nodule identification on linear structures requires consideration of more detailed features, the identification of nodules on linear structures is converted into block structures based on the contour magnification mechanism, which can reduce the identification difficulty and program complexity.

[0013] Furthermore, the process of annotating the lung medical image dataset to obtain a lung nodule image dataset specifically involves: The lung medical imaging dataset was divided into a subset of lung medical imaging data with lung nodules and a subset of lung medical imaging data without lung nodules. The outline of lung parenchyma in each lung medical image in a subset of lung medical imaging data without pulmonary nodules is marked. The lung parenchyma outline, lung nodule outline, and lung nodule center point of each lung medical image in the lung medical image data subset labeled with lung nodules; By integrating the annotated subsets of lung medical imaging data containing pulmonary nodules and those without pulmonary nodules, a pulmonary nodule image dataset is obtained:

[0014] in, This is a dataset of lung nodule images; Raw lung medical imaging data; Medical imaging data of the lungs after information annotation; This represents the total number of imaging data for lung nodules.

[0015] The beneficial effect of the above-mentioned further scheme is that the sorted dataset actually includes several data pairs, which prepares for the training of the subsequent lung nodule segmentation model.

[0016] Furthermore, the obtained trained lung nodule segmentation model specifically includes: The lung nodule image dataset is split into a lung nodule image training set and a lung nodule image test set; the ratio of lung medical image data with lung nodules to lung medical image data without lung nodules is the same in the lung nodule image training set and the lung nodule image test set. The lung parenchyma contour feature extraction module was trained using all lung nodule image data without lung nodules in the lung nodule image training set, resulting in a trained lung parenchyma contour feature extraction module. Feature aggregation is performed on the true lung parenchyma contours of all lung parenchyma image data without lung nodules in the lung nodule image training set to obtain the lung parenchyma contour aggregation features without nodules. Obtain current lung nodule images from the lung nodule image training set; The current lung nodule image is input into the lung parenchyma contour nodule confirmation module to obtain the initial lung parenchyma contour of the current lung nodule image, and is compared with the aggregated features of the lung parenchyma contour without nodules to obtain the feature similarity. If the feature similarity is greater than the similarity threshold, there are no nodules in the lung parenchyma contour; otherwise, there are nodules. Input the lung parenchyma contour nodule status of the current lung nodule image into the lung parenchyma extraction module to confirm the lung parenchyma cutting contour of the current lung nodule image and cut out the lung parenchyma image. When there are no nodules in the lung parenchyma contour, input the current lung nodule image into the trained lung parenchyma contour feature extraction module to obtain the lung parenchyma contour of the current lung nodule image, which is used as the lung parenchyma cutting contour of the current lung nodule image. When nodules are present in the lung parenchyma outline, the initial lung parenchyma outline of the current lung nodule image is expanded outward based on a preset expansion ratio to obtain the lung parenchyma cutting outline of the current lung nodule image. Input the lung parenchyma image into the lung nodule recognition module to confirm the number of lung nodules in the lung parenchyma region and obtain the lung nodule recognition result. If the number of lung nodules is 0, the process ends. Otherwise, input the lung nodule recognition result into the lung nodule center point confirmation module to obtain the coordinates of several lung nodule center points. Input the coordinates of the center point of each lung nodule into the lung nodule segmentation contour extraction module to obtain the lung nodule segmentation contour and complete the lung nodule segmentation.

[0017] The beneficial effects of the above-mentioned further scheme are as follows: the overall training of the model is divided into two parts. One is the pre-training of the lung parenchyma contour feature extraction module. After the lung parenchyma contour feature extraction module is trained, the overall training is carried out. The linear to block recognition conversion can improve the overall recognition accuracy of the model, accelerate the recognition, and reduce the complexity.

[0018] Furthermore, the lung parenchyma contour feature extraction module includes: The U-NET network is used to obtain lung segmentation results from raw lung medical imaging data without pulmonary nodules, as an initial contour; The contour processing unit is used to obtain the binary mask and grayscale image of the initial contour. The curvature feature extraction unit is used to obtain the curvature feature map based on the binary mask and grayscale image of the initial contour, using a dual-branch network.

[0019] in, Curvature feature map; Multi-scale feature maps obtained from spatial branching; For local curvature tensors; A binary mask for the initial contour; The x-coordinate is the pixel coordinate. The vertical coordinate is the pixel coordinate. For the first Weights of edge detection images at each scale; For the first Edge detection images at various scales; For scale total; For scale indexing; Enable grayscale; It is a multi-scale structural element; For scale parameters; For the first Structural elements at various scales; Grayscale closure; For expansion operation; For erosion calculation; This is a grayscale image of the initial outline; The dynamic optimization unit is used to perform feature weighting on the boundary pixels of the binary mask of the initial contour based on the curvature feature map to obtain the contour feature map; the weights of the boundary pixels are:

[0020] in, The weights for the boundary pixels; Centered on the convolution kernel; For the major axis; It is the short axis; Curvature weight; For pixels The curvature tensor; It has a small-sized elliptic kernel; It has a large, round core; The curvature threshold used to determine the kernel size; The correction unit is used to obtain a shallow feature map based on the contour feature map using bilinear interpolation, and to obtain a semantic mask based on the shallow feature map using a 1×1 convolutional layer. It also obtains the feature enhancement of pixels in the semantic mask whose organ boundary probability is greater than the probability threshold, thus obtaining the lung parenchyma contour features of the current lung medical image.

[0021] The beneficial effects of the above-mentioned further scheme are as follows: based on the dual-branch network, the grayscale image is used as curvature support to obtain the overall contour curvature features of the lung parenchyma contour, and the pixel weights are confirmed based on the curvature tensor to strengthen the contour features, thereby improving the contour recognition accuracy.

[0022] Furthermore, obtaining the initial lung parenchyma contour of the current lung nodule image specifically involves: A1. Obtain the grayscale image of the current lung nodule and set the block side length; A2. Divide the grayscale image of the current lung nodule into several image blocks based on the block side length; A3. Calculate the average gray level of each image block; A4. Based on the average gray level of each image patch, feature enhancement is performed on each image patch to obtain several enhanced image patches:

[0023] in, For the first One enhanced image patch; For the first Image block pixels grayscale value; For the first The average gray level of each image patch; For the first Rounds Gray-level gain coefficients for each image block; A5. Combine the enhanced image blocks in their original positions to obtain the lung medical enhanced image for the current round; A6. Increase the block side length, return to step A2, and use the current round of lung medical enhancement image as the grayscale image of the current lung nodule image in the next round. Perform block segmentation based on the block side length until the block side length is equal to the side length of the lung medical image to obtain the final lung medical enhancement image. A7. Perform dilation and erosion operations on the final medically enhanced lung images, and confirm the initial lung parenchyma contour of the current lung nodule image based on image morphology.

[0024] The beneficial effects of the above-mentioned further scheme are as follows: The acquisition of the final enhanced lung medical image involves several image block enhancement operations. The initial block side length is small, focusing on local features. Increasing the block side length is also a process of field magnification. Each enhancement preserves the gray-level features within the field of view as much as possible, so that each enhancement does not change the global features of the image block. This iterative enhancement based on the image block size, the next enhancement is a process of inward fusion of the magnified field of view features, thus improving the accuracy of lung parenchyma contour extraction. By performing block segmentation processing on the original lung medical image and analyzing gray-level changes within the blocks, the features of the lung parenchyma region are enhanced step by step, thereby improving the extraction accuracy of the lung parenchyma region and thus improving the overall lung nodule segmentation accuracy.

[0025] Furthermore, the number of lung nodules in the confirmed lung parenchyma region is used to obtain the lung nodule identification result, specifically as follows: The YOLOv8 deep learning model was used to detect and locate lung nodules in the lung parenchyma, and the number of lung nodules and the localization boxes of lung nodules in the lung parenchyma were obtained. Based on a preset contour magnification factor, the lung nodule localization box is magnified, and the magnified lung nodule localization box is cropped to obtain several lung nodule coarse segmentation images. The lung nodule identification results are obtained by integrating the number of lung nodules in the lung parenchyma region and the coarse segmentation images of each lung nodule.

[0026] The beneficial effects of the above-mentioned further scheme are: the contour magnification factor transforms the task of nodule recognition on the contour of lung parenchyma into the task of block recognition, reducing the difficulty and complexity of recognition.

[0027] Furthermore, the coordinates of several lung nodule center points are obtained as follows: The center point of the coarse segmentation image of the lung nodule in the lung nodule identification result is used as the center point of the lung nodule.

[0028] The beneficial effect of the above-mentioned further scheme is that it prepares for the subsequent acquisition of lung nodule segmentation contours.

[0029] Furthermore, the process of obtaining the segmentation contour of the lung nodule specifically involves: Based on the coordinates of the center point of the lung nodule, the region of interest is extracted from the coarse segmentation image of the lung nodule. The region of interest is continuously adjusted until the difference between the gray values ​​at the edge of the region of interest and the gray values ​​inside the region of interest is greater than the gray value threshold, thus obtaining the final region of interest. Binarization of the region of interest yields a rough outline of the lung nodule. The rough outline of the lung nodules is smoothed to obtain the segmentation outline of the lung nodules.

[0030] The beneficial effects of the above-mentioned further scheme are: adjusting the region of interest using grayscale values ​​to minimize the nodule outline and improve segmentation accuracy.

[0031] Furthermore, the objective function of the trained lung nodule segmentation model is:

[0032]

[0033] in, The objective function is... The loss function; The coefficient represents the extent of lung parenchyma expansion. The number of pixels beyond which the predicted lung parenchyma region exceeds the actual lung parenchyma region after the predicted lung parenchyma region overlaps with the actual lung parenchyma region. The missing value of the lung parenchyma region; The number of missing pixels in the predicted lung parenchyma region after the predicted lung parenchyma region overlaps with the actual lung parenchyma region. The loss function for the lung nodule identification module; The cross-entropy loss function; This is the mean square error loss; is the loss function for the lung nodule segmentation and contour extraction module, and is the Dice loss function.

[0034] The beneficial effects of the above-mentioned further solutions are as follows: the present invention supervises the overall module based on the segmentation accuracy of the lung parenchyma region, the identification accuracy of lung nodules, and the segmentation accuracy of lung nodules, so that the lung nodule segmentation model achieves optimal performance. Attached Figure Description

[0035] Figure 1 This is a flowchart of the method of the present invention.

[0036] Figure 2 This is a structural diagram of the lung nodule segmentation model of the present invention. Detailed Implementation

[0037] The specific embodiments of the present invention are described below to enable those skilled in the art to understand the present invention. However, it should be understood that the present invention is not limited to the scope of the specific embodiments. For those skilled in the art, various changes are obvious as long as they are within the spirit and scope of the present invention as defined and determined by the appended claims. All inventions utilizing the concept of the present invention are protected.

[0038] like Figure 1 As shown, in one embodiment of the present invention, a deep learning-based automatic lung nodule segmentation method includes: Obtain a dataset of lung medical images; Information was annotated on the lung medical image dataset to obtain a lung nodule image dataset; like Figure 2 As shown, a lung nodule segmentation model is constructed. The lung nodule segmentation model includes a lung parenchyma contour feature extraction module for obtaining the contour features of lung parenchyma without nodules, a lung parenchyma contour nodule confirmation module for confirming the nodule status of lung parenchyma based on the contour features of lung parenchyma without nodules, a lung parenchyma extraction module for confirming the lung parenchyma cutting contour based on the nodule status of lung parenchyma and extracting the lung parenchyma image, a lung nodule recognition module for obtaining the lung nodule recognition result based on the lung parenchyma image, a lung nodule center point confirmation module for confirming the coordinates of the lung nodule center point based on the lung nodule recognition result, and a lung nodule segmentation contour extraction module for segmenting lung nodules based on the coordinates of the lung nodule center point. A lung nodule segmentation model was trained using a lung nodule image dataset to obtain the trained lung nodule segmentation model. Medical images of the lungs to be tested are acquired, and a trained lung nodule segmentation model is used to identify and segment lung nodules in the medical images of the lungs to be tested.

[0039] The process of annotating the lung medical image dataset to obtain a lung nodule image dataset specifically involves: The lung medical imaging dataset was divided into a subset of lung medical imaging data with lung nodules and a subset of lung medical imaging data without lung nodules. The outline of lung parenchyma in each lung medical image in a subset of lung medical imaging data without pulmonary nodules is marked. The lung parenchyma outline, lung nodule outline, and lung nodule center point of each lung medical image in the lung medical image data subset labeled with lung nodules; By integrating the annotated subsets of lung medical imaging data containing pulmonary nodules and those without pulmonary nodules, a pulmonary nodule image dataset is obtained:

[0040] in, This is a dataset of lung nodule images; Raw lung medical imaging data; Medical imaging data of the lungs after information annotation; This represents the total number of imaging data for lung nodules.

[0041] In this embodiment, relying on manual annotation to obtain information from the training set can maximize the effectiveness of the training data. In addition, when selecting the dataset, attention should be paid to the balance between image data with and without lung nodules. The difference in the number of images should not be too large; otherwise, the training accuracy error of the subsequent model will be relatively large.

[0042] The obtained trained lung nodule segmentation model is specifically as follows: The lung nodule image dataset is split into a lung nodule image training set and a lung nodule image test set; the ratio of lung medical image data with lung nodules to lung medical image data without lung nodules is the same in the lung nodule image training set and the lung nodule image test set. The lung parenchyma contour feature extraction module was trained using all lung nodule image data without lung nodules in the lung nodule image training set, resulting in a trained lung parenchyma contour feature extraction module. Feature aggregation is performed on the true lung parenchyma contours of all lung parenchyma image data without lung nodules in the lung nodule image training set to obtain the lung parenchyma contour aggregation features without nodules. Obtain current lung nodule images from the lung nodule image training set; The current lung nodule image is input into the lung parenchyma contour nodule confirmation module to obtain the initial lung parenchyma contour of the current lung nodule image, and is compared with the aggregated features of the lung parenchyma contour without nodules to obtain the feature similarity. If the feature similarity is greater than the similarity threshold, there are no nodules in the lung parenchyma contour; otherwise, there are nodules. Input the lung parenchyma contour nodule status of the current lung nodule image into the lung parenchyma extraction module to confirm the lung parenchyma cutting contour of the current lung nodule image and cut out the lung parenchyma image. When there are no nodules in the lung parenchyma contour, input the current lung nodule image into the trained lung parenchyma contour feature extraction module to obtain the lung parenchyma contour of the current lung nodule image, which is used as the lung parenchyma cutting contour of the current lung nodule image. When nodules are present in the lung parenchyma outline, the initial lung parenchyma outline of the current lung nodule image is expanded outward based on a preset expansion ratio to obtain the lung parenchyma cutting outline of the current lung nodule image. Input the lung parenchyma image into the lung nodule recognition module to confirm the number of lung nodules in the lung parenchyma region and obtain the lung nodule recognition result. If the number of lung nodules is 0, the process ends. Otherwise, input the lung nodule recognition result into the lung nodule center point confirmation module to obtain the coordinates of several lung nodule center points. Input the coordinates of the center point of each lung nodule into the lung nodule segmentation contour extraction module to obtain the lung nodule segmentation contour and complete the lung nodule segmentation.

[0043] In this embodiment, the training of the lung parenchyma contour feature extraction module is completed first. The accuracy of the module is continuously adjusted based on the annotation information and predicted contour. The module is trained based on the image data without lung nodules in the lung nodule image training set, and its performance is tested using the image data without lung nodules in the lung nodule image test set.

[0044] In this embodiment, only image data without lung nodules is used to train and test the lung parenchyma contour feature extraction module. After training, the lung parenchyma contour feature extraction module is only used for contour feature extraction without lung nodules.

[0045] In this embodiment, several steps of lung nodule segmentation are integrated into several modules. By using the annotation information in the training set, the training status and accuracy of each stage can be monitored separately, continuously improving the prediction accuracy of each module, thereby maximizing the accuracy of lung nodule segmentation.

[0046] The lung parenchyma contour feature extraction module includes: The U-NET network is used to obtain lung segmentation results from raw lung medical imaging data without pulmonary nodules, as an initial contour; The contour processing unit is used to obtain the binary mask and grayscale image of the initial contour. The curvature feature extraction unit is used to obtain the curvature feature map based on the binary mask and grayscale image of the initial contour, using a dual-branch network.

[0047] in, Curvature feature map; Multi-scale feature maps obtained from spatial branching; For local curvature tensors; A binary mask for the initial contour; The x-coordinate is the pixel coordinate. The vertical coordinate is the pixel coordinate. For the first Weights of edge detection images at each scale; For the first Edge detection images at various scales; For scale total; For scale indexing; Enable grayscale; It is a multi-scale structural element; For scale parameters; For the first Structural elements at various scales; Grayscale closure; For expansion operation; For erosion calculation; This is a grayscale image of the initial outline; The dynamic optimization unit is used to perform feature weighting on the boundary pixels of the binary mask of the initial contour based on the curvature feature map to obtain the contour feature map; the weights of the boundary pixels are:

[0048] in, The weights for the boundary pixels; Centered on the convolution kernel; For the major axis; It is the short axis; Curvature weight; For pixels The curvature tensor; It has a small-sized elliptic kernel; It has a large, round core; The curvature threshold used to determine the kernel size; The correction unit is used to obtain a shallow feature map based on the contour feature map using bilinear interpolation, and to obtain a semantic mask based on the shallow feature map using a 1×1 convolutional layer. It also obtains the feature enhancement of pixels in the semantic mask whose organ boundary probability is greater than the probability threshold, thus obtaining the lung parenchyma contour features of the current lung medical image.

[0049] The initial lung parenchyma contour obtained from the current lung nodule image is specifically as follows: A1. Obtain the grayscale image of the current lung nodule and set the block side length; A2. Divide the grayscale image of the current lung nodule into several image blocks based on the block side length; A3. Calculate the average gray level of each image block; A4. Based on the average gray level of each image patch, feature enhancement is performed on each image patch to obtain several enhanced image patches:

[0050] in, For the first One enhanced image patch; For the first Image block pixels grayscale value; For the first The average gray level of each image patch; For the first Rounds Gray-level gain coefficients for each image block; A5. Combine the enhanced image blocks in their original positions to obtain the lung medical enhanced image for the current round; A6. Increase the block side length, return to step A2, and use the current round of lung medical enhancement image as the grayscale image of the current lung nodule image in the next round. Perform block segmentation based on the block side length until the block side length is equal to the side length of the lung medical image to obtain the final lung medical enhancement image. A7. Perform dilation and erosion operations on the final medically enhanced lung images, and confirm the initial lung parenchyma contour of the current lung nodule image based on image morphology.

[0051] In this embodiment, based on the block side length, the gray value changes are analyzed starting from the smallest size and the largest number of image blocks, and finally the gray value changes are analyzed with the original image size. The features of the lung parenchyma region are enhanced step by step, which can maximize the extraction accuracy of the lung parenchyma region and thus optimize the extraction accuracy of lung nodules.

[0052] The number of lung nodules in the confirmed lung parenchyma region is used to obtain the lung nodule identification result, specifically as follows: The YOLOv8 deep learning model was used to detect and locate lung nodules in the lung parenchyma, and the number of lung nodules and the localization boxes of lung nodules in the lung parenchyma were obtained. Based on a preset contour magnification factor, the lung nodule localization box is magnified, and the magnified lung nodule localization box is cropped to obtain several lung nodule coarse segmentation images. The lung nodule identification results are obtained by integrating the number of lung nodules in the lung parenchyma region and the coarse segmentation images of each lung nodule.

[0053] The coordinates of several lung nodule center points are obtained as follows: The center point of the coarse segmentation image of the lung nodule in the lung nodule identification result is used as the center point of the lung nodule.

[0054] The obtained lung nodule segmentation contour is specifically as follows: Based on the coordinates of the center point of the lung nodule, the region of interest is extracted from the coarse segmentation image of the lung nodule. The region of interest is continuously adjusted until the difference between the gray values ​​at the edge of the region of interest and the gray values ​​inside the region of interest is greater than the gray value threshold, thus obtaining the final region of interest. Binarization of the region of interest yields a rough outline of the lung nodule. The rough outline of the lung nodules is smoothed to obtain the segmentation outline of the lung nodules.

[0055] The objective function of the trained lung nodule segmentation model is:

[0056]

[0057] in, The objective function is... The loss function; The coefficient represents the extent of lung parenchyma expansion. The number of pixels beyond which the predicted lung parenchyma region exceeds the actual lung parenchyma region after the predicted lung parenchyma region overlaps with the actual lung parenchyma region. The missing value of the lung parenchyma region; The number of missing pixels in the predicted lung parenchyma region after the predicted lung parenchyma region overlaps with the actual lung parenchyma region. The loss function for the lung nodule identification module; The cross-entropy loss function; This is the mean square error loss; is the loss function for the lung nodule segmentation and contour extraction module, and is the Dice loss function.

Claims

1. A method for automatic lung nodule segmentation based on deep learning, characterized in that, include: Obtain a dataset of lung medical images; Information was annotated on the lung medical image dataset to obtain a lung nodule image dataset; A lung nodule segmentation model is constructed. The lung nodule segmentation model includes a lung parenchyma contour feature extraction module for obtaining the contour features of lung parenchyma without nodules, a lung parenchyma contour nodule confirmation module for confirming the nodule status of lung parenchyma based on the contour features of lung parenchyma without nodules, a lung parenchyma extraction module for confirming the lung parenchyma cutting contour based on the nodule status of lung parenchyma and extracting the lung parenchyma image, a lung nodule recognition module for obtaining the lung nodule recognition result based on the lung parenchyma image, a lung nodule center point confirmation module for confirming the coordinates of the lung nodule center point based on the lung nodule recognition result, and a lung nodule segmentation contour extraction module for segmenting lung nodules based on the coordinates of the lung nodule center point. A lung nodule segmentation model was trained using a lung nodule image dataset to obtain the trained lung nodule segmentation model. Medical images of the lungs to be tested are acquired, and a trained lung nodule segmentation model is used to identify and segment lung nodules in the medical images of the lungs to be tested.

2. The method for automatic lung nodule segmentation based on deep learning according to claim 1, characterized in that, The process of annotating the lung medical image dataset to obtain a lung nodule image dataset specifically involves: The lung medical imaging dataset was divided into a subset of lung medical imaging data with lung nodules and a subset of lung medical imaging data without lung nodules. The outline of lung parenchyma in each lung medical image in a subset of lung medical imaging data without pulmonary nodules is marked. The lung parenchyma outline, lung nodule outline, and lung nodule center point of each lung medical image in the lung medical image data subset labeled with lung nodules; By integrating the annotated subsets of lung medical imaging data containing pulmonary nodules and those without pulmonary nodules, a pulmonary nodule image dataset is obtained: in, This is a dataset of lung nodule images; Raw lung medical imaging data; Medical imaging data of the lungs after information annotation; This represents the total number of imaging data for lung nodules.

3. The method for automatic lung nodule segmentation based on deep learning according to claim 1, characterized in that, The obtained trained lung nodule segmentation model is specifically as follows: The lung nodule image dataset is split into a lung nodule image training set and a lung nodule image test set; the ratio of lung medical image data with lung nodules to lung medical image data without lung nodules is the same in both the lung nodule image training set and the lung nodule image test set. The lung parenchyma contour feature extraction module was trained using all lung nodule image data without lung nodules in the lung nodule image training set, resulting in a trained lung parenchyma contour feature extraction module. Feature aggregation is performed on the true lung parenchyma contours of all lung parenchyma image data without lung nodules in the lung nodule image training set to obtain the lung parenchyma contour aggregation features without nodules. Obtain current lung nodule images from the lung nodule image training set; The current lung nodule image is input into the lung parenchyma contour nodule confirmation module to obtain the initial lung parenchyma contour of the current lung nodule image, and is compared with the aggregated features of the lung parenchyma contour without nodules to obtain the feature similarity. If the feature similarity is greater than the similarity threshold, there are no nodules in the lung parenchyma contour; otherwise, there are nodules. Input the lung parenchyma contour nodule status of the current lung nodule image into the lung parenchyma extraction module to confirm the lung parenchyma cutting contour of the current lung nodule image and cut out the lung parenchyma image. When there are no nodules in the lung parenchyma contour, input the current lung nodule image into the trained lung parenchyma contour feature extraction module to obtain the lung parenchyma contour of the current lung nodule image, which is used as the lung parenchyma cutting contour of the current lung nodule image. When nodules are present in the lung parenchyma outline, the initial lung parenchyma outline of the current lung nodule image is expanded outward based on a preset expansion ratio to obtain the lung parenchyma cutting outline of the current lung nodule image. Input the lung parenchyma image into the lung nodule recognition module to confirm the number of lung nodules in the lung parenchyma region and obtain the lung nodule recognition result. If the number of lung nodules is 0, the process ends. Otherwise, input the lung nodule recognition result into the lung nodule center point confirmation module to obtain the coordinates of several lung nodule center points. Input the coordinates of the center point of each lung nodule into the lung nodule segmentation contour extraction module to obtain the lung nodule segmentation contour and complete the lung nodule segmentation.

4. The method for automatic lung nodule segmentation based on deep learning according to claim 3, characterized in that, The lung parenchyma contour feature extraction module includes: The U-NET network is used to obtain lung segmentation results from raw lung medical imaging data without pulmonary nodules, as an initial contour; The contour processing unit is used to obtain the binary mask and grayscale image of the initial contour. The curvature feature extraction unit is used to obtain the curvature feature map based on the binary mask and grayscale image of the initial contour, using a dual-branch network. in, Curvature feature map; Multi-scale feature maps obtained from spatial branching; For local curvature tensors; A binary mask for the initial contour; The x-coordinate is the pixel coordinate. The vertical coordinate is the pixel coordinate. For the first Weights of edge detection images at each scale; For the first Edge detection images at various scales; For scale total; For scale indexing; Enable grayscale; It is a multi-scale structural element; For scale parameters; For the first Structural elements at various scales; Grayscale closure; For expansion operation; For erosion calculation; This is a grayscale image of the initial outline; The dynamic optimization unit is used to perform feature weighting on the boundary pixels of the binary mask of the initial contour based on the curvature feature map to obtain the contour feature map; the weights of the boundary pixels are: in, The weights for the boundary pixels; Centered on the convolution kernel; For the major axis; It is the minor axis; Curvature weight; For pixels The curvature tensor; It has a small-sized elliptic kernel; It has a large, round core; The curvature threshold used to determine the kernel size; The correction unit is used to obtain a shallow feature map based on the contour feature map using bilinear interpolation, and to obtain a semantic mask based on the shallow feature map using a 1×1 convolutional layer. It also obtains the feature enhancement of pixels in the semantic mask whose organ boundary probability is greater than the probability threshold, thus obtaining the lung parenchyma contour features of the current lung medical image.

5. The method for automatic lung nodule segmentation based on deep learning according to claim 3, characterized in that, The initial lung parenchyma contour obtained from the current lung nodule image is specifically as follows: A1. Obtain the grayscale image of the current lung nodule and set the block side length; A2. Divide the grayscale image of the current lung nodule into several image blocks based on the block side length; A3. Calculate the average gray level of each image block; A4. Based on the average gray level of each image patch, feature enhancement is performed on each image patch to obtain several enhanced image patches: in, For the first One enhanced image patch; For the first Image block pixels grayscale value; For the first The average gray level of each image patch; For the first Rounds Gray-level gain coefficients for each image block; A5. Combine the enhanced image blocks in their original positions to obtain the lung medical enhanced image for the current round; A6. Increase the block side length, return to step A2, and use the current round of lung medical enhancement image as the grayscale image of the current lung nodule image in the next round. Perform block segmentation based on the block side length until the block side length is equal to the side length of the lung medical image to obtain the final lung medical enhancement image. A7. Perform dilation and erosion operations on the final medically enhanced lung images, and confirm the initial lung parenchyma contour of the current lung nodule image based on image morphology.

6. The method for automatic lung nodule segmentation based on deep learning according to claim 3, characterized in that, The number of lung nodules in the confirmed lung parenchyma region is used to obtain the lung nodule identification result, specifically as follows: The YOLOv8 deep learning model was used to detect and locate lung nodules in the lung parenchyma, and the number of lung nodules and the localization boxes of lung nodules in the lung parenchyma were obtained. Based on a preset contour magnification factor, the lung nodule localization box is magnified, and the magnified lung nodule localization box is cropped to obtain several lung nodule coarse segmentation images. The lung nodule identification results are obtained by integrating the number of lung nodules in the lung parenchyma region and the coarse segmentation images of each lung nodule.

7. The method for automatic lung nodule segmentation based on deep learning according to claim 3, characterized in that, The coordinates of several lung nodule center points are obtained as follows: The center point of the coarse segmentation image of the lung nodule in the lung nodule identification result is used as the center point of the lung nodule.

8. The method for automatic lung nodule segmentation based on deep learning according to claim 3, characterized in that, The obtained lung nodule segmentation contour is specifically as follows: Based on the coordinates of the center point of the lung nodule, the region of interest is extracted from the coarse segmentation image of the lung nodule. The region of interest is continuously adjusted until the difference between the gray values ​​at the edge of the region of interest and the gray values ​​inside the region of interest is greater than the gray value threshold, thus obtaining the final region of interest. Binarization of the region of interest yields a rough outline of the lung nodule. The rough outline of the lung nodules is smoothed to obtain the segmentation outline of the lung nodules.

9. The method for automatic lung nodule segmentation based on deep learning according to claim 1, characterized in that, The objective function of the trained lung nodule segmentation model is: in, The objective function is... The loss function; The coefficient represents the extent of lung parenchyma expansion. The number of pixels beyond which the predicted lung parenchyma region exceeds the actual lung parenchyma region after the predicted lung parenchyma region overlaps with the actual lung parenchyma region. This represents the loss coefficient for the lung parenchyma region. The number of missing pixels in the predicted lung parenchyma region after the predicted lung parenchyma region overlaps with the actual lung parenchyma region. The loss function for the lung nodule identification module; The cross-entropy loss function; This is the mean square error loss; is the loss function for the lung nodule segmentation and contour extraction module, and is the Dice loss function.

Images