An image processing method, device and storage medium

Abstract

The application provides an image processing method, device and storage medium. The method comprises: acquiring a target lung nodule image to be segmented; performing lung nodule component segmentation processing on the target lung nodule image based on a lung nodule segmentation model to obtain a lung nodule component segmentation result corresponding to the target lung nodule image; and the lung nodule segmentation model is obtained by learning and training based on a target sample lung nodule image and a corresponding lung nodule component label. The application can realize segmentation of different components in a lung nodule and improve the accuracy of lung nodule segmentation.

Description

Technical Field

[0001] This application belongs to the field of computer technology, and specifically relates to an image processing method, apparatus and storage medium. Background Technology

[0002] Lung cancer has the highest incidence and mortality rate among malignant tumors, posing the greatest threat to human health. Therefore, lung cancer screening motivated by the early detection of malignant pulmonary nodules is particularly important. Treatment monitoring and pulmonary nodule analysis using computed tomography (CT) images are currently important strategies for early lung cancer identification and improving survival.

[0003] Accurate segmentation of lung nodules from CT images is crucial for image-driven lung cancer analysis. Assessing nodule growth is a key issue in the analysis of CT-detected lung nodules. Rapid nodule growth is associated with malignant lung disease. Precise segmentation allows for quantitative measurement of each nodule's volume, shape, and the major and minor axes of its largest cross-section, facilitating follow-up of indeterminate nodules.

[0004] However, the heterogeneity of lung nodules and the similar visual features between nodules and their surrounding environment make nodule segmentation difficult. Existing techniques for lung nodule image segmentation involve uniform annotation, where the entire lung nodule region is labeled as one class. A segmentation network is then trained on the labeled images to obtain a lung nodule segmentation model. Annotation methods can be as follows: Figure 1 As shown. However, the range of Hounsfield Unit (HU) values, which reflect tissue density, varies too much from ground-glass components to calcified components. For example, the HU value range for ground-glass components can be [-750HU, -200HU], the HU value range for solid components can be [-200HU, 100HU], and the HU value for calcified components is greater than 100HU. Labeling the entire nodule region as one category makes the model less focused on learning, which easily leads to problems such as undersegmentation of ground-glass components (low brightness) and oversegmentation of calcified regions (high brightness) in actual segmentation, resulting in low lung nodule segmentation accuracy. Summary of the Invention

[0005] In order to achieve segmentation of different components in lung nodules and improve the accuracy of lung nodule segmentation, this application proposes an image processing method, apparatus and storage medium.

[0006] On one hand, this application proposes an image processing method, the method comprising:

[0007] Obtain the image of the target lung nodule to be segmented;

[0008] The lung nodule segmentation model is used to segment the lung nodule components of the target lung nodule image to obtain the lung nodule component segmentation result corresponding to the target lung nodule image; the lung nodule segmentation model is learned and trained based on the target sample lung nodule image and its corresponding lung nodule component labels.

[0009] Furthermore, after performing lung nodule component segmentation processing on the target lung nodule image based on the lung nodule segmentation model to obtain the lung nodule component segmentation result corresponding to the target lung nodule image, the method further includes:

[0010] Based on the lung nodule component segmentation results, the lung nodule density classification result corresponding to the target lung nodule image is determined.

[0011] Further, determining the lung nodule density classification result corresponding to the target lung nodule image based on the lung nodule component segmentation result includes:

[0012] When the lung nodule component segmentation result includes a single component, the density type formed by the single component is taken as the lung nodule density classification result;

[0013] When the lung nodule component segmentation result includes multiple components, the mixture of density types formed by each of the multiple components is taken as the lung nodule density classification result; or when the lung nodule component segmentation result includes multiple components, the volume of each of the multiple components is determined based on the lung nodule component segmentation result; and the lung nodule density classification result is determined based on the volume of each component.

[0014] Furthermore, when the lung nodule component segmentation result includes multiple components, the method further includes:

[0015] Based on the segmentation results of the lung nodules, the volume of each component among the multiple components is determined;

[0016] Based on the volume of each component, the proportion of each component in the lung nodule density classification results is determined.

[0017] Furthermore, prior to acquiring the image of the target lung nodule to be segmented, the method further includes:

[0018] Obtain raw medical images of the lungs;

[0019] Based on the location information of lung nodules in the original lung medical image, a lung nodule detection box is determined.

[0020] Further, acquiring the target lung nodule image to be segmented includes:

[0021] Select a target segmentation region from the original lung medical image that corresponds to the lung nodule detection frame. The center of the target segmentation region is the same as the center of the lung nodule detection frame. The size of the target segmentation region is a preset number of times the size of the lung nodule detection frame.

[0022] The size of the target segmentation region is resampled to a preset sampling value;

[0023] The resampled target segmentation region is used as the target lung nodule image.

[0024] Furthermore, the acquisition of the target lung nodule image to be segmented also includes:

[0025] Select a target segmentation region from the original lung medical image that corresponds to the lung nodule detection frame. The center of the target segmentation region is the same as the center of the lung nodule detection frame. The size of the target segmentation region is a preset number of times the size of the lung nodule detection frame.

[0026] The size of the target segmentation region is resampled to a preset sampling value;

[0027] The resampled target segmentation region is normalized under different window widths and window levels to obtain a preset number of normalized target segmentation regions, where the preset number is equal to the number of window widths and window levels.

[0028] The predetermined number of normalized target segmentation regions are used as the target lung nodule image.

[0029] Accordingly, the lung nodule component segmentation processing of the target lung nodule image based on the lung nodule segmentation model to obtain the lung nodule component segmentation result corresponding to the target lung nodule image includes:

[0030] Based on the lung nodule segmentation model, the lung nodule component segmentation process is performed on the preset number of normalized target segmentation regions to obtain the lung nodule component segmentation result corresponding to the target lung nodule image.

[0031] Furthermore, the method further includes the step of obtaining the lung nodule segmentation model, wherein obtaining the lung nodule segmentation model includes:

[0032] Obtain target sample lung nodule images labeled with lung nodule component tags;

[0033] The neural network is trained to segment lung nodule components based on the target sample lung nodule image. During the training process, the model parameters of the neural network are adjusted so that the output of the neural network matches the lung nodule component labels marked in the target sample lung nodule image.

[0034] The neural network corresponding to the current model parameters is used as the lung nodule segmentation model.

[0035] Furthermore, obtaining the target sample lung nodule image labeled with lung nodule component tags includes:

[0036] Based on the location information of lung nodules in the medical images of the lungs, the detection box for lung nodules is determined.

[0037] A target sample segmentation region corresponding to the lung nodule detection frame is selected from the lung medical image of the sample. The center of the target sample segmentation region is the same as the center of the lung nodule detection frame. The size of the target sample segmentation region is a preset number of times the size of the lung nodule detection frame.

[0038] The size of the target sample segmentation region is resampled to a preset sampling value;

[0039] Based on the density type of the lung nodules in the sample, determine at least one sample component included in the segmented region of the target sample after resampling;

[0040] Each sample component in the segmented region of the resampled target sample is labeled with its corresponding lung nodule component label, resulting in a target sample lung nodule image labeled with lung nodule component labels.

[0041] Furthermore, the step of obtaining the target sample lung nodule image labeled with lung nodule component tags also includes:

[0042] Based on the location information of lung nodules in the medical images of the lungs, the detection box for lung nodules is determined.

[0043] A target sample segmentation region corresponding to the lung nodule detection frame is selected from the lung medical image of the sample. The center of the target sample segmentation region is the same as the center of the lung nodule detection frame. The size of the target sample segmentation region is a preset number of times the size of the lung nodule detection frame.

[0044] The size of the target sample segmentation region is resampled to a preset sampling value;

[0045] The resampled target sample segmentation regions are normalized under different window widths and window levels to obtain a preset number of normalized target sample segmentation regions, where the preset number is equal to the number of window widths and window levels.

[0046] The normalized target sample segmentation region corresponding to the preset window width and window level is used as the target normalized sample segmentation region.

[0047] Based on the density type of the lung nodules in the sample, at least one sample component included in the target normalized sample segmentation region is determined;

[0048] For each sample component in the target normalized sample segmentation region, label the lung nodule component corresponding to each sample component to obtain the target normalized sample segmentation region labeled with lung nodule component labels.

[0049] Accordingly, the learning and training of the neural network for lung nodule component segmentation based on the target sample lung nodule image, and adjusting the model parameters of the neural network during the learning and training process until the output of the neural network matches the lung nodule component labels marked in the target sample lung nodule image, includes:

[0050] The neural network is trained to segment lung nodules based on the preset number of normalized target sample segmentation regions. During the training process, the model parameters of the neural network are adjusted so that the output of the neural network matches the lung nodule component labels marked in the target normalized sample segmentation regions.

[0051] On the other hand, this application proposes an image processing apparatus, the apparatus comprising:

[0052] The acquisition module is used to acquire images of the target lung nodules to be segmented;

[0053] The segmentation module is used to perform lung nodule component segmentation processing on the target lung nodule image based on the lung nodule segmentation model to obtain the lung nodule component segmentation result corresponding to the target lung nodule image; the lung nodule segmentation model is learned and trained based on the target sample lung nodule image and its corresponding lung nodule component labels.

[0054] Furthermore, the device may also include a determining module, which is used to determine the lung nodule density classification result corresponding to the target lung nodule image based on the lung nodule component segmentation result.

[0055] Furthermore, the determining module includes:

[0056] The first density classification result determination unit is used to determine the density type formed by the single component when the lung nodule component segmentation result includes a single component;

[0057] The second density classification result determination unit is used to, when the lung nodule component segmentation result includes multiple components, take the mixture result of the density types formed by each of the multiple components as the lung nodule density classification result; or, when the lung nodule component segmentation result includes multiple components, determine the volume of each of the multiple components based on the lung nodule component segmentation result; and determine the lung nodule density classification result based on the volume of each component.

[0058] Furthermore, when the lung nodule component segmentation result includes multiple components, the device may further include:

[0059] A volume determination module is used to determine the volume of each component among the multiple components based on the lung nodule component segmentation results;

[0060] The component proportion determination module is used to determine the component proportion of each component in the lung nodule density classification results based on the volume of each component.

[0061] Furthermore, the device may also include:

[0062] The raw lung medical image acquisition module is used to acquire raw lung medical images;

[0063] The lung nodule detection box determination module is used to determine the lung nodule detection box based on the location information of the lung nodules in the original lung medical image.

[0064] Furthermore, the acquisition module includes:

[0065] The first target segmentation region determination unit is used to select a target segmentation region corresponding to the lung nodule detection frame from the original lung medical image. The center of the target segmentation region is the same as the center of the lung nodule detection frame, and the size of the target segmentation region is a preset number of times the size of the lung nodule detection frame.

[0066] The first resampling unit is used to resample the size of the target segmented region to a preset sampling value;

[0067] The first target lung nodule image determination unit is used to use the resampled target segmentation region as the target lung nodule image.

[0068] Furthermore, the acquisition module also includes:

[0069] The second target segmentation region determination unit is used to select a target segmentation region corresponding to the lung nodule detection frame from the original lung medical image. The center of the target segmentation region is the same as the center of the lung nodule detection frame, and the size of the target segmentation region is a preset number of times the size of the lung nodule detection frame.

[0070] The second resampling unit is used to resample the size of the target segmentation region to a preset sampling value;

[0071] The first normalization processing unit is used to normalize the resampled target segmentation region under different window widths and window levels to obtain a preset number of normalized target segmentation regions, wherein the preset number is equal to the number of window widths and window levels.

[0072] The second target lung nodule image determination unit is used to take the preset number of normalized target segmentation regions as the target lung nodule image.

[0073] Accordingly, the segmentation module is used to perform lung nodule component segmentation processing on the preset number of normalized target segmentation regions based on the lung nodule segmentation model, so as to obtain the lung nodule component segmentation result corresponding to the target lung nodule image.

[0074] Furthermore, the device also includes a lung nodule segmentation model acquisition module, which includes:

[0075] The target sample lung nodule image acquisition module is used to acquire target sample lung nodule images labeled with lung nodule component tags;

[0076] The training module is used to train the neural network to segment lung nodule components based on the target sample lung nodule image. During the training process, the model parameters of the neural network are adjusted so that the output of the neural network matches the lung nodule component labels marked in the target sample lung nodule image.

[0077] The lung nodule segmentation model determination unit is used to use the neural network corresponding to the current model parameters as the lung nodule segmentation model.

[0078] Furthermore, the target sample lung nodule image acquisition module includes:

[0079] The first sample lung nodule detection box determination unit is used to determine the sample lung nodule detection box based on the location information of the sample lung nodule in the sample lung medical image;

[0080] The first target sample segmentation region determination unit is used to select a target sample segmentation region corresponding to the sample lung nodule detection frame from the sample lung medical image. The center of the target sample segmentation region is the same as the center of the sample lung nodule detection frame, and the size of the target sample segmentation region is a preset number of times the size of the sample lung nodule detection frame.

[0081] The third resampling unit is used to resample the size of the target sample segmentation region to a preset sampling value;

[0082] The first sample component determination unit is used to determine at least one sample component included in the target sample segmentation region after resampling, based on the density type of the lung nodules in the sample.

[0083] The first annotation unit is used to annotate each sample component in the segmented region of the resampled target sample with the corresponding lung nodule component label, thereby obtaining the target sample lung nodule image with the lung nodule component label.

[0084] Furthermore, the target sample lung nodule image acquisition module further includes:

[0085] The second sample lung nodule detection box determination unit is used to determine the sample lung nodule detection box based on the location information of the sample lung nodule in the sample lung medical image;

[0086] The second target sample segmentation region determination unit is used to select a target sample segmentation region corresponding to the sample lung nodule detection frame from the sample lung medical image. The center of the target sample segmentation region is the same as the center of the sample lung nodule detection frame, and the size of the target sample segmentation region is a preset number of times the size of the sample lung nodule detection frame.

[0087] The fourth resampling unit is used to resample the size of the target sample segmentation region to a preset sampling value;

[0088] The second normalization processing unit is used to normalize the resampled target sample segmentation region under different window widths and window levels to obtain a preset number of normalized target sample segmentation regions, wherein the preset number is equal to the number of window widths and window levels.

[0089] The target normalized sample segmentation region determination unit is used to take the target sample segmentation region after normalization processing, which corresponds to the preset window width and window level, as the target normalized sample segmentation region.

[0090] The second sample component determination unit is used to determine at least one sample component included in the target normalized sample segmentation region based on the density type of the lung nodules in the sample.

[0091] The second annotation unit is used to annotate each sample component in the target normalized sample segmentation region with the corresponding lung nodule component label, thereby obtaining the target normalized sample segmentation region annotated with lung nodule component labels.

[0092] Accordingly, the training module is used to train the neural network to segment lung nodule components based on the preset number of normalized target sample segmentation regions. During the training process, the model parameters of the neural network are adjusted so that the output of the neural network matches the lung nodule component labels marked in the target normalized sample segmentation regions.

[0093] On the other hand, this application proposes an image processing device, the electronic device including a processor and a memory, the memory storing at least one instruction or at least one program, the at least one instruction or the at least one program being loaded and executed by the processor to implement the image processing method as described above.

[0094] On the other hand, this application proposes a computer-readable storage medium storing at least one instruction or at least one program, which is loaded and executed by a processor to implement the image processing method as described above.

[0095] The image processing method, apparatus, and storage medium proposed in this application use a lung nodule segmentation model trained based on the target sample lung nodule image and its corresponding lung nodule component labels to perform lung nodule component segmentation processing on the target lung nodule image, thereby obtaining the lung nodule component segmentation result corresponding to the target lung nodule image. Since the lung nodule segmentation model is trained based on the lung nodule component labels, that is, during the training process of the lung nodule segmentation model, the entire nodule region is not labeled as one class for training, but different components in the nodule are labeled differently. This allows the trained lung nodule segmentation model to focus on the segmentation of components of different densities, avoiding problems such as undersegmentation of ground glass components (low brightness) and oversegmentation of calcified regions (high brightness), thereby improving the accuracy of lung nodule segmentation. Attached Figure Description

[0096] To more clearly illustrate the technical solutions and advantages in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0097] Figure 1 This is a schematic diagram of a structure in the existing technology that labels the entire lung nodule region as a class.

[0098] Figure 2 This is a flowchart illustrating an image processing method provided in an embodiment of this application.

[0099] Figure 3This is a schematic diagram of a process for obtaining a lung nodule segmentation model provided in an embodiment of this application.

[0100] Figure 4 This is a schematic flowchart of a process for obtaining a target sample lung nodule image labeled with lung nodule component tags, provided in an embodiment of this application.

[0101] Figure 5 This is a schematic diagram of a process in the training of a lung nodule segmentation model provided in an embodiment of this application.

[0102] Figure 6 This is a schematic diagram of the structure of lung nodule images obtained by using the fix box resampling method for lung nodules of different sizes provided in the embodiments of this application.

[0103] Figure 7 This is a schematic diagram showing the structure of different components under different window widths and window levels provided in the embodiments of this application.

[0104] Figure 8 This is a schematic diagram of the annotation process provided in the embodiments of this application.

[0105] Figure 9 This is a schematic flowchart of a lung nodule detection frame provided in an embodiment of this application.

[0106] Figure 10 This is a schematic flowchart of a process for obtaining an image of a target lung nodule to be segmented, provided in an embodiment of this application.

[0107] Figure 11 This is a schematic diagram of a process for segmenting lung nodule components in a target lung nodule image based on a lung nodule segmentation model, provided in an embodiment of this application.

[0108] Figure 12 This is a flowchart illustrating another image processing method provided in an embodiment of this application.

[0109] Figure 13 This is a flowchart illustrating how, based on the lung nodule component segmentation results, the lung density classification result corresponding to the target lung nodule image is determined according to an embodiment of this application.

[0110] Figure 14 This is a schematic diagram of the structure of an image processing device provided in an embodiment of this application.

[0111] Figure 15 This is a schematic diagram of another image processing device provided in an embodiment of this application. Detailed Implementation

[0112] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort should fall within the scope of protection of the present application.

[0113] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or server that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or devices.

[0114] Figure 2 This is a flowchart illustrating an image processing method provided in an embodiment of this application. This specification provides the operational steps of the method as described in the embodiments or flowcharts, but based on conventional or non-inventive methods, more or fewer operational steps may be included. The order of steps listed in the embodiments is merely one possible execution order among many and does not represent the only possible execution order. In actual system or server product execution, the method can be executed sequentially or in parallel (e.g., in a parallel processor or multi-threaded processing environment) as shown in the embodiments or accompanying drawings. Specifically, as follows... Figure 2 As shown, the method may include:

[0115] S101. Obtain the image of the target lung nodule to be segmented.

[0116] In the embodiments of this application, such as Figure 3 As shown, prior to S101, the method may further include a step S001 of obtaining a lung nodule segmentation model, which may include:

[0117] S0011. Obtain the target sample lung nodule image labeled with lung nodule component tags.

[0118] S0013. Based on the target sample lung nodule image, the neural network is trained to segment lung nodule components. During the training process, the model parameters of the neural network are adjusted until the output of the neural network matches the lung nodule component labels marked in the target sample lung nodule image.

[0119] S0015. Use the neural network corresponding to the current model parameters as the lung nodule segmentation model.

[0120] Specifically, such as Figure 4 As shown, S0011 may include:

[0121] S00111. Determine the detection box for lung nodules based on the location information of lung nodules in the medical images of lung samples.

[0122] S00113. Select a target sample segmentation region from the medical image of the lung sample that corresponds to the lung nodule detection frame of the sample, wherein the center of the target sample segmentation region is the same as the center of the lung nodule detection frame of the sample, and the size of the target sample segmentation region is a preset number of times the size of the lung nodule detection frame of the sample.

[0123] S00115. Resample the size of the target sample segmentation region to a preset sampling value.

[0124] S00117. Based on the density type of the lung nodules in the sample, determine at least one sample component included in the target sample segmentation region after resampling.

[0125] S00119. Label each sample component in the segmented region of the resampled target sample with the corresponding lung nodule component label to obtain the target sample lung nodule image labeled with lung nodule component label.

[0126] Because lung nodules vary in size, ranging from 3mm to 50mm, this application proposes a method for extracting lung nodule regions using a fixed box during training to avoid low segmentation accuracy caused by inconsistent nodule sizes. Specifically:

[0127] Figure 5 The image shows a flowchart illustrating the training process of a lung nodule segmentation model. Figure 5As shown, firstly, sample lung medical images are acquired for training. Then, the sample lung nodule detection box is determined according to S00111. In some embodiments, the sample lung nodule detection box can be directly obtained based on the location information of the sample lung nodules contained in the sample lung nodule detection results obtained by detecting lung nodules in the sample lung medical images based on a neural network model. In other embodiments, the location information of the sample lung nodules in the sample lung medical images can also be manually marked, and the sample lung nodule detection box can be determined based on the location information of the sample lung nodules. For example, several marking points can be marked around the sample lung nodules in the sample lung medical images, and the sample lung nodule detection box can be determined by the two-point rule (the upper left and lower right corners of the rectangle).

[0128] Continue as Figure 5 As shown, in S00113, the target sample segmentation region can be selected from the sample lung medical image by a preset number of times (e.g., 2 to 3 times) the size of the sample lung nodule detection box obtained in S00111. Then, according to S00115, the size of the target sample segmentation region is uniformly resampled to a fixed value. For example, the target sample segmentation region is resampled to a size of 96*96*96 so that nodules of different sizes occupy the same proportion in the image. Figure 6 6A and Figure 6 B represents a schematic diagram of the lung nodule structure obtained by using the fix box resampling method for lung nodules of different sizes. Figure 6 As can be seen from the image, after using the fix box resampling method, nodules of different sizes occupy the same proportion in the image.

[0129] Because lung nodules contain components of different densities (solid, ground-glass, calcified), and these different components exhibit varying contrast at different window widths and levels, Figure 7 The diagram shows the structure of different components displayed under different window widths and window levels. Figure 7 A is a schematic diagram of the solid components displayed in the lung window. It can be seen that the solid components have a high contrast under the lung window. Figure 7 B is a schematic diagram of the structure of the ground-glass opacity in the lung window. Figure 7 C is a schematic diagram of the structure of the ground glass component displayed in the ground glass window. It can be seen that the ground glass component has low contrast and appears pale under the window, and is prone to undersegmentation. Adjusting to a window width and window level with higher contrast (i.e., the ground glass window) improves the brightness of the ground glass component. Figure 7 D is a schematic diagram of the structure of calcified components in the lung window. Figure 7E is a schematic diagram of the structure of calcified components displayed in the mediastinal window. It can be seen that calcified components have higher brightness and brighter edges under the lung window. Direct segmentation under the lung window easily leads to oversegmentation. Adjusting to the mediastinal window makes it easier to identify calcified components. Furthermore, if calcified components are connected to blood vessels or solid components, they are difficult to distinguish under the lung window. Therefore, to ensure high contrast for components of different densities, thereby improving the subsequent segmentation performance and accuracy, this embodiment continues as follows after resampling... Figure 5 As shown, the resampled target sample segmentation region can also be normalized under different window widths and window levels to obtain a preset number of normalized target sample segmentation regions, which is the same as the number of window widths and window levels. The purpose of normalization is to control the brightness value of the target sample segmentation region within a specified range, such as controlling the brightness value of the target sample segmentation region within [-1, 1].

[0130] In this embodiment, the process of normalizing the resampled target sample segmentation region under different window widths and window levels to obtain a preset number of normalized target sample segmentation regions can be as follows: Taking different window widths and window levels as lung window (window level: -600, window width: 1500), ground glass window (window level: -750, window width: 800), and mediastinal window (window level: 40, window width: 350) as examples, the resampled target sample segmentation region can be normalized under lung window, ground glass window, and mediastinal window respectively, which is equivalent to performing three normalization processes (these three normalization processes can be performed in parallel). Each normalization process will yield a normalized target sample segmentation region. That is, after a target sample segmentation region is normalized under lung window, ground glass window, and mediastinal window respectively, three normalized target sample segmentation regions can be obtained. By normalizing the resampled target sample segmentation region under different window widths and levels, regardless of the components contained in the target sample segmentation region, there will always be one of the different window widths and levels that is suitable for the components contained in the target sample segmentation region (i.e., the components contained in the target sample segmentation region can always obtain high contrast), which can fully reflect the anatomical content and lesion image manifestations, thereby ensuring that the components in the resampled target sample segmentation region can obtain high contrast, avoiding the defect of low contrast of some components caused by normalization under the same fixed window width and level.

[0131] Among them, window technology is a display technology used in CT examination to observe normal tissues or lesions of different densities. The definition of window includes window width and window level. Window width is the pixel range of the target area in the CT image, and window level is the center CT value of the window width.

[0132] The following example, using the normalization of the segmented region of the resampled target sample at the lung window, further illustrates the normalization process:

[0133] Normalization is performed at the lung window; the window level (WL) can be -400, and the window width (WW) can be 1500.

[0134] Given the window minimum Wmin = -400 - 1500 / 2 = -1150 and Wmax = -400 + 1500 / 2 = 350, the normalization steps can include:

[0135] 1) Set the brightness values ​​greater than Wmax in the segmented region of the resampled target sample to Wmax, and set the brightness values ​​less than Wmin to Wmin;

[0136] 2) Normalize the brightness values ​​in the range [Wmin, Wmax] to [-1, 1], using the following formula:

[0137]

[0138] Where x∈[Wmin,Wmax], mean=WL, std=WW / 2, and y is the normalized brightness value, ranging from [-1,1].

[0139] In this embodiment, the fixed-box resampling method can normalize images of lung nodules of different sizes to a fixed size, ensuring that the target lung nodule occupies the same proportion in the image. This avoids the situation in traditional algorithms where small nodules are too small and large nodules are too large, resulting in low segmentation accuracy.

[0140] In practical applications, since the sample lung medical images are used for training, the location of the sample lung nodules and the density type of the sample lung nodules in the sample lung medical images are known. Therefore, in S00117, at least one sample component included in the resampled target sample segmentation region can be determined based on the density type of the sample lung nodules. Lung nodules generally fall into several density types, including solid nodules, ground-glass nodules, mixed ground-glass nodules, and calcified nodules. Since solid nodules consist of solid components, ground-glass nodules consist of ground-glass components, mixed ground-glass nodules consist of both solid and ground-glass components, and calcified nodules consist of calcified components, the target sample segmentation region can include at least three sample components (solid component, ground-glass component, and calcified component). Therefore, in S00119, each sample component can be labeled with its corresponding lung nodule component label. For example, pixels on all solid components of a solid nodule can be labeled as Label1; pixels on all ground-glass components of a ground-glass nodule can be labeled as Label2; pixels on solid components of a mixed ground-glass nodule can be labeled as Label1, and pixels on ground-glass components as Label2; pixels on calcified components of a calcified nodule can be labeled as Label3; and the background area can be labeled as Label0. A schematic diagram of the labeling process can be shown below. Figure 8 As shown.

[0141] Furthermore, if normalization is performed after resampling at different window widths and window levels, then after S00115, the method may further include:

[0142] The resampled target sample segmentation regions are normalized under different window widths and window levels to obtain a preset number of normalized target sample segmentation regions, where the preset number is equal to the number of window widths and window levels.

[0143] The normalized target sample segmentation region corresponding to the preset window width and window level is used as the target normalized sample segmentation region.

[0144] In one feasible embodiment, since the image is ultimately displayed in the lung window, in order to improve the accuracy of lung nodule segmentation, the preset window width and window level can be the lung window, that is, the target sample segmentation region after lung window normalization is used as the target normalized sample segmentation region.

[0145] Accordingly, S00117 may also include:

[0146] Based on the density type of the lung nodules in the sample, at least one sample component included in the target normalized sample segmentation region is determined.

[0147] Accordingly, S00119 may also include:

[0148] For each sample component in the target normalized sample segmentation region, label the lung nodule component corresponding to each sample component to obtain the target normalized sample segmentation region labeled with lung nodule component labels.

[0149] Assuming that the normalization process is performed under the lung window, ground-glass window, and mediastinal window respectively, although three normalized target sample segmentation regions can be obtained after normalization, since there is only one gold standard for labeling, only one target normalized sample segmentation region labeled with lung nodule component labels can be obtained in S00119.

[0150] Because the HU value varies greatly from ground-glass components to calcified components—for example, the HU value range for ground-glass components can be [-750HU, -200HU], the HU value range for solid components can be [-200HU, 100HU], and the HU value for calcified components is greater than 100HU—compared to the traditional method of labeling the entire nodule region as one class, this embodiment of the application labels different components in the nodule differently. This allows the model to focus on learning the segmentation of components with different densities, avoiding problems such as undersegmentation of ground-glass components (low brightness) and oversegmentation of calcified regions (high brightness) in actual segmentation, thereby improving the segmentation accuracy of lung nodules.

[0151] Continue as Figure 5 As shown, after obtaining the target sample lung nodule image labeled with lung nodule component tags, in steps S0013-S0015, the neural network can be trained to segment lung nodule components based on this target sample lung nodule image to obtain a lung nodule segmentation model. Further, if normalization is performed after resampling under different window widths and levels, then in steps S0013-S0015, a preset number of normalized target sample segmentation regions can be used to train the neural network to segment lung nodule components, thereby obtaining a lung nodule segmentation model. Assuming the normalization is performed under the lung window, ground-glass window, and mediastinal window respectively, then in steps S0013-S0015, the normalized target sample segmentation regions obtained in the aforementioned steps can be used as the three-channel input of the neural network to train the lung nodule segmentation model.

[0152] In this embodiment, different components can be preprocessed using normalization at different window widths and levels, which can fully reflect the anatomical content and lesion imaging manifestations, ensuring that each component achieves high contrast. When normalization is performed under lung window, ground-glass window, and mediastinal window, the segmented region of the normalized target sample can be used as the three-channel input of the neural network during model training to train the neural network for lung nodule component segmentation, thereby obtaining a lung nodule segmentation model with good segmentation performance for different components.

[0153] In this embodiment of the invention, the neural network model during the training process can be a V-Net network, specifically a network structure in which a bottleneck layer is added to the residual module of V-Net, i.e., a VB-Net network structure.

[0154] In the embodiments of this application, such as Figure 9 As shown, prior to S101, the method may further include the step of determining a lung nodule detection frame, which may include:

[0155] S10001. Obtain raw medical images of the lungs.

[0156] S10003. Based on the location information of the lung nodules in the original lung medical image, determine the lung nodule detection box.

[0157] In this embodiment, when performing lung nodule segmentation, it is first necessary to acquire the original lung medical image. This original lung medical image can be an image that has already undergone lung nodule detection, meaning that lung nodule detection results have been determined in the original lung medical image. The location information of the lung nodules can be obtained from the lung nodule detection results, and the lung nodule detection box can be determined based on the location information of the lung nodules. In some embodiments, lung nodule detection can be performed on the original lung medical image using a neural network model to obtain lung nodule detection results. Based on the location information of the sample lung nodules corresponding to the lung nodule detection results, the sample lung nodule detection box can be directly obtained. In other embodiments, the location information of lung nodules in the original lung medical image can also be manually annotated, and the lung nodule detection box can be determined based on the location information of the lung nodules. For example, several annotation points can be marked around the lung nodules in the original lung medical image, and the lung nodule detection box can be determined using the two-point rule (the upper left and lower right corners of the rectangle).

[0158] Accordingly, such as Figure 10 As shown, S101 may include:

[0159] S10101. Select a target segmentation region corresponding to the lung nodule detection frame from the original lung medical image. The center of the target segmentation region is the same as the center of the lung nodule detection frame. The size of the target segmentation region is a preset number of times the size of the lung nodule detection frame.

[0160] S10103. Resample the size of the target segmentation region to a preset sampling value.

[0161] S10105. Use the resampled target segmentation region as the target lung nodule image.

[0162] Figure 11The diagram illustrates a process for segmenting lung nodule components in a target lung nodule image based on a lung nodule segmentation model. Figure 11 As shown, in actual testing, due to the inconsistent size of lung nodules (small nodules can be 3mm, large nodules can be 50mm), the size difference is too large. To avoid the problem of low segmentation accuracy caused by inconsistent lung nodule sizes, this embodiment of the application uses a fixed box to extract the lung nodule region during use. Specifically, as described in S10101, the target segmentation region can be selected from the original lung medical image by a preset multiple (e.g., 2 to 3 times) of the size of the lung nodule detection box obtained in S10003. Then, according to S10103, the size of the target segmentation region is uniformly resampled to a fixed value so that nodules of different sizes occupy the same proportion in the image. For example, the target segmentation region is resampled to a size of 96*96*96.

[0163] In practical applications, to further improve the accuracy of lung nodule component segmentation, after resampling, continue as follows: Figure 11 As shown, after S10103, the target segmented region can be normalized to control the brightness value of the target segmented region within a specified range, such as controlling the brightness value of the target segmented region within [-1, 1]. The specific steps of the normalization process can be: normalizing the resampled target segmented region under different window widths and window levels to obtain a preset number of normalized target segmented regions, wherein the preset number is equal to the number of window widths and window levels.

[0164] In this embodiment, the process of normalizing the resampled target segmentation region under different window widths and window levels to obtain a preset number of normalized target segmentation regions can be as follows: Taking different window widths and window levels as lung window (window level: -600, window width: 1500), ground glass window (window level: -750, window width: 800), and mediastinal window (window level: 40, window width: 350) as examples, the resampled target segmentation region can be normalized under lung window, ground glass window, and mediastinal window respectively, which is equivalent to performing three normalization processes (these three normalization processes can be performed in parallel). Each normalization process will yield a normalized target segmentation region. That is, after a target segmentation region is normalized under lung window, ground glass window, and mediastinal window respectively, three normalized target segmentation regions can be obtained in the end. By normalizing the resampled target segmentation region under different window widths and levels, regardless of the components contained in the target segmentation region, there will always be one of the different window widths and levels that is suitable for the components contained in the target segmentation region (i.e., the components contained in the target segmentation region will always have high contrast), which can fully reflect the anatomical content and lesion image manifestations, thereby ensuring that the components in the resampled target segmentation region can have high contrast, avoiding the defect of low contrast of some components caused by normalization under the same fixed window width and level.

[0165] When considering the process of normalizing the resampled target segmentation region under different window widths and window levels, S10105 may further include:

[0166] The predetermined number of normalized target segmentation regions are used as the target lung nodule image.

[0167] In this embodiment, the fixed-box resampling method can normalize images of lung nodules of different sizes to a fixed size, ensuring that the target lung nodule occupies the same proportion in the image. This avoids the situation in traditional algorithms where small nodules are too small and large nodules are too large, resulting in low segmentation accuracy.

[0168] S103. Perform lung nodule component segmentation processing on the target lung nodule image based on the lung nodule segmentation model to obtain the lung nodule component segmentation result corresponding to the target lung nodule image; the lung nodule segmentation model is obtained by learning and training based on the target sample lung nodule image and its corresponding lung nodule component labels.

[0169] In this embodiment of the application, the following continues... Figure 11As shown, after determining the target lung nodule image, the normalized target lung nodule image can be input into the lung nodule segmentation model for lung nodule component segmentation to obtain the lung nodule component segmentation result. Since the training process involves different labels for different nodule components (i.e., label0 is the background, label1 is the solid component, label2 is the ground-glass component, and label3 is the calcified component), the lung nodule component segmentation result should be at least one of label0, label1, label2, and label3.

[0170] Furthermore, if the normalization process is performed at different window widths and window levels, then S103 may further include:

[0171] Based on the lung nodule segmentation model, the lung nodule component segmentation process is performed on the preset number of normalized target segmentation regions to obtain the lung nodule component segmentation result corresponding to the target lung nodule image.

[0172] In one feasible embodiment, such as Figure 12 As shown, after S103, the method may further include:

[0173] S105. Based on the lung nodule component segmentation results, determine the lung nodule density classification results corresponding to the target lung nodule image.

[0174] Specifically, such as Figure 13 As shown, S105 may include:

[0175] S10501. When the lung nodule component segmentation result includes a single component, the density type formed by the single component shall be used as the lung nodule density classification result.

[0176] S10503. When the lung nodule component segmentation result includes multiple components, the mixture result of the density types formed by each of the multiple components is taken as the lung nodule density classification result; or when the lung nodule component segmentation result includes multiple components, the volume of each of the multiple components is determined based on the lung nodule component segmentation result; and the lung nodule density classification result is determined based on the volume of each component.

[0177] In this embodiment of the application, lung nodule quantitative analysis can be performed based on the lung nodule component segmentation results.

[0178] In a feasible embodiment, the density type of lung nodules can be predicted based on the lung nodule component segmentation results. As described in S10501, if the lung nodule component segmentation results include a single component, the density type formed by the single component is taken as the lung nodule density classification result. For example, if only label1 is included, it is a solid nodule; if only label2 is included, it is a ground-glass nodule; and if only label3 is included, it is a calcified nodule.

[0179] As described in S10503, when the lung nodule component segmentation result includes multiple components, in some embodiments, the mixed result of the density types formed by each of the multiple components can be used as the lung nodule density classification result. For example, a mixed ground-glass nodule containing label1 and label2 can be represented. In other embodiments, if the lung nodule component segmentation result includes label1 and label3, the volumes of label1 and label3 can be determined, and the lung nodule density classification result is determined by the component with the larger volume. If the volume of label1 is greater than the volume of label3, the lung nodule density classification result is a solid nodule; if the volume of label3 is greater than the volume of label1, the lung nodule density classification result is a calcified nodule; if the volume of label1 is equal to the volume of label3, the lung nodule density classification result can be determined by other rules: for example, the average HU value of the lung nodule region can be calculated based on the volumes of label1 and label3. If the average HU value is greater than a preset threshold (e.g., 100), the lung nodule density classification result is determined to be a calcified nodule; otherwise, the lung nodule density classification result is determined to be a solid nodule.

[0180] In this embodiment of the application, after obtaining the lung nodule component segmentation result, the segmentation result can be resampled to the original lung medical image based on the coordinate origin and image sampling interval in the original lung medical image.

[0181] In a feasible embodiment, when the lung nodule component segmentation result includes multiple components, after S105, a quantitative analysis of the component proportion can be performed. Specifically, the method can further include:

[0182] Based on the segmentation results of the lung nodules, the volume of each component among the multiple components is determined;

[0183] Based on the volume of each component, the proportion of each component in the lung nodule density classification results is determined.

[0184] In this embodiment, when the lung nodule component segmentation result includes multiple components, such as solid components and ground-glass components, the volume of the solid components and ground-glass components can be determined based on the lung nodule component segmentation result. Based on the volume of the solid components and ground-glass components, the proportion of solid components and ground-glass components in the mixed ground-glass nodule can be accurately calculated, thereby providing accurate quantitative reference for doctors' clinical research analysis.

[0185] In some feasible embodiments, after obtaining the lung nodule component segmentation results, in addition to predicting density type and calculating component proportion, other lung nodule quantitative analysis processing can be performed, including but not limited to: extracting lung nodule segmentation mask, calculating the maximum lung nodule layer, calculating lung nodule volume, calculating lung nodule HU value, extracting lung nodule contour points, calculating lung nodule major and minor axes, calculating lung nodule solid proportion, calculating lung nodule histogram, and extracting lung nodule radiomics features (such as lung nodule volume, minimum HU value, maximum HU value, average HU value, median HU value, standard deviation, skewness value, kurtosis value, entropy value, quality, etc.).

[0186] In practical applications, the volume of lung nodules can be calculated based on the segmentation mask, using the following formula:

[0187] Volume = volume × n;

[0188] Where Volume represents the volume of the lung nodule segmentation mask, volume represents the volume of each voxel, and n represents the number of voxels with a value of 1 in the mask.

[0189] The formula for calculating the volume of a voxel is as follows:

[0190] Volume=spacing_x×spacing_y×spacing_z;

[0191] Here, spacing_x represents the size of a voxel in the x-axis direction, which can be considered as the length of the voxel; spacing_y represents the size of a voxel in the y-axis direction, which can be considered as the width of the voxel; and spacing_z represents the size of a voxel in the z-axis direction, which can be considered as the height of the voxel.

[0192] For lung nodule contour points, contour points of each layer of the nodule can be extracted based on the lung nodule segmentation results.

[0193] The long axis, short axis, and diameter of lung nodules can be calculated using the following formula:

[0194] 1) Calculation of major diameter:

[0195] The major axis is defined as the distance between the two farthest points within the largest cross-sectional profile. Extract the profile points p1, p2, p3, ..., p of the cross-section. n Calculate any two contour points (p) l1 ,p l2 Distance (length):

[0196]

[0197] The longest distance between two points is taken as the major axis distance of the cross section, and the two contour points with the longest length are taken as the two endpoints of the major axis.

[0198] 2) Calculation of the minor diameter:

[0199] The minor axis can be defined as: the longest perpendicular line segment within the nodule mask is the line drawn perpendicular to the major axis.

[0200] Specifically, after determining the major axis of the cross-section, take each point on the major axis as the origin, and draw a perpendicular line in the direction perpendicular to the major axis. The longest distance of the perpendicular line segment intersecting the nodule boundary point is taken as the minor axis of the cross-section. The two contour points where the longest perpendicular line segment intersects the boundary point are taken as the endpoints of the minor axis (p). s1 ,p s2 ).

[0201] 3) Calculation of average diameter:

[0202] The mean diameter is defined as half the sum of the major and minor diameters of the nodule, calculated using the following formula:

[0203] mean diameter=(long diameter+short diameter) / 2;

[0204] The solid component ratio of lung nodules is defined as the proportion of solid components in the entire nodule area.

[0205] The histogram of lung nodules can be obtained by statistically analyzing the volume of voxels with different HU values ​​within the lung nodules.

[0206] The minimum and maximum HU values ​​can be obtained by calculating the minimum and maximum HU values ​​within the lung nodule.

[0207] The average HU value can be obtained using the following formula:

[0208]

[0209] Where X is the HU value of the voxel, volume is the volume of the voxel, and Total_volume is the volume of the nodule.

[0210] The median HU value of lung nodules can be calculated by taking the HU value that is in the middle of the range of lung nodules and arranging them in order.

[0211] The standard deviation (δ) of the HU value for lung nodules can be obtained using the following formula:

[0212]

[0213] The skewness of the HU value for lung nodules is a measure of the direction and degree of skewness in the distribution of HU values ​​for lung nodules. It can be calculated using the following formula:

[0214]

[0215] For the kurtosis (HU) value of lung nodules, the kurtosis value characterizes the peak value of the probability density distribution curve at the mean. Intuitively, kurtosis reflects the sharpness of the peak, and its calculation formula can be as follows:

[0216]

[0217] The entropy of the HU value of a lung nodule can be calculated using the following formula:

[0218]

[0219] Where, p HU This represents the probability that the voxel represents the current HU value.

[0220] The mass of a lung nodule can be calculated using the following formula:

[0221] Mass=Total_volume*[Average_HU+1000]*0.001.

[0222] like Figure 14 As shown in the illustration, this application also provides an image processing apparatus, which may include:

[0223] The acquisition module 201 can be used to acquire images of the target lung nodules to be segmented.

[0224] In this embodiment of the application, the device may further include:

[0225] The raw lung medical image acquisition module can be used to acquire raw lung medical images;

[0226] The lung nodule detection box determination module can be used to determine the lung nodule detection box based on the location information of the lung nodules in the original lung medical image.

[0227] Furthermore, the acquisition module 201 may further include:

[0228] The first target segmentation region determination unit can be used to select a target segmentation region corresponding to the lung nodule detection frame from the original lung medical image. The center of the target segmentation region is the same as the center of the lung nodule detection frame, and the size of the target segmentation region is a preset number of times the size of the lung nodule detection frame.

[0229] The first resampling unit can be used to resample the size of the target segmentation region to a preset sampling value.

[0230] The first target lung nodule image determination unit can be used to use the resampled target segmentation region as the target lung nodule image.

[0231] In one feasible embodiment, the acquisition module may further include:

[0232] The second target segmentation region determination unit can be used to select a target segmentation region corresponding to the lung nodule detection frame from the original lung medical image. The center of the target segmentation region is the same as the center of the lung nodule detection frame, and the size of the target segmentation region is a preset number of times the size of the lung nodule detection frame.

[0233] The second resampling unit can be used to resample the size of the target segmentation region to a preset sampling value;

[0234] The first normalization processing unit can be used to normalize the resampled target segmentation region under different window widths and window levels to obtain a preset number of normalized target segmentation regions, wherein the preset number is equal to the number of window widths and window levels.

[0235] The second target lung nodule image determination unit can be used to take the preset number of normalized target segmentation regions as the target lung nodule image.

[0236] Accordingly, the segmentation module can be used to perform lung nodule component segmentation processing on the preset number of normalized target segmentation regions based on the lung nodule segmentation model, so as to obtain the lung nodule component segmentation result corresponding to the target lung nodule image.

[0237] The segmentation module 203 can be used to perform lung nodule component segmentation processing on the target lung nodule image based on the lung nodule segmentation model to obtain the lung nodule component segmentation result corresponding to the target lung nodule image; the lung nodule segmentation model is learned and trained based on the target sample lung nodule image and its corresponding lung nodule component labels.

[0238] In one feasible embodiment, such as Figure 15 As shown, the device may further include a determining module 205, which can be used to determine the lung nodule density classification result corresponding to the target lung nodule image based on the lung nodule component segmentation result.

[0239] Furthermore, the determining module 205 may include:

[0240] The first density classification result determination unit can be used to determine the density type formed by the single component as the density classification result of the lung nodule when the lung nodule component segmentation result includes a single component.

[0241] The second density classification result determination unit can be used to, when the lung nodule component segmentation result includes multiple components, take the mixture result of the density types formed by each of the multiple components as the lung nodule density classification result; or, when the lung nodule component segmentation result includes multiple components, determine the volume of each of the multiple components based on the lung nodule component segmentation result; and determine the lung nodule density classification result based on the volume of each component.

[0242] In a feasible embodiment, when the lung nodule component segmentation result includes multiple components, the device may further include:

[0243] The volume determination module can be used to determine the volume of each component among the multiple components based on the lung nodule component segmentation results.

[0244] The component proportion determination module can be used to determine the component proportion of each component in the lung nodule density classification result based on the volume of each component.

[0245] In this embodiment of the application, the device may further include a lung nodule segmentation model acquisition module, which may include:

[0246] The target sample lung nodule image acquisition module can be used to acquire target sample lung nodule images labeled with lung nodule component tags.

[0247] The training module can be used to train a neural network to segment lung nodule components based on the target sample lung nodule image. During the training process, the model parameters of the neural network are adjusted so that the output of the neural network matches the lung nodule component labels marked in the target sample lung nodule image.

[0248] The lung nodule segmentation model determination unit can be used to use the neural network corresponding to the current model parameters as the lung nodule segmentation model.

[0249] Furthermore, the target sample lung nodule image acquisition module may include:

[0250] The first sample lung nodule detection box determination unit can be used to determine the sample lung nodule detection box based on the location information of the sample lung nodule in the sample lung medical image.

[0251] The first target sample segmentation region determination unit can be used to select a target sample segmentation region corresponding to the sample lung nodule detection frame from the sample lung medical image. The center of the target sample segmentation region is the same as the center of the sample lung nodule detection frame, and the size of the target sample segmentation region is a preset number of times the size of the sample lung nodule detection frame.

[0252] The third resampling unit can be used to resample the size of the target sample segmentation region to a preset sampling value.

[0253] The first sample component determination unit can be used to determine at least one sample component included in the target sample segmentation region after resampling, based on the density type of the lung nodules in the sample.

[0254] The first annotation unit can be used to annotate each sample component in the segmented region of the resampled target sample with the corresponding lung nodule component label, thereby obtaining a target sample lung nodule image with labeled lung nodule component labels.

[0255] In a feasible embodiment, the target sample lung nodule image acquisition module may further include:

[0256] The second sample lung nodule detection box determination unit can be used to determine the sample lung nodule detection box based on the location information of the sample lung nodule in the sample lung medical image;

[0257] The second target sample segmentation region determination unit can be used to select a target sample segmentation region corresponding to the sample lung nodule detection frame from the sample lung medical image. The center of the target sample segmentation region is the same as the center of the sample lung nodule detection frame, and the size of the target sample segmentation region is a preset number of times the size of the sample lung nodule detection frame.

[0258] The fourth resampling unit can be used to resample the size of the target sample segmentation region to a preset sampling value;

[0259] The second normalization processing unit can be used to normalize the resampled target sample segmentation region under different window widths and window levels to obtain a preset number of normalized target sample segmentation regions, wherein the preset number is equal to the number of window widths and window levels.

[0260] The target normalized sample segmentation region determination unit can be used to take the target sample segmentation region after normalization processing, which corresponds to the preset window width and window level, as the target normalized sample segmentation region.

[0261] The second sample component determination unit can be used to determine at least one sample component included in the target normalized sample segmentation region based on the density type of the lung nodules in the sample.

[0262] The second annotation unit can be used to annotate each sample component in the target normalized sample segmentation region with the corresponding lung nodule component label, thereby obtaining the target normalized sample segmentation region with the lung nodule component label.

[0263] Accordingly, the training module can also be used to train the neural network to segment lung nodule components based on the preset number of normalized target sample segmentation regions. During the training process, the model parameters of the neural network are adjusted so that the output of the neural network matches the lung nodule component labels marked in the target normalized sample segmentation region.

[0264] It should be noted that the device embodiments provided in this application are based on the same inventive concept as the method embodiments described above.

[0265] This application also provides an image processing device, which includes a processor and a memory. The memory stores at least one instruction or at least one program. The processor loads and executes the at least one instruction or at least one program to implement the image processing method provided in the above method embodiments.

[0266] Embodiments of this application also provide a computer-readable storage medium that can be disposed in a terminal to store at least one instruction or at least one program related to implementing an image processing method in the method embodiments. The at least one instruction or at least one program is loaded and executed by a processor to implement the image processing method provided in the above method embodiments.

[0267] Optionally, in this embodiment, the storage medium may be located at at least one of the multiple network servers in a computer network. Optionally, in this embodiment, the storage medium may include, but is not limited to, various media capable of storing program code, such as USB flash drives, read-only memory (ROM), random access memory (RAM), portable hard drives, magnetic disks, or optical disks.

[0268] The memory described in this application embodiment can be used to store software programs and modules. The processor executes various functional applications and data processing by running the software programs and modules stored in the memory. The memory may mainly include a program storage area and a data storage area. The program storage area may store the operating system, applications required for the functions, etc.; the data storage area may store data created according to the use of the device, etc. In addition, the memory may include high-speed random access memory, and may also include non-volatile memory, such as at least one disk storage device, flash memory device, or other volatile solid-state storage device. Accordingly, the memory may also include a memory controller to provide the processor with access to the memory.

[0269] The image processing method, apparatus, and storage medium provided in this application have the following beneficial effects:

[0270] 1) Due to the inconsistent size of lung nodules, ranging from 3mm to 50mm, the size difference is too large. To avoid the problem of low segmentation accuracy caused by inconsistent lung nodule size, this application embodiment uses the fix box method to extract lung nodule regions during training and testing. This method can resample images of lung nodules of different sizes to a fixed size and make the proportion of the target lung nodule in the image the same, thereby improving the accuracy of lung nodule component segmentation.

[0271] 2) To further improve the segmentation performance of the lung nodule segmentation model and the accuracy of lung nodule component segmentation, after resampling, this embodiment can also perform normalization processing on the target sample segmentation region under different window widths and window levels to control the brightness value of the target sample segmentation region within a specified range. Normalizing the target sample segmentation region under different window widths and window levels allows for processing of different components using different window widths and window levels, which can fully reflect the anatomical content and lesion image manifestations. This ensures that the components in the resampled target sample segmentation region can obtain high contrast, avoiding the defect of low contrast for some components caused by normalization processing under the same fixed window width and window level, thereby improving the subsequent segmentation performance and accuracy of components with different densities.

[0272] 3) The training process in this embodiment employs an improved V-Net neural network (i.e., the VB-Net network structure). By using the VB-Net network structure, the number of convolutional kernel parameters is reduced while maintaining the same tensor dimension for input and output, and ensuring no reduction in model accuracy. Ultimately, without sacrificing segmentation accuracy, the size of the deep learning V-Net model can be compressed from the original 250MB to 8.8MB. After network size compression, the model loads faster, and the segmentation network can be easily deployed not only on high-end devices but also on mobile devices.

[0273] 4) Based on the lung nodule segmentation results, this application embodiment can perform various quantitative analyses, including but not limited to predicting the density type of lung nodules and accurately calculating the proportion of solid and ground-glass components in mixed ground-glass nodules based on the volume of solid and ground-glass components, thereby providing accurate quantitative references for doctors' clinical research and analysis.

[0274] It should be noted that the order of the embodiments described above is merely for descriptive purposes and does not represent the superiority or inferiority of the embodiments. Furthermore, specific embodiments have been described above. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps described in the claims can be performed in a different order than that shown in the embodiments and still achieve the desired result. Additionally, the processes depicted in the drawings do not necessarily require a specific or sequential order to achieve the desired result. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0275] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the device and server embodiments are basically similar to the method embodiments, so the descriptions are relatively simple; relevant parts can be referred to the descriptions of the method embodiments.

[0276] Those skilled in the art will understand that all or part of the steps of the above embodiments can be implemented by hardware or by a program instructing related hardware. The program can be stored in a computer-readable storage medium, such as a read-only memory, a disk, or an optical disk.

[0277] The above description is only a preferred embodiment of this application and is not intended to limit this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application.

Claims

1. An image processing method, characterized by, The method comprises: obtaining a target lung nodule image to be segmented; performing lung nodule component segmentation processing on the target lung nodule image based on a lung nodule segmentation model to obtain a lung nodule component segmentation result corresponding to the target lung nodule image; the lung nodule segmentation model is obtained based on target sample lung nodule images and lung nodule component labels corresponding to different density component regions in the target sample lung nodule images; and the lung nodule component segmentation result comprises at least one of a solid component, a ground glass component, a calcification component, and a background; based on the lung nodule component segmentation result, determining a lung nodule density classification result corresponding to the target lung nodule image, comprising: when the lung nodule component segmentation result comprises multiple components, determining the volume of each component in the multiple components based on the lung nodule component segmentation result; and determining the lung nodule density classification result based on the volume of each component; the lung nodule density classification result comprises a solid nodule, a ground glass nodule, a mixed ground glass nodule, and a calcification nodule.

2. The method of claim 1, wherein, The method further comprises: when the lung nodule component segmentation result comprises a single component, the density type formed by the single component is taken as the lung nodule density classification result; when the lung nodule component segmentation result comprises the multiple components, the mixed result of the density types formed by each component in the multiple components is taken as the lung nodule density classification result.

3. The method of claim 1, wherein, When the lung nodule component segmentation result comprises multiple components, the method further comprises: determining the volume of each component in the multiple components based on the lung nodule component segmentation result; determining the component proportion of each component in the lung nodule density classification result based on the volume of each component.

4. The method of claim 1, wherein, Before the target lung nodule image to be segmented is obtained, the method further comprises: obtaining an original lung medical image; determining a lung nodule detection box based on the position information of the lung nodule in the original lung medical image.

5. The method of claim 4, wherein, The target segmentation region corresponding to the lung nodule detection box is selected from the original lung medical image, the center of the target segmentation region is the same as the center of the lung nodule detection box, and the size of the target segmentation region is a preset number times the size of the lung nodule detection box; the size of the target segmentation region is resampled to a preset sampling value; the resampled target segmentation region is taken as the target lung nodule image. The method further comprises the step of obtaining the lung nodule segmentation model, and the obtaining of the lung nodule segmentation model comprises:

6. The method of claim 1, wherein, obtaining a target sample lung nodule image labeled with a lung nodule component label; performing lung nodule component segmentation learning training on a neural network based on the target sample lung nodule image, and adjusting the model parameters of the neural network in the learning training process until the result output by the neural network matches the lung nodule component label labeled in the target sample lung nodule image; the neural network corresponding to the current model parameters is taken as the lung nodule segmentation model. ​ 7. The method of claim 6, wherein, The target sample lung nodule image labeled with the lung nodule component label comprises: Determine a sample lung nodule bounding box based on position information of a sample lung nodule in a sample lung medical image; Select a target sample segmentation region corresponding to the sample lung nodule bounding box from the sample lung medical image, the center of the target sample segmentation region being the same as the center of the sample lung nodule bounding box, and the size of the target sample segmentation region being a preset number of times the size of the sample lung nodule bounding box; and resample the size of the target sample segmentation region to a preset sampling value; Determine at least one sample component included in the resampled target sample segmentation region based on the density type of the sample lung nodule; Label each sample component in the resampled target sample segmentation region with a lung nodule component label corresponding to each sample component, to obtain a target sample lung nodule image labeled with a lung nodule component label.

8. An image processing apparatus characterized by comprising: The device comprises: An acquisition module configured to acquire a target lung nodule image to be segmented; A segmentation module configured to perform lung nodule component segmentation processing on the target lung nodule image based on a lung nodule segmentation model, to obtain a lung nodule component segmentation result corresponding to the target lung nodule image; the lung nodule segmentation model is obtained by learning and training based on a target sample lung nodule image and lung nodule component labels corresponding to different density component regions in the target sample lung nodule image; and the lung nodule component segmentation result comprises at least one of a solid component, a ground glass component, a calcification component, and a background; A determination module configured to determine a lung nodule density classification result corresponding to the target lung nodule image based on the lung nodule component segmentation result; the lung nodule density classification result comprises a solid nodule, a ground glass nodule, a mixed ground glass nodule, and a calcification nodule. The determination module comprises a second density classification result determination unit configured to, when the lung nodule component segmentation result comprises multiple components, determine the volume of each component in the multiple components based on the lung nodule component segmentation result, and determine the lung nodule density classification result based on the volume of each component.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores at least one instruction or at least one program, which is loaded and executed by the processor to implement the image processing method of any one of claims 1-7.

10. An image processing apparatus characterized by comprising: The image processing device comprises a processor and a memory, and the memory stores at least one instruction, which is loaded and executed by the processor to implement the image processing method of any one of claims 1-7.

Images