Image segmentation model training method, image processing method and device, and equipment

Abstract

The present invention discloses a training method, an image processing method, a device and an apparatus for an image segmentation model. The training method for the image segmentation model includes: obtaining a training sample image set; inputting a lung nodule sample image into a preset segmentation model to predict a solid component area, and obtaining a predicted segmentation result; calculating a first loss of pixels in the solid component area, a second loss of pixels in the blood vessel area, and a third loss of pixels in the background area; adjusting the parameters of the preset segmentation model until the convergence condition is met, and obtaining a trained image segmentation model. The present invention adds a second mask image for indicating the blood vessel area in the lung nodule sample image to the training sample image set, so that in the process of training the image segmentation model, a penalty for predicting the blood vessel area as a solid component area is added, thereby avoiding the image segmentation model from segmenting the blood vessel area into a solid component area, and improving the accuracy of the solid component area segmentation.

Description

Technical Field

[0001] The present invention relates to the field of medical image processing, and in particular to a training method for an image segmentation model, an image processing method and device, an electronic device and a storage medium. Background Art

[0002] Mixed ground-glass nodules, also known as partially solid nodules, mainly refer to ground-glass nodules that contain both ground-glass components and solid components. The mixture of the two is called a mixed ground-glass nodule. The characteristics of the solid components in mixed ground-glass nodules are correlated with the pathological classification of carcinoma in situ, microinvasive adenocarcinoma, and invasive adenocarcinoma. The larger the lesion, the more solid components, the greater the proportion of solid components, or the higher the average solid CT (Computed Tomography, computer tomography) value, the greater the possibility that the nodule is invasive cancer. Therefore, a comprehensive evaluation of the CT characteristics of the nodule is helpful for preoperative diagnosis and guiding clinical treatment.

[0003] At present, the method for evaluating the characteristics of solid components in mixed ground-glass nodules is to manually adjust the HU (Hounsfield Unit) value within the segmented lung nodule range, segment the solid components by threshold, and then calculate the proportion of solid components and the average solid CT value. Although this method can segment the solid components to a certain extent, different people adjust the threshold, and the segmentation results obtained are different, and the blood vessels passing through the mixed ground-glass nodules will also be mistakenly segmented as solid components. In addition, this method is a semi-automatic segmentation method, which is time-consuming and labor-intensive. Summary of the invention

[0004] The technical problem to be solved by the present invention is to overcome the defect in the prior art that blood vessels in mixed ground glass nodules are easily segmented into solid components, resulting in inaccurate segmentation and low segmentation efficiency, and to provide a training method for an image segmentation model, an image processing method and device, an electronic device and a storage medium.

[0005] The present invention solves the above technical problems through the following technical solutions:

[0006] A first aspect of the present invention provides a method for training an image segmentation model, comprising:

[0007] Acquire a training sample image set; the training sample image set includes a lung nodule sample image, a first mask image, and a second mask image, wherein the first mask image is used to indicate a solid component area and a non-solid component area in the lung nodule sample image, and the second mask image is used to indicate a blood vessel area in the lung nodule sample image;

[0008] Inputting the pulmonary nodule sample image into a preset segmentation model to predict the solid component area, and obtaining a predicted segmentation result;

[0009] Calculate, according to the predicted segmentation result, the first mask image and the second mask image, a first loss of pixels in the solid component area, a second loss of pixels in the blood vessel area and a third loss of pixels in the background area; wherein the background area is an area other than the blood vessel area in the non-solid component area;

[0010] The parameters of the preset segmentation model are adjusted according to the first loss, the second loss and the third loss until a convergence condition is met to obtain a trained image segmentation model.

[0011] Optionally, adjusting the parameters of the preset segmentation model according to the first loss, the second loss, and the third loss specifically includes:

[0012] Performing a weighted summation on the first loss, the second loss and the third loss to obtain a target loss; wherein the weight of the second loss is greater than the weight of the third loss;

[0013] The parameters of the preset segmentation model are adjusted according to the target loss.

[0014] A second aspect of the present invention provides an image processing method, comprising:

[0015] Acquire a lung nodule image to be segmented;

[0016] The lung nodule image is input into an image segmentation model to perform segmentation processing on the solid component area to obtain a first segmentation result; wherein the image segmentation model is trained based on the training method described in the first aspect.

[0017] Optionally, after acquiring the pulmonary nodule image to be segmented, the method further includes:

[0018] Inputting the lung nodule image into a nodule segmentation model to perform segmentation processing on the nodule region to obtain a second segmentation result; wherein the nodule segmentation model is obtained based on training samples;

[0019] The solid component features in the pulmonary nodule image are analyzed according to the first segmentation result and the second segmentation result.

[0020] Optionally, analyzing the solid component features in the pulmonary nodule image according to the first segmentation result and the second segmentation result includes:

[0021] If the major diameter of the nodule in the second segmentation result is greater than a preset value, the solid component in the pulmonary nodule image is analyzed.

[0022] Optionally, the analyzing the solid components in the pulmonary nodule image includes:

[0023] Determine a first volume of a solid component region in the pulmonary nodule image according to the first segmentation result;

[0024] Determine a second volume of the nodule area in the pulmonary nodule image according to the second segmentation result;

[0025] The proportion of the solid component in the nodule is calculated based on the first volume and the second volume.

[0026] Optionally, the analyzing the solid components in the pulmonary nodule image includes: extracting the contour line of each cross section according to the first segmentation result.

[0027] Optionally, the analyzing the solid components in the pulmonary nodule image includes:

[0028] Extracting the largest connected domain of the first segmentation result;

[0029] Determine the maximum cross section according to the segmented area of ​​each cross section in the maximum connected domain;

[0030] Extracting contour points of the maximum cross section;

[0031] The long diameter of the solid component area is determined based on the two contour points with the farthest distance from each other.

[0032] A third aspect of the present invention provides an image processing device, comprising:

[0033] An image acquisition module, used for acquiring an image of a lung nodule to be segmented;

[0034] An image processing module is used to input the lung nodule image into an image segmentation model to perform segmentation processing on the solid component area to obtain a first segmentation result; wherein the image segmentation model is trained based on the training method described in the first aspect.

[0035] The fourth aspect of the present invention provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the training method for the image segmentation model described in the first aspect or the image processing method described in the second aspect is implemented.

[0036] The fifth aspect of the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the image segmentation model training method described in the first aspect or the image processing method described in the second aspect.

[0037] The positive progressive effect of the present invention is that by adding a second mask image for indicating the vascular area in the lung nodule sample image to the training sample image set, a penalty for predicting the vascular area as a solid component area is added in the process of training the image segmentation model, thereby avoiding the image segmentation model from segmenting the vascular area into a solid component area, thereby improving the accuracy of the solid component area segmentation. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] Figure 1 A flowchart of a method for training an image segmentation model provided in Example 1 of the present invention.

[0039] Figure 2 A regional schematic diagram of a lung nodule sample image provided in Example 1 of the present invention.

[0040] Figure 3 A flow chart of step S14 provided in embodiment 1 of the present invention.

[0041] Figure 4 A schematic diagram of training an image segmentation model provided in Example 1 of the present invention.

[0042] Figure 5 This is a structural block diagram of a training device for an image segmentation model provided in Example 1 of the present invention.

[0043] Figure 6 This is a flowchart of an image processing method provided in Example 2 of the present invention.

[0044] Figure 7 This is a flowchart of another image processing method provided in Example 2 of the present invention.

[0045] Figure 8 A schematic flow chart of another image processing method provided in Embodiment 2 of the present invention.

[0046] Fig. 9 A flow chart for calculating the major diameter of a solid component region provided in Example 2 of the present invention.

[0047] Fig.10 A flow chart for calculating the proportion of solid components in a nodule provided in Example 2 of the present invention.

[0048] Fig.11a This is a display interface diagram of a lung nodule provided in Example 2 of the present invention.

[0049] Fig.11b This is a display interface diagram of a lung nodule provided in Example 2 of the present invention.

[0050] Fig.11c This is a display interface diagram of a lung nodule provided in Example 2 of the present invention.

[0051] Fig.12 This is another display interface diagram of lung nodules provided in Example 2 of the present invention.

[0052] Fig.13 This is a structural block diagram of an image processing device provided in Example 2 of the present invention.

[0053] Fig.14 A schematic diagram of the structure of an electronic device provided in Example 3 of the present invention. DETAILED DESCRIPTION

[0054] The present invention is further described below by way of examples, but the present invention is not limited to the scope of the examples.

[0055] Example 1

[0056] Figure 1 A flowchart of a method for training an image segmentation model provided in the present embodiment. The method for training an image segmentation model can be executed by a training device for an image segmentation model. The training device for an image segmentation model can be implemented by software and / or hardware. The training device for an image segmentation model can be part or all of an electronic device. The electronic device in the present embodiment can be a personal computer (PC), such as a desktop computer, an all-in-one computer, a laptop computer, a tablet computer, etc. It can also be a terminal device such as a mobile phone, a wearable device, a PDA (Personal Digital Assistant), etc. The following introduces the method for training an image segmentation model provided in the present embodiment with the electronic device as the execution subject.

[0057] like Figure 1 As shown, the training method of the image segmentation model provided in this embodiment may include the following steps S11 to S14:

[0058] Step S11, obtaining a training sample image set. The training sample image set includes a lung nodule sample image, a first mask image, and a second mask image. The first mask image is a gold standard, which is used to indicate the solid component area and the non-solid component area in the lung nodule sample image, and the second mask image is used to indicate the blood vessel area in the lung nodule sample image.

[0059] In this embodiment, the nodules in the lung nodule sample image are mixed ground-glass nodules, that is, partially solid nodules, including solid component areas and ground-glass component areas.

[0060] Since lung nodules vary in size, if a fixed-length method is used to crop lung nodule sample images, small nodules will be too small to be easily found in the lung nodule sample images, and large nodules will be too large to fit in the lung nodule sample images. To avoid this problem, a fixed-box method is used to obtain lung nodule sample images. Specifically, with the nodule detection frame as a reference, the lung nodules are cropped from the original lung sample medical image at a preset number of times the size of the detection frame, such as 2.5 times, and the cropped lung nodule sample images are resampled to a fixed size, such as 96×96×96, to ensure that nodules of different sizes account for the same proportion in the lung nodule sample images.

[0061] In order to improve the accuracy of the segmentation of the solid component area of ​​the pulmonary nodule, after the above resampling, the pulmonary nodule sample image can also be normalized to control the grayscale distribution of the pulmonary nodule sample image within a specified range, such as [-1, 1]. In a specific example, the resampled pulmonary nodule sample image is normalized under the lung window (window width: 1500, window level: -400).

[0062] Step S12: Input the pulmonary nodule sample image into a preset segmentation model to predict the solid component area and obtain a predicted segmentation result. In a specific implementation, the pulmonary nodule sample image after resampling and normalization can be input into the preset segmentation model.

[0063] The preset segmentation model is a two-class segmentation model, and the segmentation result is a solid component area or a non-solid component area. The preset segmentation model can be a V-Net network or a VB-Net network. A VB-Net network can be obtained by adding a bottleneck layer to the residual module in the V-Net network.

[0064] Step S13: Calculate the first loss of pixels in the solid component area, the second loss of pixels in the blood vessel area, and the third loss of pixels in the background area according to the predicted segmentation result, the first mask image, and the second mask image, wherein the background area is the area of ​​the non-solid component area excluding the blood vessel area.

[0065] In such Figure 2 In the pulmonary nodule sample image shown, the black part represents the background area, the white part represents the solid component area, and the gray part represents the blood vessel area. The background area, the solid component area, and the blood vessel area together constitute the pulmonary nodule sample image.

[0066] In the specific implementation of step S13, for pixels whose gold standard belongs to the solid component area, the corresponding pixel prediction probability in the predicted segmentation result is substituted into the loss function to calculate the first loss; for pixels whose gold standard is in the non-solid component area, it is determined whether they are in the vascular area according to the second mask image. If they are in the vascular area, the corresponding pixel prediction probability in the predicted segmentation result is substituted into the loss function to calculate the second loss; if they are not in the vascular area, it means that they are in the background area, and the corresponding pixel prediction probability in the predicted segmentation result is substituted into the loss function to calculate the third loss.

[0067] In the formula of the loss function, is the pixel label, is the pixel prediction probability, i.e. the prediction result of the pixel, is the weight coefficient. Used to indicate that a pixel is within a solid component area or within a non-solid component area. indicates that it is in the solid component area, then the corresponding The loss function calculates the first loss if It indicates that the pixel is in the non-solid component area. It is also necessary to determine whether the pixel in the non-solid component area is in the blood vessel area according to the second mask image. If it is in the blood vessel area, the corresponding Substitute the loss function to calculate the second loss. If it is not in the blood vessel area, it means it is in the background area. Substitute into the loss function to calculate the third loss.

[0068] Taking the cross entropy loss (CE loss) as an example, the specific formula is as follows:

[0069]

[0070] In this example, is the function for calculating the first loss, is the function for calculating the second loss, is the function for calculating the third loss.

[0071] Taking the Focal loss function as an example, the specific formula is as follows:

[0072]

[0073] In this example, is the function for calculating the first loss, is the function for calculating the second loss, To calculate the third loss function, It is a regulation factor used to adjust the rate at which the weight of simple samples is reduced.

[0074] Step S14: adjust the parameters of the preset segmentation model according to the first loss, the second loss and the third loss until a convergence condition is met to obtain a trained image segmentation model.

[0075] In this embodiment, by adding a second mask image for indicating the vascular area in the lung nodule sample image to the training sample image set, a penalty for predicting the vascular area as a solid component area is added in the process of training the image segmentation model, thereby avoiding the image segmentation model from segmenting the vascular area into a solid component area, thereby improving the accuracy of the solid component area segmentation.

[0076] In an optional embodiment, Figure 3 As shown, step S14 specifically includes steps S141-S142:

[0077] Step S141: Perform a weighted summation on the first loss, the second loss and the third loss to obtain a target loss, wherein the weight of the second loss is greater than the weight of the third loss.

[0078] Step S142: adjusting the parameters of the preset segmentation model according to the target loss. Specifically, adjusting the parameters of the preset segmentation model until a convergence condition is satisfied, and the preset segmentation model satisfying the convergence condition is a trained image segmentation model.

[0079] In this embodiment, the weight of the second loss is set to be greater than the weight of the third loss, so that the penalty for pixels in the vascular area predicted to be solid component areas is increased, further avoiding the image segmentation model from segmenting the vascular area into solid component areas, thereby further improving the accuracy of solid component area segmentation.

[0080] Figure 4 A training diagram for illustrating an image segmentation model. Figure 4 As shown, the lung nodule sample images in the training sample image set are resampled and normalized in turn, and the processed lung nodule sample images are input into the preset segmentation model. The loss is calculated based on the output predicted segmentation results and the first mask image and the second mask image as the gold standard, and the parameters of the preset segmentation model are adjusted according to the loss until the convergence condition is met to obtain a trained image segmentation model. In this example, the preset segmentation model adopts the VB-Net network structure.

[0081] This embodiment also provides a training device 40 for an image segmentation model, such as Figure 5 As shown, it includes a sample acquisition module 41, a prediction module 42, a loss calculation module 43 and a parameter adjustment module 44.

[0082] The sample acquisition module 41 is used to acquire a training sample image set. The training sample image set includes a pulmonary nodule sample image, a first mask image, and a second mask image, wherein the first mask image is used to indicate a solid component area and a non-solid component area in the pulmonary nodule sample image, and the second mask image is used to indicate a blood vessel area in the pulmonary nodule sample image.

[0083] The prediction module 42 is used to input the lung nodule sample image into a preset segmentation model to perform segmentation processing on the solid component area to obtain a predicted segmentation result.

[0084] The loss calculation module 43 is used to calculate the first loss of pixels in the solid component area, the second loss of pixels in the blood vessel area, and the third loss of pixels in the background area based on the predicted segmentation result, the first mask image, and the second mask image; wherein the background area is the area excluding the blood vessel area in the non-solid component area.

[0085] The parameter adjustment module 44 is used to adjust the parameters of the preset segmentation model according to the first loss, the second loss and the third loss until the convergence condition is met to obtain a trained image segmentation model.

[0086] In an optional embodiment, the parameter adjustment module specifically includes a weighting unit and an adjustment unit. The weighting unit is used to perform a weighted summation of the first loss, the second loss and the third loss to obtain a target loss. The weight of the second loss is greater than the weight of the third loss. The adjustment unit is used to adjust the parameters of the preset segmentation model according to the target loss.

[0087] It should be noted that the training device for the image segmentation model in this embodiment may specifically be a separate chip, a chip module or an electronic device, or may be a chip or a chip module integrated into an electronic device.

[0088] The various modules / units included in the training device of the image segmentation model described in this embodiment may be software modules / units or hardware modules / units, or may be partly software modules / units and partly hardware modules / units.

[0089] Example 2

[0090] Figure 6The present invention provides a flowchart of an image processing method provided in this embodiment. The image processing method can be executed by an image processing device, which can be implemented by software and / or hardware, and the image processing device can be part or all of an electronic device. The electronic device in this embodiment can be a personal computer, such as a desktop computer, an all-in-one computer, a laptop computer, a tablet computer, etc., and can also be a terminal device such as a mobile phone, a wearable device, a handheld computer, etc. The image processing method provided in this embodiment is introduced below with the electronic device as the execution subject.

[0091] like Figure 6 As shown, the image processing method provided in this embodiment may include the following steps S21-S22:

[0092] Step S21, obtaining a lung nodule image to be segmented.

[0093] The pulmonary nodule image to be segmented can be obtained based on the original lung medical image. For example, the original lung medical image is subjected to pulmonary nodule detection processing, and when the pulmonary nodules are detected, the original lung medical image can be cropped to obtain the pulmonary nodule image. The original lung medical image can be obtained by scanning the patient's lungs using a CT device, or by downloading from the cloud or a server. In addition, the pulmonary nodule image to be segmented can also be downloaded from the cloud or a server.

[0094] Step S22: Input the pulmonary nodule image into an image segmentation model to perform segmentation processing on the solid component region to obtain a first segmentation result. The image segmentation model is trained based on the training method described in Example 1.

[0095] In this embodiment, since a penalty for predicting a vascular region as a solid component region is added to the training process of the image segmentation model, in the process of using the image segmentation model to perform solid component region segmentation on the lung nodule image, it is possible to avoid segmenting the vascular region into a solid component region, thereby improving the accuracy of the segmentation of the solid component region in the lung nodule image.

[0096] like Figure 7 and 8 As shown, the image processing method provided in this embodiment may include the following steps S31-S34:

[0097] Step S31, obtaining a lung nodule image to be segmented. The specific implementation is similar to the above-mentioned step S21, and will not be repeated here.

[0098] Step S32: Input the pulmonary nodule image into an image segmentation model to perform segmentation processing on the solid component region to obtain a first segmentation result. The image segmentation model is trained based on the training method described in Example 1.

[0099] Step S33: input the lung nodule image into a nodule segmentation model to perform segmentation processing on the nodule area to obtain a second segmentation result.

[0100] The nodule segmentation model is trained based on training samples, the training samples include sample images and corresponding labels, and the labels include pulmonary nodule areas and non-pulmonary nodule areas. The nodule segmentation model is used to segment the nodule area of ​​the pulmonary nodule image to obtain a segmentation result of the pulmonary nodule area, i.e., a second segmentation result.

[0101] Step S34: analyzing the solid component features in the pulmonary nodule image according to the first segmentation result and the second segmentation result.

[0102] In this embodiment, the lung nodule image is processed using a nodule segmentation model and an image segmentation model respectively, and the segmentation result of the lung nodule area, i.e., the second segmentation result, and the segmentation result of the solid component area in the lung nodule, i.e., the first segmentation result, can be obtained. Based on the lung nodule area and the more accurate solid component area, the solid component characteristics in the lung nodule image can be specifically analyzed, thereby better assisting medical diagnosis.

[0103] In an optional implementation manner, the above step S34 specifically includes the following steps S41:

[0104] Step S41: If the major diameter of the nodule in the second segmentation result is greater than a preset value, the solid component in the pulmonary nodule image is analyzed. The preset value can be set according to actual conditions, for example, it can be set to 6 mm.

[0105] In a specific implementation, the long diameter of the nodule can be determined by the following method: extracting the maximum connected domain of the second segmentation result, determining the maximum cross-section based on the segmentation area of ​​each cross-section within the maximum connected domain, extracting the contour points of the maximum cross-section, and determining the long diameter of the nodule based on the two contour points with the farthest distance.

[0106] In this embodiment, the first segmentation result of the lung nodule image includes a solid component area and a ground glass component area, indicating that the nodule in the lung nodule image is a mixed ground glass nodule, that is, a partially solid nodule. The long diameter of the nodule is determined according to the second segmentation result of the lung nodule image. If the long diameter of the nodule is greater than a preset value, it indicates that the partially solid nodule is more likely to have a lesion such as invasive cancer. At this time, by analyzing the solid component of the partially solid nodule, the doctor can be assisted in further diagnosis of the nodule. Specifically, the solid component of the partially solid nodule can be analyzed in response to the user's operation.

[0107] In the specific implementation of analyzing the solid component in step S41, the average HU value of the solid component can be calculated according to the first segmentation result, so as to better assist medical diagnosis. Specifically, the number of voxels in the solid component region is determined according to the first segmentation result, and the average HU value of the solid component region is calculated according to the number of voxels and the HU value in the solid component region. Specifically, the average HU value of the solid component region = the sum of the HU values ​​of all voxels in the solid component region / the number of voxels in the solid component region.

[0108] In the specific implementation of analyzing the solid component in step S41, the contour line of each cross section can also be extracted according to the first segmentation result. It should be noted that if the cross section has multiple connected domains, the contour line of each connected domain is extracted.

[0109] In the specific implementation of analyzing the solid component in step S41, as Fig. 9 As shown, the following steps S411a to S411d are also included:

[0110] Step S411a: extracting the largest connected component of the first segmentation result.

[0111] Step S411b: determine the maximum cross section according to the segmented area of ​​each cross section in the maximum connected domain.

[0112] Step S411c, extracting the contour points of the maximum cross section.

[0113] Step S411d: determine the major diameter of the solid component area according to the two contour points with the farthest distance.

[0114] This embodiment provides a method for determining the long diameter of the solid component area in the pulmonary nodule area. Specifically, the long diameter of the solid component area can be determined by extracting the largest connected domain, determining the largest cross section, extracting contour points, and determining the long diameter of the solid component area based on the farthest contour point.

[0115] In the specific implementation of analyzing the solid component in step S41, the volume of the solid component area in the pulmonary nodule image may also be calculated according to the first segmentation result.

[0116] In the specific implementation of analyzing the solid component in step S41, as Fig.10 As shown, the following steps S412a to S3412c may also be included:

[0117] Step S412a: determining a first volume of a solid component region in the pulmonary nodule image according to the first segmentation result.

[0118] Step S412b: determine a second volume of the nodule area in the pulmonary nodule image according to the second segmentation result.

[0119] Step S412c: Calculate the proportion of solid components in the nodule based on the first volume and the second volume. Specifically, the ratio of the first volume to the second volume is the proportion of solid components in the nodule, that is, the solid component proportion.

[0120] In a specific implementation, the volume of the solid component region can be calculated according to the following formula:

[0121] Volume=volume×n;

[0122] Where n is the number of voxels with a value of 1 in the first segmentation result, volume is the volume of each voxel, volume=spacing_x×spacing_y×spacing_z; spacing_x represents the size of a voxel in the x-axis direction, which can be considered as the length of the voxel, spacing_y represents the size of a voxel in the y-axis direction, which can be considered as the width of the voxel, and spacing_z represents the size of a voxel in the z-axis direction, which can be considered as the height of the voxel.

[0123] In this embodiment, the lung nodule is a partially solid nodule. The proportion of the solid component in the nodule in the lung nodule image is analyzed. Specifically, the proportion of the solid component in the nodule can be obtained based on the first volume of the solid component area in the lung nodule image and the second volume of the nodule area. Based on this proportion, medical diagnosis can be better assisted.

[0124] In a specific implementation, in response to the user's operation, the extracted cross-sectional contour line of the solid component area, the calculated average HU value of the solid component area, the calculated solid component ratio, the determined long diameter of the solid component area and / or the volume of the solid component area can be displayed in the interface to facilitate doctors to further diagnose lung nodules based on the contour line, long diameter and / or volume of the solid component area.

[0125] Fig.11a , Fig.11b , Fig.11c In a specific example, if the long diameter of a lung nodule in a lung nodule image is greater than a preset value, such as a long diameter greater than 6 mm, the display is as follows: Fig.11a The interface shown in the figure, in response to the user checking the "Partial Solid Nodule Analysis" check box in the display interface, can display the outline of the solid component area in the lung nodule in the interface, and can also display the long diameter, volume, solid component ratio and specific values ​​of the average HU value of the solid component area, where the long diameter of the solid component area is 12.21mm and the volume is 459.49mm 3, the solid component ratio is 0.33, and the average HU value of the solid component area is -284.1. In this example, the long diameter of the pulmonary nodule is greater than the preset value, and the relevant parameters of the pulmonary nodule can be displayed on the display interface in response to the user's operation.

[0126] In a specific implementation, in response to a user Fig.11a The sliding operation in the display interface can display the contours of different layers of the solid component area. Fig.11b Used to show the contour line of the 32nd layer of the solid component area, Fig.11c The outline of the 33rd layer used to show the solid component area.

[0127] Fig.12 Another display interface diagram for showing another type of lung nodule. In another specific example, if the major diameter of the lung nodule in the lung nodule image is smaller than a preset value, for example, the major diameter is smaller than 6 mm, then the display is as follows: Fig.12 In the interface shown, the check boxes of "Partial Solid Nodule Analysis" in the display interface are all gray, and the user cannot check them, so the analysis results of the solid component area of ​​the lung nodule cannot be displayed in the display interface.

[0128] In other optional implementations, other omics analyses may be performed based on the first segmentation result and the second segmentation result, so as to better assist medical diagnosis.

[0129] This embodiment also provides an image processing device 80, such as Fig.13 As shown, it includes an image acquisition module 81 and an image processing module 82.

[0130] The image acquisition module 81 is used to acquire the lung nodule image to be segmented.

[0131] The image processing module 82 is used to input the pulmonary nodule image into an image segmentation model to perform segmentation processing on the solid component area to obtain a first segmentation result. The image segmentation model is trained based on the training method described in Example 1.

[0132] In an optional embodiment, the image processing device further includes a nodule segmentation module and a feature analysis module. The nodule segmentation module is used to input the pulmonary nodule image into a nodule segmentation model to perform segmentation processing on the nodule area to obtain a second segmentation result; wherein the nodule segmentation model is obtained based on training samples. The feature analysis module is used to analyze the solid component features in the pulmonary nodule image according to the first segmentation result and the second segmentation result.

[0133] In an optional embodiment, the above-mentioned feature analysis module is specifically used to determine the first volume of the solid component area in the lung nodule image according to the first segmentation result; determine the second volume of the nodule area in the lung nodule image according to the second segmentation result; and calculate the proportion of the solid component in the nodule based on the first volume and the second volume.

[0134] In an optional implementation, the image processing device further includes a contour line extraction module, configured to extract the contour line of each cross section according to the first segmentation result.

[0135] In an optional embodiment, the image processing device further includes a connected domain extraction unit, a cross-section determination unit, a contour point extraction unit, and a major diameter determination unit. The connected domain extraction unit is used to extract the largest connected domain of the first segmentation result. The cross-section determination unit determines the largest cross section according to the segmented area of ​​each cross section in the largest connected domain. The contour point extraction unit is used to extract the contour points of the largest cross section. The major diameter determination unit is used to determine the major diameter of the solid component area according to the two contour points with the longest distance.

[0136] It should be noted that, in the present embodiment, the image processing device may specifically be a separate chip, a chip module or an electronic device, or may be a chip or a chip module integrated into an electronic device.

[0137] The various modules / units included in the image processing apparatus described in this embodiment may be software modules / units or hardware modules / units, or may be partially software modules / units and partially hardware modules / units.

[0138] Example 3

[0139] Fig.14 A schematic diagram of the structure of an electronic device provided for this embodiment. The electronic device includes at least one processor and a memory connected to the at least one processor. The memory stores a computer program that can be run by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the training method of the image segmentation model of embodiment 1 or the image processing method of embodiment 2. The electronic device provided in this embodiment can be a personal computer, such as a desktop computer, an all-in-one computer, a laptop computer, a tablet computer, etc., and can also be a terminal device such as a mobile phone, a wearable device, a handheld computer, etc. Fig.14 The electronic device 3 shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present invention.

[0140] The components of the electronic device 3 may include, but are not limited to: the at least one processor 4 mentioned above, the at least one memory 5 mentioned above, and a bus 6 connecting different system components (including the memory 5 and the processor 4).

[0141] The bus 6 includes a data bus, an address bus and a control bus.

[0142] The memory 5 may include a volatile memory, such as a random access memory (RAM) 51 and / or a cache memory 52 , and may further include a read only memory (ROM) 53 .

[0143] The memory 5 may also include a program / utility 55 having a set (at least one) of program modules 54, such program modules 54 including but not limited to: an operating system, one or more application programs, other program modules and program data, each of which or some combination may include the implementation of a network environment.

[0144] The processor 4 executes various functional applications and data processing by running the computer program stored in the memory 5, such as the training method of the above-mentioned image segmentation model or the image processing method.

[0145] The electronic device 3 may also communicate with one or more external devices 7 (e.g., keyboards, pointing devices, etc.). Such communication may be performed via an input / output (I / O) interface 8. Furthermore, the electronic device 3 may also communicate with one or more networks (e.g., a local area network (LAN), a wide area network (WAN), and / or a public network, such as the Internet) via a network adapter 9. Fig.14 As shown, the network adapter 9 communicates with other modules of the electronic device 3 via the bus 6. It should be understood that although Fig.14 Not shown, other hardware and / or software modules may be used in conjunction with the electronic device 3, including but not limited to: microcode, device drivers, redundant processors, external disk drive arrays, RAID (disk array) systems, tape drives, and data backup storage systems.

[0146] It should be noted that although several units / modules or sub-units / modules of the electronic device are mentioned in the above detailed description, this division is merely exemplary and not mandatory. In fact, according to an embodiment of the present invention, the features and functions of two or more units / modules described above can be embodied in one unit / module. Conversely, the features and functions of one unit / module described above can be further divided into multiple units / modules to be embodied.

[0147] Example 4

[0148] This embodiment provides a computer-readable storage medium storing a computer program, and when the computer program is executed by a processor, the image segmentation model training method of embodiment 1 or the image processing method of embodiment 2 is implemented.

[0149] The readable storage medium may include but is not limited to: a portable disk, a hard disk, a random access memory, a read-only memory, an erasable programmable read-only memory, an optical storage device, a magnetic storage device or any suitable combination of the above.

[0150] In a possible implementation, the present invention can also be implemented in the form of a program product, which includes a program code. When the program product is run on an electronic device, the program code is used to enable the electronic device to execute the training method of the image segmentation model of Example 1 or the image processing method of Example 2.

[0151] Among them, the program code for executing the present invention can be written in any combination of one or more programming languages, and the program code can be executed completely on the electronic device, partially on the electronic device, as an independent software package, partially on the electronic device and partially on a remote device, or completely on the remote device.

[0152] Although the specific embodiments of the present invention are described above, it should be understood by those skilled in the art that this is only for illustration and the protection scope of the present invention is defined by the appended claims. Those skilled in the art may make various changes or modifications to these embodiments without departing from the principles and essence of the present invention, but these changes and modifications all fall within the protection scope of the present invention.

Claims

1. A training method for an image segmentation model, characterized in that: include: Obtain a training sample image set; The training sample image set includes a lung nodule sample image, a first mask image and a second mask image, wherein the nodules in the lung nodule sample image are mixed ground glass nodules; the first mask image is a gold standard, the first mask image is used to indicate a solid component area and a non-solid component area in the lung nodule sample image, and the second mask image is used to indicate a blood vessel area in the lung nodule sample image; Inputting the pulmonary nodule sample image into a preset segmentation model to predict the solid component area, and obtaining a predicted segmentation result; Calculate, according to the predicted segmentation result, the first mask image and the second mask image, a first loss of pixels in the solid component area, a second loss of pixels in the blood vessel area and a third loss of pixels in the background area; wherein the background area is an area other than the blood vessel area in the non-solid component area; Adjusting the parameters of the preset segmentation model according to the first loss, the second loss, and the third loss until a convergence condition is met, thereby obtaining a trained image segmentation model; The step of calculating the first loss of pixels in the solid component area, the second loss of pixels in the blood vessel area, and the third loss of pixels in the background area according to the predicted segmentation result, the first mask image, and the second mask image comprises: For pixels in the solid component region belonging to the gold standard, substituting the corresponding pixel prediction probability in the predicted segmentation result into the loss function to calculate the first loss; For pixels in the non-solid component area where the gold standard is located, determine whether they are in the vascular area based on the second mask image. If they are in the vascular area, substitute the corresponding pixel prediction probability in the predicted segmentation result into the loss function to calculate the second loss; if they are not in the vascular area, substitute the corresponding pixel prediction probability in the predicted segmentation result into the loss function to calculate the third loss.

2. The image segmentation model training method according to claim 1, characterized in that: The adjusting the parameters of the preset segmentation model according to the first loss, the second loss and the third loss specifically includes: Performing a weighted summation on the first loss, the second loss and the third loss to obtain a target loss; wherein the weight of the second loss is greater than the weight of the third loss; The parameters of the preset segmentation model are adjusted according to the target loss.

3. An image processing method, characterized in that: include: Acquire a lung nodule image to be segmented; The lung nodule image is input into an image segmentation model to perform segmentation processing on the solid component area to obtain a first segmentation result; wherein the image segmentation model is trained based on the training method described in claim 1 or 2.

4. The image processing method according to claim 3, characterized in that: After acquiring the pulmonary nodule image to be segmented, the method further includes: Inputting the lung nodule image into a nodule segmentation model to perform segmentation processing on the nodule region to obtain a second segmentation result; wherein the nodule segmentation model is obtained based on training samples; The solid component features in the pulmonary nodule image are analyzed according to the first segmentation result and the second segmentation result.

5. The image processing method according to claim 4, characterized in that: Analyzing the solid component features in the pulmonary nodule image according to the first segmentation result and the second segmentation result includes: If the major diameter of the nodule in the second segmentation result is greater than a preset value, the solid component in the pulmonary nodule image is analyzed.

6. The image processing method according to claim 5, characterized in that: The analyzing the solid components in the pulmonary nodule image includes: Determine a first volume of a solid component region in the pulmonary nodule image according to the first segmentation result; Determine a second volume of the nodule area in the pulmonary nodule image according to the second segmentation result; The proportion of the solid component in the nodule is calculated based on the first volume and the second volume.

7. The image processing method according to claim 5, characterized in that: The analyzing the solid components in the pulmonary nodule image includes: Extracting a contour line of each cross section according to the first segmentation result; and / or, Extracting the largest connected domain of the first segmentation result; Determine the maximum cross section according to the segmented area of ​​each cross section in the maximum connected domain; Extracting contour points of the maximum cross section; The long diameter of the solid component area is determined based on the two contour points with the farthest distance from each other.

8. An image processing device, characterized in that: include: An image acquisition module, used for acquiring an image of a lung nodule to be segmented; An image processing module is used to input the lung nodule image into an image segmentation model to perform segmentation processing on the solid component area to obtain a first segmentation result; wherein the image segmentation model is trained based on the training method described in claim 1 or 2.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, it implements the training method of the image segmentation model as described in claim 1 or 2 or the image processing method as described in any one of claims 3-7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, it implements the training method of the image segmentation model as described in claim 1 or 2 or the image processing method as described in any one of claims 3-7.

Images