ResUnet medical image segmentation system based on edge operator improvement

Abstract

The present invention provides an improved ResUnet medical image segmentation system based on an edge operator, comprising: a data acquisition module, configured to: acquire medical image data to be processed; a lung segmentation module, configured to: obtain lung segmentation results according to the acquired medical image data and a preset ResUnet network model; wherein a first loss function is obtained according to a prediction result and labeled standard data, a second loss function is obtained according to the prediction result and the labeled data processed by the edge operator, and a weighted sum of the first loss function and the second loss function is used as the loss function of the ResUnet network model; the present invention performs optimization adjustment during the training process and redesigns the loss function of model training and learning, which can perform intelligent feature learning more quickly and greatly improve the accuracy of organ structure segmentation.

Description

Technical Field

[0001] The present invention relates to the technical field of medical image processing, and in particular to a ResUnet medical image segmentation system based on edge operator improvement. Background Art

[0002] The statements in this section merely provide background art related to the present invention and do not necessarily constitute prior art.

[0003] For deeper lung nodules, segmental resection is a reliable surgical procedure that can ensure complete removal of the nodules. Affected by the anatomical structure of the segmental hilum, the surgical procedures for different lung segments are not the same, and the three major structures of the segmental hilum in different lung segments: arteries, veins, and trachea are handled differently. Especially when there are variations in the anatomical structure, the difficulty of segmental surgery increases significantly. Therefore, in the diagnosis of lung nodules, by providing information about the lung segment where the lung nodules are located, more accurate information can be provided for the diagnosis and treatment of lung nodules by the physician, thereby better assisting the physician in obtaining more accurate diagnostic information and facilitating more accurate subsequent treatment plans.

[0004] With the development of deep learning and other technologies, the application of deep learning technology to medical auxiliary diagnosis has become an inevitable trend. Through auxiliary diagnosis such as deep learning algorithms, doctors' work efficiency can be greatly improved and more accurate diagnosis results can be provided to patients. Through deep learning, lung CT image information can be better utilized to better segment lung segments and other parts.

[0005] However, the inventors found that various existing deep learning-based solutions mostly use a large amount of labeled data for model training, and do not make better use of medical-related anatomical structure knowledge and edge feature information of imaging data, resulting in poor segmentation of structure edges. Summary of the invention

[0006] In order to address the shortcomings of the prior art, the present invention provides a ResUnet medical image segmentation system based on an improved edge operator, which performs optimization adjustment and redesigns the loss function of model training learning during the training process, enables faster intelligent feature learning, and greatly improves the accuracy of organ structure segmentation.

[0007] In order to achieve the above object, the present invention adopts the following technical solution:

[0008] A medical image segmentation system based on ResUnet improved by edge operator, comprising:

[0009] The data acquisition module is configured to: acquire medical image data to be processed;

[0010] The lung segmentation module is configured to: obtain lung segmentation results according to the acquired medical image data and the preset ResUnet network model;

[0011] Among them, the first loss function is obtained according to the prediction result and the labeled standard data, and the second loss function is obtained according to the prediction result and the labeled data processed by the edge operator. The weighted sum of the first loss function and the second loss function is the loss function of the ResUnet network model.

[0012] A second aspect of the present invention provides a computer-readable storage medium having a program stored thereon, which implements the following steps when executed by a processor:

[0013] Acquiring medical imaging data to be processed;

[0014] According to the acquired medical imaging data and the preset ResUnet network model, the lung segmentation results are obtained;

[0015] Among them, the first loss function is obtained according to the prediction result and the labeled standard data, and the second loss function is obtained according to the prediction result and the labeled data processed by the edge operator. The weighted sum of the first loss function and the second loss function is the loss function of the ResUnet network model.

[0016] A third aspect of the present invention provides an electronic device, comprising a memory, a processor, and a program stored in the memory and executable on the processor, wherein the processor implements the following steps when executing the program:

[0017] Acquiring medical imaging data to be processed;

[0018] According to the acquired medical imaging data and the preset ResUnet network model, the lung segmentation results are obtained;

[0019] Among them, the first loss function is obtained according to the prediction result and the labeled standard data, and the second loss function is obtained according to the prediction result and the labeled data processed by the edge operator. The weighted sum of the first loss function and the second loss function is the loss function of the ResUnet network model.

[0020] Compared with the prior art, the present invention has the following beneficial effects:

[0021] The improved ResUnet medical image segmentation system based on the edge operator described in the present invention obtains a first loss function according to the prediction results and the labeled standard data during the training process, and obtains a second loss function according to the prediction results and the labeled data processed by the Canny operator. The weighted sum of the first loss function and the second loss function is used as the loss function of the ResUnet network model, which can perform intelligent feature learning more quickly and greatly improve the accuracy of organ structure segmentation.

[0022] The edge detection loss function used in the present invention can effectively improve the speed of model training, reduce the training time, reduce the demand for large amounts of model data, reduce the amount of labeled data, and further improve the segmentation accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] The accompanying drawings in the specification, which constitute a part of the present invention, are used to provide a further understanding of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute improper limitations on the present invention.

[0024] Figure 1 A schematic diagram of the workflow of the ResUnet medical image segmentation system based on edge operator improvement provided in Example 1 of the present invention.

[0025] Figure 2 A schematic diagram of the network structure provided for Example 1 of the present invention.

[0026] Figure 3 This is a schematic diagram of the algorithm flow provided in Example 1 of the present invention. DETAILED DESCRIPTION

[0027] The present invention will be further described below in conjunction with the accompanying drawings and embodiments.

[0028] It should be noted that the following detailed descriptions are all illustrative and intended to provide further explanation of the present invention. Unless otherwise specified, all technical and scientific terms used herein have the same meanings as those commonly understood by those skilled in the art to which the present invention belongs.

[0029] It should be noted that the terms used herein are only for describing specific embodiments and are not intended to limit exemplary embodiments according to the present invention. As used herein, unless the context clearly indicates otherwise, the singular form is also intended to include the plural form. In addition, it should be understood that when the terms "comprising" and / or "including" are used in this specification, it indicates the presence of features, steps, operations, devices, components and / or combinations thereof.

[0030] In the absence of conflict, the embodiments of the present invention and the features of the embodiments may be combined with each other.

[0031] Embodiment 1:

[0032] like Figure 1 , Figure 2 and Figure 3 As shown, Embodiment 1 of the present invention provides a ResUnet medical image segmentation system based on edge operator improvement, including:

[0033] The data acquisition module is configured to: acquire medical image data to be processed;

[0034] The lung segmentation module is configured to: obtain lung segmentation results according to the acquired medical image data and the preset ResUnet network model;

[0035] Among them, the first loss function is obtained according to the prediction result and the labeled standard data, and the second loss function is obtained according to the prediction result and the labeled data processed by the Canny operator. The weighted sum of the first loss function and the second loss function is the loss function of the ResUnet network model.

[0036] Specifically, the workflow is as follows:

[0037] (1) Constructing a dataset for lung segmentation: Using CT data of the lungs, experienced physicians annotate the structures of each lung segment.

[0038] (2) Data inspection: The annotated data are submitted to the chief physician of another tertiary hospital for inspection. When both experts have no objection to the annotated data, the data annotation results are used as standard annotated data. If there are any objections, the experts will discuss and reconfirm the annotation modification opinions until there is no objection to the data, which is used as standard annotated data.

[0039] (3) Data preprocessing: The labeled data is windowed according to the HU value [-1200, 600] and normalized. The output is processed into the format of [128, 128, 128] as the model input standard. Secondly, the Canny operator is used to perform edge detection calculation on the labeled data to obtain the labeled data of the more obvious organ segmentation edge features such as the oblique fissure and bronchus in CT. The data is divided into a training set and a test set, where the test set does not participate in training.

[0040] (4) Loss function design, Loss = (1-β)·CELoss + β·CYLoss;

[0041] CELoss is the cross entropy loss function, Calculate the cross entropy loss function between the predicted results and the labeled standard data;

[0042] CYLoss also uses the cross entropy loss function, and calculates the loss function through the prediction results and the labeled data processed by the Canny operator;

[0043] The β factor is a variable factor that will be adjusted during the training process to guide the model's training effect on edge learning and overall features. For example, it will be set to 0.9, 0.7, 0.5, 0.3, and 0.1 during training, and the weight will decay as the training progresses.

[0044] (5) Network training: The preprocessed data (1, 128, 128, 128) is input into the ResUnet network for training. During the training process, random sampling is used to sample the overall data to increase data diversity and enhance the generalization ability of the model.

[0045] (6) Prediction of lung segmentation model: After iterative training is completed, the model is used to predict the lung segmentation of the test set data, and the segmentation prediction results are returned to the physician as auxiliary results of lung nodules to assist the physician in diagnosis.

[0046] The following is a specific example:

[0047] S1: Select intact lung CT data, and have experts label the data. Label different lung segments with different colors and labels and save them in nii.gz format. The overall data is divided into 19 parts, of which 0 is set as the background label.

[0048] S2: After the preliminary annotation is completed, another group of experts will review the annotated data. If the conclusions are consistent, they will be used as the final results. If there are disagreements, further discussions will be held to modify the annotated data or keep the original annotated data.

[0049] S3: The audited data is divided into a training set and a test set according to a ratio of 7:3. The training set uses the KFold method. Before the training begins, the data is preprocessed. The HU value of the lung CT data is windowed, the lung data is obtained, and resampled. The data spacing is guaranteed to be 1mm thick in three dimensions, and the data is normalized and mapped to 255 grayscale data. Then it is converted to unit8 format. The three dimensions of the data are randomly windowed, and the data of [128, 128, 128] dimensions are taken out as the input of the network.

[0050] S4: The network consists of an encoder module, a down_conv module, a decoder module, an up_conv module and a map module.

[0051] Among them, the encoder module is mainly composed of 3 3D convolutions and 3 PRule activation functions, the down_conv module is mainly composed of a 3D convolution and a PRule activation function, the decoder module is mainly composed of 3 3D convolutions and 3 PRule activation functions, and the up_conv module is mainly composed of a 3D transposed convolution and a PRule activation function. The map module is composed of a 3D convolution and a UPSample upsampling layer. Finally, a softmax function is used to obtain the final result.

[0052] S5: During the training process, the Adam algorithm is selected to optimize the model, and the initial learning rate is 0.0001. When the number of training epochs reaches 50, the initial learning rate is multiplied by the decay coefficient to obtain a new learning rate. The model weights are saved every 50 epochs.

[0053] S6: Set the total number of training rounds to 2000, and record the changes in loss values. Check the changes in loss during training and verify the model effect. When the model loss tends to be stable, take the model as the use model.

[0054] S7: Use the saved model to verify on the test set data and calculate the accuracy, dice value and other evaluation parameters, and output the prediction results. In addition to the above evaluation criteria, the generated mask segmentation results are also reconstructed in 3D and compared to see if the results are consistent and meet the usage criteria.

[0055] Embodiment 2:

[0056] Embodiment 2 of the present invention provides a computer-readable storage medium on which a program is stored. When the program is executed by a processor, the following steps are implemented:

[0057] Acquiring medical imaging data to be processed;

[0058] According to the acquired medical imaging data and the preset ResUnet network model, the lung segmentation results are obtained;

[0059] Among them, the first loss function is obtained according to the prediction result and the labeled standard data, and the second loss function is obtained according to the prediction result and the labeled data processed by the Canny operator. The weighted sum of the first loss function and the second loss function is the loss function of the ResUnet network model.

[0060] The detailed steps are the same as those of the working method provided in Example 1 and will not be repeated here.

[0061] Embodiment 3:

[0062] Embodiment 3 of the present invention provides an electronic device, including a memory, a processor, and a program stored in the memory and executable on the processor, wherein the processor implements the following steps when executing the program:

[0063] Acquiring medical imaging data to be processed;

[0064] According to the acquired medical imaging data and the preset ResUnet network model, the lung segmentation results are obtained;

[0065] Among them, the first loss function is obtained according to the prediction result and the labeled standard data, and the second loss function is obtained according to the prediction result and the labeled data processed by the Canny operator. The weighted sum of the first loss function and the second loss function is the loss function of the ResUnet network model.

[0066] The detailed steps are the same as those of the working method provided in Example 1 and will not be repeated here.

[0067] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of hardware embodiments, software embodiments, or embodiments combining software and hardware. Furthermore, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage and optical storage, etc.) containing computer-usable program code.

[0068] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0069] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0070] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0071] A person skilled in the art can understand that all or part of the processes in the above-mentioned embodiments can be implemented by instructing the relevant hardware through a computer program, and the program can be stored in a computer-readable storage medium, and when the program is executed, it can include the processes of the embodiments of the above-mentioned methods. The storage medium can be a disk, an optical disk, a read-only memory (ROM) or a random access memory (RAM), etc.

[0072] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A medical image segmentation system based on ResUnet improved by edge operator, characterized by: include: The data acquisition module is configured to: acquire medical image data to be processed; The lung segmentation module is configured to: obtain lung segmentation results according to the acquired medical image data and the preset ResUnet network model; Among them, a first loss function is obtained according to the prediction result and the labeled standard data, and a second loss function is obtained according to the prediction result and the labeled data processed by the edge operator, and the weighted sum of the first loss function and the second loss function is used as the loss function of the ResUnet network model; The first loss function and the second loss function are both cross entropy loss functions; The loss function of the ResUnet network model is the product of the first loss function and the first coefficient, and the sum of the product of the second loss function and the second coefficient. The sum of the first coefficient and the second coefficient is 1. The first coefficient is a variable factor; ResUnet network model, including: encoder module, down_conv module, decoder module, up_conv module and map module. The encoder module includes 3 3D convolutions and 3 PRule activation functions. The down_conv module includes a 3D convolution and a PRule activation function, and the decoder module includes 3 3D convolutions and 3 PRule activation functions; The up_conv module includes a 3D transposed convolution and a PRule activation function, and the map module includes a 3D convolution and a UPSample upsampling layer.

2. The ResUnet medical image segmentation system based on edge operator improvement as claimed in claim 1, characterized in that: The training of the ResUnet network model includes: For the labeled data of the image, the window value is calculated according to the HU value, and the labeled data in a preset format is obtained after normalization processing; the Canny operator is used to perform edge detection calculation on the labeled data in the preset format to obtain the labeled data with obvious organ segmentation edge features of the oblique fissure and bronchus in CT.

3. A computer-readable storage medium having a program stored thereon, characterized in that: When the program is executed by the processor, the following steps are implemented: Acquiring medical imaging data to be processed; According to the acquired medical imaging data and the preset ResUnet network model, the lung segmentation results are obtained; Among them, a first loss function is obtained according to the prediction result and the labeled standard data, and a second loss function is obtained according to the prediction result and the labeled data processed by the edge operator, and the weighted sum of the first loss function and the second loss function is used as the loss function of the ResUnet network model; The first loss function and the second loss function are both cross entropy loss functions; The loss function of the ResUnet network model is the product of the first loss function and the first coefficient, and the sum of the product of the second loss function and the second coefficient. The sum of the first coefficient and the second coefficient is 1. The first coefficient is a variable factor; ResUnet network model, including: encoder module, down_conv module, decoder module, up_conv module and map module. The encoder module includes 3 3D convolutions and 3 PRule activation functions. The down_conv module includes a 3D convolution and a PRule activation function, and the decoder module includes 3 3D convolutions and 3 PRule activation functions; The up_conv module includes a 3D transposed convolution and a PRule activation function, and the map module includes a 3D convolution and a UPSample upsampling layer.

4. An electronic device comprising a memory, a processor, and a program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the following steps are implemented: Acquiring medical imaging data to be processed; According to the acquired medical imaging data and the preset ResUnet network model, the lung segmentation results are obtained; Among them, a first loss function is obtained according to the prediction result and the labeled standard data, and a second loss function is obtained according to the prediction result and the labeled data processed by the edge operator, and the weighted sum of the first loss function and the second loss function is used as the loss function of the ResUnet network model; The first loss function and the second loss function are both cross entropy loss functions; The loss function of the ResUnet network model is the product of the first loss function and the first coefficient, and the sum of the product of the second loss function and the second coefficient. The sum of the first coefficient and the second coefficient is 1. The first coefficient is a variable factor; ResUnet network model, including: encoder module, down_conv module, decoder module, up_conv module and map module. The encoder module includes 3 3D convolutions and 3 PRule activation functions. The down_conv module includes a 3D convolution and a PRule activation function, and the decoder module includes 3 3D convolutions and 3 PRule activation functions; The up_conv module includes a 3D transposed convolution and a PRule activation function, and the map module includes a 3D convolution and a UPSample upsampling layer.

Images