Liver tumor automatic segmentation method and device

Abstract

The application discloses a liver tumor automatic segmentation method and device, wherein the method comprises the following steps: collecting CT data of a liver cancer patient; extracting at least one liver tumor feature of the liver cancer patient from the CT data; inputting the at least one liver tumor feature into a preset liver tumor automatic segmentation model to obtain a liver tumor automatic segmentation result of the liver cancer patient, wherein the preset liver tumor automatic segmentation model is trained by a preset Transformer neural network of a Paddle-Paddle deep learning framework in a back end. Thus, the technical problem that the efficiency of liver tumor segmentation depending on expert experience and liver tumor feature selection is low and the precision of liver tumor segmentation relying on a deep learning algorithm is poor in the prior art is solved.

Description

Technical Field

[0001] This application relates to the field of medical image processing technology, and in particular to an automatic segmentation method and apparatus for liver tumors. Background Technology

[0002] Liver tumor segmentation is a crucial area in medical imaging segmentation. The main challenge lies in the significant individual differences among liver tumors, exhibiting difficult-to-distinguish characteristics in size, shape, texture, and color. For example, the boundaries of liver tumors are often similar to those of normal tissue, resulting in indistinct outlines. Furthermore, even within the same lesion area, substantial image differences are common, including uneven texture. Additionally, different imaging devices significantly impact the acquired images. Therefore, developing precise, efficient, and automated liver tumor segmentation methods is of paramount importance.

[0003] Currently, liver tumor segmentation technologies mainly employ three methods: manual segmentation, traditional machine learning segmentation, and deep learning segmentation. Manual segmentation and traditional machine learning segmentation both rely on expert experience and the selection of liver tumor features, making efficient and batch processing impossible. Deep learning segmentation is the latest approach, using deep neural networks to classify pixels of lesions and tissues to create segmented images. However, the accuracy of deep learning algorithms used for liver tumor segmentation, such as U-Net and its derivative architectures, still needs further improvement.

[0004] In summary, it is difficult to achieve both efficiency and accuracy in liver tumor segmentation using related technologies, and improvements are needed. Summary of the Invention

[0005] This application is based on the inventor's knowledge and discoveries regarding the following issues:

[0006] The development of modern medical imaging technology has brought about a large amount of medical imaging data, such as CT (Computed Tomography), MRI (Magnetic Resonance Imaging), and US (Ultrasound). This massive amount of imaging data has greatly helped in early disease screening, disease diagnosis, clinical decision-making, and prognostic observation.

[0007] However, typical imaging data is often highly noisy, containing a large amount of irrelevant tissue, while the ROI (region of interest) exists only within a small area. This significant noise not only imposes a huge computational burden on subsequent data analysis but also affects the analysis results. Therefore, medical image segmentation technology is particularly important. Medical image segmentation involves dividing the region of interest, such as lesions, into segments using specific methods. Subsequent analysis only focuses on the segmented portions, which not only improves work efficiency but also enhances the accuracy of the research.

[0008] Traditional medical image segmentation is primarily performed manually. While this method offers high accuracy, it is time-consuming and labor-intensive, hindering efficient, high-volume workflows. With advancements in radiomics and artificial intelligence, developing semi-automatic or even fully automated image segmentation methods can improve physician efficiency and facilitate rapid, systematic analysis and judgment.

[0009] In summary, based on the development of modern medical imaging technology and in order to address the shortcomings of traditional medical imaging, this application provides an automatic liver tumor segmentation method and apparatus to solve the technical problems in related technologies, such as the low efficiency of liver tumor segmentation relying on expert experience and liver tumor feature selection, and the poor accuracy of liver tumor segmentation relying on deep learning algorithms.

[0010] The first aspect of this application provides an automatic liver tumor segmentation method, comprising the following steps: acquiring CT data of a liver cancer patient; extracting at least one liver tumor feature from the CT data; and inputting the at least one liver tumor feature into a preset automatic liver tumor segmentation model to obtain an automatic segmentation result of the liver tumor of the liver cancer patient, wherein the preset automatic liver tumor segmentation model is trained by a preset Transformer neural network using the Paddle-Paddle deep learning framework in the backend.

[0011] Optionally, in one embodiment of this application, before inputting the at least one liver tumor feature into the preset automatic liver tumor segmentation model, the method further includes: collecting CT data from multiple liver cancer patients, labeling the tumor regions in the CT data, and generating a training set and a validation set; constructing the preset Transformer neural network, wherein the backend of the preset Transformer neural network adopts a preset Paddle-Paddle deep learning framework to generate a deep learning model; and training the deep learning model using the training set and the validation set to generate the preset automatic liver tumor segmentation model.

[0012] Optionally, in one embodiment of this application, after annotating the tumor region in the CT data, the method further includes: preprocessing the annotated CT data until the preset model input conditions are met to obtain preprocessed CT data.

[0013] Optionally, in one embodiment of this application, the preprocessing of the annotated CT data includes: slicing the annotated CT data, trimming the remaining portion outside the annotations to obtain sliced ​​CT data; and converting the sliced ​​CT data into data of a preset format.

[0014] Optionally, in one embodiment of this application, the construction of the preset Transformer neural network includes: segmenting the data into multiple sub-images according to the preset format and mapping them into linear embedding sequences; encoding the multiple sub-images using an encoder, wherein the Transformer layer in the encoder consists of a multi-head self-attention mechanism and a multilayer perceptron; upsampling the output bilinearity to the full image resolution using a decoder, followed by a classification layer with pixel-level cross-entropy loss; and constructing an automatic liver tumor segmentation model based on the Transformers architecture using the PaddlePaddle deep learning framework.

[0015] A second aspect of this application provides an automatic liver tumor segmentation device, comprising: an acquisition module for acquiring CT data of a liver cancer patient; an extraction module for extracting at least one liver tumor feature from the CT data of the liver cancer patient; and a segmentation module for inputting the at least one liver tumor feature into a preset automatic liver tumor segmentation model to obtain an automatic segmentation result of the liver tumor of the liver cancer patient, wherein the preset automatic liver tumor segmentation model is trained by a preset Transformer neural network using the Paddle-Paddle deep learning framework in the backend.

[0016] Optionally, in one embodiment of this application, it further includes: an annotation module, used to collect CT data from multiple liver cancer patients and annotate the tumor regions in the CT data to generate a training set and a validation set; a first modeling module, used to build the preset Transformer neural network, wherein the backend of the preset Transformer neural network adopts a preset Paddle-Paddle deep learning framework to generate a deep learning model; and a second modeling module, used to train the deep learning model using the training set and the validation set to generate the preset automatic liver tumor segmentation model.

[0017] Optionally, in one embodiment of this application, the annotation module is further used to preprocess the annotated CT data until the preset model input conditions are met, so as to obtain the preprocessed CT data.

[0018] Optionally, in one embodiment of this application, the annotation module includes: a slicing unit, used to slice the annotated CT data, trim the remaining portion outside the annotation, to obtain sliced ​​CT data; and a conversion unit, used to convert the sliced ​​CT data into data in a preset format.

[0019] Optionally, in one embodiment of this application, the first modeling module includes: a mapping unit, used to segment data into multiple sub-images according to the preset format and map them into linear embedding sequences; an encoding unit, used to encode the multiple sub-images using an encoder, wherein the Transformer layer in the encoder consists of a multi-head self-attention mechanism and a multilayer perceptron; a sampling unit, used to upsample the output bilinear to the full image resolution using a decoder, followed by a classification layer with pixel-level cross-entropy loss; and a modeling unit, used to build an automatic liver tumor segmentation model based on the Transformers architecture using the Paddle Paddle deep learning framework.

[0020] A third aspect of this application provides an electronic device, including: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the automatic liver tumor segmentation method as described in the above embodiments.

[0021] A fourth aspect of this application provides a computer-readable storage medium storing computer instructions for causing the computer to execute the automatic liver tumor segmentation device as described in the above embodiments.

[0022] This application's embodiments utilize a Transformer neural network based on the Paddle-Paddle deep learning framework to train an automatic liver tumor segmentation model. Based on the liver tumor characteristics of a liver cancer patient, it obtains the automatic segmentation result of the patient's liver tumor, ensuring efficiency while capturing as much effective contextual information as possible, thereby improving the accuracy of automatic liver tumor segmentation points. This solves the technical problems in related technologies where relying on expert experience and liver tumor feature selection for liver tumor segmentation is inefficient, and relying on deep learning algorithms for liver tumor segmentation has poor accuracy.

[0023] Additional aspects and advantages of this application will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of this application. Attached Figure Description

[0024] The above and / or additional aspects and advantages of this application will become apparent and readily understood from the following description of the embodiments taken in conjunction with the accompanying drawings, wherein:

[0025] Figure 1 This is a flowchart of an automatic liver tumor segmentation method provided according to an embodiment of this application;

[0026] Figure 2 This is a schematic diagram illustrating the principle of an automatic liver tumor segmentation method according to an embodiment of this application;

[0027] Figure 3 This is a schematic diagram of the architecture of a Transformers deep learning algorithm model according to an embodiment of this application;

[0028] Figure 4 This is a schematic diagram illustrating the effect of an automatic liver tumor segmentation method according to an embodiment of this application;

[0029] Figure 5 This is a flowchart of an automatic liver tumor segmentation method according to an embodiment of this application;

[0030] Figure 6 This is a schematic diagram of an automatic liver tumor segmentation device provided according to an embodiment of this application;

[0031] Figure 7 This is a schematic diagram of the structure of an electronic device provided according to an embodiment of this application. Detailed Implementation

[0032] The embodiments of this application are described in detail below. Examples of these embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and intended to explain this application, and should not be construed as limiting this application.

[0033] The following describes an automatic liver tumor segmentation method and apparatus according to embodiments of this application with reference to the accompanying drawings. Addressing the technical problems mentioned in the background section of the aforementioned related technologies, where relying on expert experience and liver tumor feature selection for liver tumor segmentation is inefficient, and relying on deep learning algorithms for liver tumor segmentation has poor accuracy, this application provides an automatic liver tumor segmentation method. In this method, embodiments of this application can utilize a Transformer neural network based on the Paddle-Paddle deep learning framework to train an automatic liver tumor segmentation model. This allows for the automatic segmentation of the liver tumor of a liver cancer patient based on the patient's liver tumor features, ensuring efficiency while capturing as much effective contextual information as possible, thereby improving the accuracy of automatic liver tumor segmentation points. Thus, this solves the technical problems of low efficiency in liver tumor segmentation relying on expert experience and liver tumor feature selection, and poor accuracy in liver tumor segmentation relying on deep learning algorithms in related technologies.

[0034] Specifically, Figure 1 This is a flowchart illustrating an automatic liver tumor segmentation method provided in an embodiment of this application.

[0035] like Figure 1 As shown, the automatic segmentation method for liver tumors includes the following steps:

[0036] In step S101, CT data of liver cancer patients are collected.

[0037] In actual implementation, the embodiments of this application can use a three-level enhanced scan of the upper abdomen, with the scanning range from the top of the liver of the liver cancer patient to the lower edge of the two kidneys. The specific images can include the arterial phase, portal venous phase and delayed phase, thereby obtaining complete CT data of the liver cancer patient.

[0038] In step S102, at least one liver tumor feature of the liver cancer patient is extracted from the CT data.

[0039] Furthermore, embodiments of this application can extract at least one liver tumor feature from the CT data of liver cancer patients, which facilitates the subsequent use of an automatic liver tumor segmentation model to obtain the automatic segmentation results of liver tumors in liver cancer patients. This can improve the accuracy of the automatic segmentation results of liver tumors while ensuring efficiency.

[0040] In step S103, at least one liver tumor feature is input into a preset automatic liver tumor segmentation model to obtain the automatic segmentation result of the liver tumor of the liver cancer patient. The preset automatic liver tumor segmentation model is trained by a preset Transformer neural network using the Paddle-Paddle deep learning framework in the backend.

[0041] As one possible implementation, this application embodiment can be based on a preset Transformer neural network using the Paddle-Paddle deep learning framework in the backend, and after training, a preset automatic liver tumor segmentation model is obtained. Then, the liver tumor features obtained in the above steps are input into the preset automatic liver tumor segmentation model to obtain the automatic segmentation result of the liver tumor of the liver cancer patient, and perform DICOM format conversion and 3D reconstruction. Compared with U-Net and its derivative architectures used for liver tumor segmentation in related technologies, this application embodiment can further improve the segmentation accuracy by using a preset Transformer neural network using the Paddle-Paddle deep learning framework in the backend.

[0042] Optionally, in one embodiment of this application, before inputting at least one liver tumor feature into a preset automatic liver tumor segmentation model, the method further includes: collecting CT data from multiple liver cancer patients, labeling tumor regions in the CT data, and generating a training set and a validation set; constructing a preset Transformer neural network, wherein the backend of the preset Transformer neural network adopts a preset Paddle-Paddle deep learning framework to generate a deep learning model; and training the deep learning model using the training set and validation set to generate the preset automatic liver tumor segmentation model.

[0043] Specifically, the embodiments of this application can collect CT data from multiple liver cancer patients. The sources of CT data from liver cancer patients can be various, such as medical databases. Specifically, the CT data from liver cancer patients can include three-level enhanced scans of the upper abdomen, with the scanning range from the top of the liver to the lower edge of the two kidneys, obtaining arterial phase, portal venous phase, and delayed phase images.

[0044] Furthermore, embodiments of this application can annotate tumor regions in CT data and preprocess the raw data to generate training and validation sets, thereby ensuring that the raw data meets the model input standards. When annotating tumor regions in CT data, those skilled in the art can be divided into three groups: the first group manually marks all liver tumor regions in the raw data; the second group can independently use 3D Slicer (Boston, MA, USA) software to mark the CT scans of each patient; and the third group can review the annotation results to ensure the accuracy of the annotation.

[0045] Furthermore, in this embodiment, a Transformer neural network can be built for liver tumor segmentation. The backend uses the Paddle-Paddle deep learning framework and is trained using supervised learning. The model input consists of a training set, a validation set, and corresponding manually labeled data, which are randomly divided into training and validation sets according to a set ratio.

[0046] Furthermore, in this embodiment, the training set, validation set, and corresponding labeled data can be input into the constructed neural network, and the model can be trained using the PaddlePaddle deep learning framework. After the model has been iterated to a certain number of rounds and converged, and the validation set has achieved the desired effect, the training is stopped and the network parameters are saved, and finally a preset automatic segmentation model for liver tumors is generated.

[0047] Optionally, in one embodiment of this application, after annotating the tumor region in the CT data, the method further includes: preprocessing the annotated CT data until the preset model input conditions are met to obtain the preprocessed CT data.

[0048] It is understandable that labeled CT data cannot be directly used to train an automatic liver tumor segmentation model. Therefore, in this embodiment, the labeled CT data can be preprocessed until the CT data meets the preset input conditions of the automatic liver tumor segmentation model, thereby generating a training set and a validation set, so that the original data meets the model input standards, which facilitates the subsequent training and validation of the model.

[0049] Optionally, in one embodiment of this application, the annotated CT data is preprocessed, including: slicing the annotated CT data, trimming the remaining parts outside the annotations to obtain sliced ​​CT data; and converting the sliced ​​CT data into data of a preset format.

[0050] In some embodiments, preprocessing the labeled CT data may specifically include: normalizing the CT scan by standardizing it with a preset threshold, wherein the preset threshold can be -1000Hu or 400Hu; retaining the manually labeled CT slices in the CT data and cropping the remaining parts; converting the data to the appropriate format, such as from NIFIT format to PNG format, with an image pixel size of 512×512; and then, on a patient-by-patient basis, randomly dividing the dataset and the labeled data in the above steps into a training set, a validation set, and an independent test set in a ratio of 8:1:1, wherein the training set can perform data augmentation operations to balance the data categories, which may specifically include horizontal flipping, random rotation, and random blurring.

[0051] Optionally, in one embodiment of this application, a preset Transformer neural network is constructed, comprising: segmenting data into multiple sub-images according to a preset format and mapping them to linear embedding sequences; encoding the multiple sub-images using an encoder, wherein the Transformer layer in the encoder consists of a multi-head self-attention mechanism and a multilayer perceptron; upsampling the output bilinearity to the full image resolution using a decoder, followed by a classification layer with pixel-level cross-entropy loss; and constructing an automatic liver tumor segmentation model based on the Transformers architecture using the PaddlePaddle deep learning framework.

[0052] In actual implementation, the embodiments of this application can adopt the encoder-decoder architecture of Transformers, that is, the image generated after format conversion is segmented into small blocks and mapped into linear embedding sequences, and then encoded by an encoder. The Transformer layer in the encoder can be composed of MSA (multi-head self-attention) and MLP (Multilayer Perceptron).

[0053] In addition, in the decoding part, the decoder can go through a naive upsampling stage to upsample the output bilinearly to the full image resolution, followed by a classification layer with pixel-level cross-entropy loss.

[0054] Furthermore, in this embodiment, the PaddlePaddle deep learning framework can be used to build an automatic liver tumor segmentation model based on the Transformers architecture, which facilitates the subsequent use of the automatic liver tumor segmentation model to obtain high-precision automatic liver tumor segmentation results.

[0055] Furthermore, for the trained model, the embodiments of this application can test the model by inputting an independent test set in the dataset. When the test set achieves the desired effect, it can be verified that the automatic liver tumor segmentation model has the ability to accurately segment liver tumors automatically.

[0056] The following is combined with Figures 2 to 5 As shown, a specific embodiment of the automatic liver tumor segmentation method of this application will be described in detail.

[0057] like Figure 2 As shown, the embodiments of this application may include two parts: a model training part and an online segmentation part.

[0058] The online segmentation part will be described below, and the model training part may include the following steps:

[0059] Step S201: Acquisition of raw CT data. This embodiment of the application can retrospectively collect preoperative CT data of liver cancer patients undergoing liver surgery as raw CT data. In actual implementation, this embodiment of the application can use CT scanning equipment from German CT (SOMATOM Definition Flash, Siemens) and American Discovery CT (GE Healthcare). The scanning method can be a three-level contrast-enhanced scan of the upper abdomen, ranging from the top of the liver to the lower edges of the two kidneys. Scanning parameters can include a voltage of 120kV, a current of 200-350mA, a slice thickness of 5mm, a slice spacing of 5mm, and a resolution pixel matrix size of 512×512. Alternatively, this embodiment of the application can use a pressure injector to inject iohexol and 350mg / ml of iodine via a peripheral vein at a flow rate of 3.0ml / s and a dose of 1.5ml / kg.

[0060] In this embodiment of the application, the delay times for the arterial phase, venous phase, and equilibrium phase during CT scanning can be 30 seconds, 60 seconds, and 120 seconds, respectively, thereby acquiring images of AP (arterial phase), PVP (portal venous phase), and DP (delay period) for research.

[0061] Step S202: Manual annotation. In actual implementation, this embodiment of the application can select two radiologists with more than 15 years of experience to manually annotate the tumor areas in the CT data. One physician independently uses 3DSlicer (Boston, MA, USA) software to annotate the CT scans of each patient. The annotation method mainly relies on the expert's prior knowledge to meticulously delineate the tumor boundaries. The other physician reviews the annotation results. If there are no objections, the annotation is adopted; if there are objections, it is returned for discussion, and the annotation is redone if necessary.

[0062] Step S203: Data Preprocessing. It is understood that CT scans store raw voxel intensities in Hounsfield units (HU). In this embodiment, -1000 HU and 400 HU can be used as thresholds to normalize the CT scan, i.e., a normalization operation. Since the raw data contains many slices without tumors, increasing unnecessary computation, this embodiment can find the slices from the beginning to the end of the tumor in the manually labeled data and crop out the remaining portion.

[0063] Furthermore, since the input channel of the deep learning framework in this embodiment is 2-dimensional, the data undergoes a corresponding format conversion from NIFIT format to PNG format, with each image having a pixel size of 512×512.

[0064] Furthermore, in this embodiment, the dataset and manually labeled data can be randomly divided into training, validation, and independent test sets in an 8:1:1 ratio, with each patient as a unit. The training set undergoes data augmentation operations to balance the data categories, specifically including horizontal flipping, random rotation, and random blurring.

[0065] Step S204: Construction of Transformers Neural Network. Specifically, in this embodiment, a Transformers neural network with the Paddle-Paddle deep learning framework can be constructed for liver tumor segmentation. In this embodiment, the preprocessed CT data obtained in the above steps can be divided into small blocks of 4×4 and mapped into linear embedding sequences, which are then encoded by an encoder. The Transformer layer in the encoder consists of MSA and MLP.

[0066] Furthermore, in the decoding section, the decoder performs a naive upsampling process to upsample the output bilinearly to the full image resolution, followed by a classification layer with pixel-level cross-entropy loss. Specifically, the decoder's approach of mapping the output features to the class space can be understood as constructing a 2-layer network that transforms the number of channels into the number of classes. Its architecture is: 1×1 convolutional kernel + synchronous BatchNorm (w / ReLU) + 1×1 convolutional kernel.

[0067] Furthermore, such as Figure 3 As shown, the embodiments of this application can use the PaddlePaddle deep learning framework to build an automatic liver tumor segmentation model based on the Transformers architecture.

[0068] Step S205: Deep learning model training. This embodiment of the application can utilize supervised learning to train a deep learning model, wherein the model's input consists of a training set, a validation set, and their corresponding manually labeled data, randomly divided according to a set ratio.

[0069] For example, this embodiment of the application can be implemented in a Windows 10 operating system, and the relevant computing device is configured with an AMD Ryzen 7 5800H CPU with 16GB of memory, and two GPUs, respectively. GeForce RTX 3070 and Both the Tesla V100 (32GB of VRAM) and the PaddlePaddle deep learning framework are supported by CUDA acceleration. All operations are performed in the Python 3.8 programming language.

[0070] In this embodiment of the application, the model converges after 100,000 iterations and the loss value of the validation set tends to be stable and less than 0.001. At this point, training is stopped and the optimized model parameters are saved.

[0071] After testing, the deep learning model built in step S204 of this application embodiment was tested and executed 100,000 iterations. At the end of the iterations, the model converged and the loss value of the validation set remained constant, with no overfitting observed.

[0072] In the training of this application embodiment, key hyperparameters are optimized, wherein the optimizer uses momentum, the momentum factor is set to 0.9, the initial learning rate is set to 0.001, the weight_decay is 3.0×10-5, and the batch_size is 2.

[0073] Step S206: Model Testing and Evaluation. After training stops, this embodiment of the application can use an independent test set and corresponding labeled data to test the model. The result shows an mIoU of 0.9107, indicating that the model has excellent performance and strong generalization ability. The test results can be as follows: Figure 4 As shown.

[0074] Online segmentation, such as Figure 5 As shown, embodiments of this application may include the following steps:

[0075] Step S501: Input raw upper abdominal CT data. In actual implementation, this embodiment of the application can input raw upper abdominal CT data of a liver cancer patient, which can be in NIFIT format.

[0076] Step S502: Normalization Model. In this embodiment of the application, -1000Hu and 400Hu can be used as thresholds to normalize CT scans, i.e., a normalization operation is performed to preprocess the raw CT data of liver cancer patients.

[0077] Step S503: Data Dimension Conversion. Since the input channel of the deep learning framework in this embodiment is 2-dimensional, the data undergoes a corresponding format conversion, changing the CT data from NIFIT format to PNG format.

[0078] Step S504: Input into the trained model.

[0079] Step S505: Load the trained parameters.

[0080] Step S506: Automatically segment the liver tumor region.

[0081] Step S507: Data format conversion. To facilitate subsequent 3D reconstruction, this embodiment of the application can convert the data to DICOM format.

[0082] Step S508: 3D reconstruction using the MC algorithm. This embodiment of the application can be combined with the VTK library in Python for 3D reconstruction.

[0083] The automatic liver tumor segmentation method proposed in this application utilizes a Transformer neural network based on the Paddle-Paddle deep learning framework to train an automatic liver tumor segmentation model. This model obtains the automatic segmentation result of the liver tumor based on the characteristics of the liver tumor in the patient, ensuring efficiency while capturing as much effective contextual information as possible, thereby improving the accuracy of automatic liver tumor segmentation points. This solves the technical problems in related technologies where relying on expert experience and liver tumor feature selection for liver tumor segmentation is inefficient, and relying on deep learning algorithms for liver tumor segmentation has poor accuracy.

[0084] Next, the automatic liver tumor segmentation device according to the embodiments of this application is described with reference to the accompanying drawings.

[0085] Figure 6 This is a block diagram of an automatic liver tumor segmentation device according to an embodiment of this application.

[0086] like Figure 6 As shown, the automatic liver tumor segmentation device 10 includes: a data acquisition module 100, an extraction module 200, and a segmentation module 300.

[0087] Specifically, the acquisition module 100 is used to acquire CT data from liver cancer patients.

[0088] Extraction module 200 is used to extract at least one liver tumor feature from CT data of a liver cancer patient.

[0089] The segmentation module 300 is used to input at least one liver tumor feature into a preset automatic liver tumor segmentation model to obtain the automatic segmentation result of liver tumor in liver cancer patients. The preset automatic liver tumor segmentation model is trained by a preset Transformer neural network using the Paddle-Paddle deep learning framework in the backend.

[0090] Optionally, in one embodiment of this application, the automatic liver tumor segmentation device 10 further includes: a labeling module, a first modeling module, and a second modeling module.

[0091] The annotation module is used to collect CT data from multiple liver cancer patients, annotate the tumor regions in the CT data, and generate training and validation sets.

[0092] The first modeling module is used to build a preset Transformer neural network. The backend of the preset Transformer neural network uses the preset Paddle-Paddle deep learning framework to generate a deep learning model.

[0093] The second modeling module is used to train a deep learning model using the training and validation sets to generate a pre-defined automatic segmentation model for liver tumors.

[0094] Optionally, in one embodiment of this application, the annotation module is further used to preprocess the annotated CT data until the preset model input conditions are met, so as to obtain the preprocessed CT data.

[0095] Optionally, in one embodiment of this application, the annotation module includes: a slicing unit and a conversion unit.

[0096] The slicing unit is used to slice the labeled CT data, trimming the remaining parts outside the labels to obtain the sliced ​​CT data.

[0097] The conversion unit is used to convert the sliced ​​CT data into data in a preset format.

[0098] Optionally, in one embodiment of this application, the first modeling module includes: a mapping unit, an encoding unit, a sampling unit, and a modeling unit.

[0099] The mapping unit is used to segment data into multiple sub-images according to a preset format and map them into a linear embedding sequence.

[0100] The encoding unit is used to encode multiple sub-images using an encoder, wherein the Transformerlayer in the encoder consists of a multi-head self-attention mechanism and a multilayer perceptron.

[0101] The sampling unit is used to upsample the output bilinearly to the full image resolution using a decoder, followed by a classification layer with pixel-level cross-entropy loss.

[0102] The modeling unit is used to build an automatic liver tumor segmentation model based on the Transformers architecture using the PaddlePaddle deep learning framework.

[0103] It should be noted that the foregoing explanation of the automatic liver tumor segmentation method embodiment also applies to the automatic liver tumor segmentation device of this embodiment, and will not be repeated here.

[0104] According to the automatic liver tumor segmentation device proposed in this application, this embodiment can utilize a Transformer neural network based on the Paddle-Paddle deep learning framework to train an automatic liver tumor segmentation model. Based on the liver tumor characteristics of a liver cancer patient, it obtains the automatic segmentation result of the patient's liver tumor, ensuring efficiency while capturing as much effective contextual information as possible, thereby improving the accuracy of automatic liver tumor segmentation points. This solves the technical problems in related technologies where relying on expert experience and liver tumor feature selection for liver tumor segmentation is inefficient, and relying on deep learning algorithms for liver tumor segmentation has poor accuracy.

[0105] Figure 7 A schematic diagram of the structure of an electronic device provided in an embodiment of this application. The electronic device may include:

[0106] The memory 701, the processor 702, and the computer program stored on the memory 701 and executable on the processor 702.

[0107] When the processor 702 executes the program, it implements the automatic liver tumor segmentation method provided in the above embodiments.

[0108] Furthermore, electronic devices also include:

[0109] Communication interface 703 is used for communication between memory 701 and processor 702.

[0110] The memory 701 is used to store computer programs that can run on the processor 702.

[0111] The memory 701 may include high-speed RAM memory, and may also include non-volatile memory, such as at least one disk storage device.

[0112] If the memory 701, processor 702, and communication interface 703 are implemented independently, then the communication interface 703, memory 701, and processor 702 can be interconnected via a bus to complete communication between them. The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. Buses can be categorized as address buses, data buses, control buses, etc. For ease of representation, Figure 7 The bus is represented by a single thick line, but this does not mean that there is only one bus or one type of bus.

[0113] Optionally, in a specific implementation, if the memory 701, processor 702, and communication interface 703 are integrated on a single chip, then the memory 701, processor 702, and communication interface 703 can communicate with each other through an internal interface.

[0114] The processor 702 may be a central processing unit (CPU), an application specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of this application.

[0115] This embodiment also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the above-described automatic liver tumor segmentation method.

[0116] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of this application. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.

[0117] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this application, "N" means at least two, such as two, three, etc., unless otherwise explicitly specified.

[0118] Any process or method described in the flowchart or otherwise herein can be understood as representing a module, segment, or portion of code comprising one or more N executable instructions for implementing custom logic functions or processes, and the scope of the preferred embodiments of this application includes additional implementations in which functions may be performed not in the order shown or discussed, including substantially simultaneously or in reverse order depending on the functions involved, as should be understood by those skilled in the art to which embodiments of this application pertain.

[0119] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as a sequenced list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (such as a computer-based system, a processor-included system, or other system that can fetch and execute instructions from, an instruction execution system, apparatus, or device). For the purposes of this specification, "computer-readable medium" can be any means that can contain, store, communicate, propagate, or transmit programs for use by, or in conjunction with, an instruction execution system, apparatus, or device. More specific examples (a non-exhaustive list) of computer-readable media include: an electrical connection having one or more wires (electronic device), a portable computer disk drive (magnetic device), random access memory (RAM), read-only memory (ROM), erasable and editable read-only memory (EPROM or flash memory), fiber optic devices, and portable optical disc read-only memory (CDROM). Alternatively, the computer-readable medium may be paper or other suitable media on which the program can be printed, since the program can be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, interpreting, or otherwise processing as necessary, and then stored in a computer memory.

[0120] It should be understood that the various parts of this application can be implemented using hardware, software, firmware, or a combination thereof. In the above embodiments, the N steps or methods can be implemented using software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware as in another embodiment, it can be implemented using any one or a combination of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.

[0121] Those skilled in the art will understand that all or part of the steps of the methods in the above embodiments can be implemented by a program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, the program includes one or a combination of the steps of the method embodiments.

[0122] Furthermore, the functional units in the various embodiments of this application can be integrated into a processing module, or each unit can exist physically separately, or two or more units can be integrated into a module. The integrated module can be implemented in hardware or as a software functional module. If the integrated module is implemented as a software functional module and sold or used as an independent product, it can also be stored in a computer-readable storage medium.

[0123] The storage medium mentioned above can be a read-only memory, a disk, or an optical disk, etc. Although embodiments of this application have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting this application. Those skilled in the art can make changes, modifications, substitutions, and variations to the above embodiments within the scope of this application.

Claims

1. An automatic segmentation method for liver tumors, characterized in that, Includes the following steps: Collect CT data from liver cancer patients; Extract at least one liver tumor feature from the CT data of the liver cancer patient; as well as The at least one liver tumor feature is input into a preset automatic liver tumor segmentation model to obtain the automatic segmentation result of the liver tumor of the liver cancer patient; The method further includes, before inputting the at least one liver tumor feature into the preset automatic liver tumor segmentation model: collecting CT data from multiple liver cancer patients, labeling the tumor regions in the CT data, and generating a training set and a validation set; constructing a preset Transformer neural network, wherein the backend of the preset Transformer neural network adopts a preset Paddle-Paddle deep learning framework to generate a deep learning model; and training the deep learning model using the training set and the validation set to generate the preset automatic liver tumor segmentation model. The construction of the preset Transformer neural network includes: segmenting image data of a preset format into multiple sub-images and mapping the multiple sub-images into a linear embedding sequence, wherein the image data of the preset format is obtained from labeled CT data; encoding the linear embedding sequence using an encoder, wherein the Transformer layer in the encoder consists of a multi-head self-attention mechanism and a multilayer perceptron; decoding using a decoder and performing bilinear upsampling on the output features to restore the feature output to the full image resolution, followed by a classification layer with pixel-level cross-entropy loss.

2. The method according to claim 1, characterized in that, After labeling the tumor region in the CT data, the process also includes: The labeled CT data is preprocessed until the preset model input conditions are met, resulting in preprocessed CT data.

3. The method according to claim 2, characterized in that, The preprocessing of the labeled CT data includes: The annotated CT data slices are cropped, and the remaining parts outside the annotations are trimmed to obtain the sliced ​​CT data. The sliced ​​CT data is converted into image data in a preset format.

4. An automatic liver tumor segmentation device, characterized in that, include: The acquisition module is used to acquire CT data from liver cancer patients. An extraction module is used to extract at least one liver tumor feature from the CT data of the liver cancer patient; as well as The segmentation module is used to input the at least one liver tumor feature into a preset automatic liver tumor segmentation model to obtain the automatic segmentation result of the liver tumor of the liver cancer patient; The annotation module is used to collect CT data from multiple liver cancer patients, annotate the tumor regions in the CT data, and generate training and validation sets. The first modeling module is used to build a preset Transformer neural network, wherein the backend of the preset Transformer neural network adopts a preset Paddle-Paddle deep learning framework to generate a deep learning model. The second modeling module is used to train the deep learning model using the training set and the validation set to generate the preset automatic liver tumor segmentation model. The first modeling module includes: a mapping unit, used to segment image data of a preset format into multiple sub-images and map the multiple sub-images into a linear embedding sequence, wherein the image data of the preset format is obtained from labeled CT data; an encoding unit, used to encode the linear embedding sequence using an encoder, wherein the Transformer layer in the encoder consists of a multi-head self-attention mechanism and a multilayer perceptron; and a sampling unit, used to decode using a decoder and perform bilinear upsampling on the output features to restore the feature output to the full image resolution, followed by a classification layer with pixel-level cross-entropy loss.

5. The apparatus according to claim 4, characterized in that, The annotation module is further used to preprocess the annotated CT data until the preset model input conditions are met, so as to obtain the preprocessed CT data.

6. The apparatus according to claim 5, characterized in that, The annotation module includes: The slicing unit is used to slice the labeled CT data, trim the remaining parts outside the labels, and obtain the sliced ​​CT data. The conversion unit is used to convert the sliced ​​CT data into image data in a preset format.

7. An electronic device, characterized in that, include: The method includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the automatic segmentation method for liver tumors as described in any one of claims 1-3.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that, The program is executed by the processor to implement the automatic segmentation method for liver tumors as described in any one of claims 1-3.

Images