Tumor image segmentation methods, apparatus, devices and computer-readable storage media
By using segmentation networks and tumor lesion detection models, combined with feature images and spatial registration techniques, the accuracy problem of small-volume tumor image segmentation was solved, enabling precise identification and classification of small-volume tumors such as adrenal glands.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- PING AN TECH (SHENZHEN) CO LTD
- Filing Date
- 2023-04-28
- Publication Date
- 2026-06-30
AI Technical Summary
Existing technologies are prone to missed detections and false detections when segmenting images of small tumors, making it difficult to accurately identify small tumors such as those in the adrenal glands.
A pre-trained segmentation network is used to locate the target organ by using a segmentation mask of the target organ and surrounding organs. Combined with spatial registration of feature images and reference images, features are extracted and tumor lesions are identified by a convolutional neural network. The nnUNet network is used for preprocessing and automatic hyperparameter setting. Combined with a tumor lesion detection model, accurate segmentation is achieved.
It improves the segmentation accuracy of small-volume tumor images, reduces the segmentation difficulty, and ensures accurate identification and classification of tumor lesion areas.
Smart Images
Figure CN116563539B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the fields of artificial intelligence and digital healthcare, and in particular to a tumor image segmentation method, apparatus, electronic device, and computer-readable storage medium. Background Technology
[0002] With the rise of machine learning technology, models based on deep convolutional neural networks are increasingly being used in medical imaging examinations, such as the segmentation of images from CT (Computed Tomography), MRI (Magnetic Resonance Imaging), and OCT (Optical Coherence Tomography).
[0003] Doctors and researchers have explored using deep convolutional neural networks to differentiate between benign and malignant chest nodules and segment liver and pancreatic tumors on CT images, achieving promising results. However, the medical effects of various modalities differ significantly, including differences in study cohort size, image size and dimensions, resolution, and voxel intensity. For instance, the adrenal glands and most adrenal tumors are relatively small, making segmentation challenging. Using traditional radiomics methods for classifying and predicting adrenal tumor types is prone to missed and false positives. Summary of the Invention
[0004] This invention provides a tumor image segmentation method, apparatus, electronic device, and computer-readable storage medium, the main purpose of which is to improve the segmentation accuracy of small-volume tumor images.
[0005] To achieve the above objectives, the present invention provides a tumor image segmentation method, comprising:
[0006] Acquire the patient's medical image to be segmented, and use a pre-trained segmentation network to locate the target organ in the medical image to be segmented according to the preset segmentation mask of the target organ and the preset segmentation mask of the surrounding organs of the target organ, so as to obtain the target organ image.
[0007] The lesion image of the target organ is obtained as a reference image. The features of the reference image are extracted using the segmentation network to form a reference feature image. The features of the target organ image are extracted using the segmentation network to form a target feature image.
[0008] Based on the feature image and the reference feature image, the tumor lesion region in the target organ image is determined.
[0009] Optionally, the step of using a pre-trained segmentation network to identify the target organ in the medical image to be segmented based on a preset organ segmentation mask includes:
[0010] Using the segmentation network, the bounding box of the target organ is obtained from the medical image to be segmented based on the preset segmentation mask of the target organ;
[0011] Using the segmentation network, based on the preset segmentation mask of the surrounding organs of the target organ, the identification box of each surrounding organ in the medical image to be segmented is identified, and the identification box of each surrounding organ is removed from the medical image to be segmented to obtain the reference identification box of the target organ.
[0012] Based on the spatial relationship between the target organ and surrounding organs, the size of the overlapping area between the target organ's recognition frame and its reference recognition frame is adjusted, and the image corresponding to the adjusted overlapping area is used as the image of the target organ.
[0013] Optionally, the step of extracting features from the reference image using the segmentation network to form a reference feature image includes:
[0014] The reference image is convolved sequentially using each convolutional layer of the segmentation network to obtain the convolutional feature image corresponding to each layer;
[0015] The attention weights for each of the convolutional feature images are calculated using the normalization layer of the segmentation network;
[0016] Multiply the convolutional feature image output by each convolutional layer by the corresponding attention weight to obtain the weighted convolutional feature image corresponding to each convolutional layer;
[0017] The weighted convolutional feature images corresponding to each convolutional layer are merged to obtain the reference feature image.
[0018] Optionally, the step of extracting features from the reference image using the segmentation network to form a reference feature image includes:
[0019] The reference image is downsampled using a predetermined number of convolutional layers of the segmentation network to obtain a downsampled feature image;
[0020] The remaining convolutional layers of the segmentation network are used to perform an upsampling operation on the downsampled feature image to obtain an upsampled feature image;
[0021] The downsampled feature images and upsampled feature images of the same size are merged to generate at least one reference feature image of different sizes.
[0022] Optionally, determining the tumor lesion region in the target organ image based on the feature image and the reference feature image includes:
[0023] Identify the anatomical points corresponding to the reference feature image, and use the anatomical points as the origin of the spatial coordinates;
[0024] Spatial registration is performed on the target feature image and the reference feature image based on the spatial coordinate origin;
[0025] A pre-trained tumor lesion detection model is used to determine the reference tumor lesion identification box from the reference feature image;
[0026] The region where the reference tumor lesion identification box overlaps with the target feature image is taken as the tumor lesion region in the target organ image.
[0027] Optionally, after determining the tumor lesion region in the target organ image based on the target feature image and the reference feature image, the method further includes:
[0028] The segmentation network is used to extract tumor voxel features from the tumor lesion region, and the probability value between the tumor voxel features and each preset tumor classification label is calculated. The tumor classification label corresponding to the highest probability value is selected as the tumor classification result of the patient.
[0029] Optionally, the step of extracting tumor voxel features of the tumor lesion region using the segmentation network includes:
[0030] The encoder of the segmentation network is used to perform a preset number of downsampling operations on the tumor lesion area to obtain a downsampled image;
[0031] The voxel feature values of the downsampled image are calculated sequentially using convolutional layers of different preset depths of the segmentation network to obtain the downsampled voxel feature matrix corresponding to each convolutional layer.
[0032] The decoder of the segmentation network performs the preset number of upsampling operations on the downsampled voxel feature matrix from the deepest convolutional layer, and connects it with the downsampled voxel feature matrix corresponding to the convolutional layer of the same depth to obtain the fused voxel feature matrix of the corresponding convolutional layer.
[0033] The fusion voxel feature matrices corresponding to each convolutional layer of the segmentation network are concatenated using the fully connected layers of the segmentation network to obtain the tumor voxel features of the tumor lesion region.
[0034] To address the above problems, the present invention also provides a tumor image segmentation device, the device comprising:
[0035] The target organ image segmentation module is used to acquire the patient's medical image to be segmented. Using a pre-trained segmentation network, the target organ in the medical image to be segmented is located according to the preset segmentation mask of the target organ and the preset segmentation mask of the surrounding organs of the target organ, so as to obtain the target organ image.
[0036] The lesion feature image generation module is used to acquire the lesion image of the target organ as a reference image, extract the features of the reference image using the segmentation network to form a reference feature image, and extract the features of the target organ image using the segmentation network to form a target feature image.
[0037] The lesion image segmentation module is used to determine the tumor lesion region in the target organ image based on the feature image and the reference feature image.
[0038] To address the above problems, the present invention also provides an electronic device, the electronic device comprising:
[0039] Memory, storing at least one computer program; and
[0040] The processor executes the program stored in the memory to implement the tumor image segmentation method described above.
[0041] To address the aforementioned problems, the present invention also provides a computer-readable storage medium storing at least one computer program, which is executed by a processor in an electronic device to implement the tumor image segmentation method described above.
[0042] This invention enables rapid identification of target organs in medical images to be segmented by using a pre-defined organ segmentation mask. Furthermore, by utilizing the target feature image of the target organ and a reference feature image composed of the lesion image of the target organ, the tumor lesion region in the target organ image is accurately identified. For tumors with relatively small volume, the above method can accurately limit the area of the medical image to be segmented that needs to be analyzed, reducing the difficulty of image segmentation and improving the accuracy of segmenting small-volume tumor images. Attached Figure Description
[0043] Figure 1 This is a schematic flowchart of a tumor image segmentation method provided in an embodiment of the present invention;
[0044] Figure 2 This is a detailed implementation flowchart of one step in a tumor image segmentation method provided in an embodiment of the present invention;
[0045] Figure 3 This is a detailed implementation flowchart of another step in the tumor image segmentation method provided in an embodiment of the present invention;
[0046] Figure 4 This is a detailed implementation flowchart of another step in the tumor image segmentation method provided in an embodiment of the present invention;
[0047] Figure 5 This is a functional block diagram of a tumor image segmentation device provided in an embodiment of the present invention;
[0048] Figure 6 This is a schematic diagram of the structure of an electronic device for implementing the tumor image segmentation method according to an embodiment of the present invention.
[0049] The realization of the objective, functional features and advantages of the present invention will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation
[0050] It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.
[0051] This application provides a tumor image segmentation method. The execution entity of the tumor image segmentation method includes, but is not limited to, at least one of the following: a server, a terminal, or other electronic devices that can be configured to execute the method provided in this application. In other words, the tumor image segmentation method can be executed by software or hardware installed on a terminal device or a server device, and the software can be a blockchain platform. The server can be an independent server or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks (CDNs), and big data and artificial intelligence platforms.
[0052] Reference Figure 1 The diagram shown is a flowchart illustrating a tumor image segmentation method according to an embodiment of the present invention. In this embodiment, the tumor image segmentation method includes:
[0053] S1. Acquire the patient's medical image to be segmented, and use a pre-trained segmentation network to locate the target organ in the medical image to be segmented according to the preset segmentation mask of the target organ and the preset segmentation mask of the surrounding organs of the target organ, so as to obtain the target organ image.
[0054] In this embodiment of the invention, the CT scan image of an adrenal patient is used as the medical image to be segmented. It should be noted that the medical image to be segmented includes, but is not limited to, adrenal tumor medical images in any of the three phases: arterial phase, venous phase, and plain scan phase.
[0055] In this embodiment of the invention, the medical images of patients to be segmented can be obtained from a specific authorized patient information resource pool. It is necessary to perform a desensitization operation on the patient's sensitive personal information to ensure that the patient's personal information is not infringed or leaked.
[0056] In this embodiment of the invention, the pre-trained segmentation network can be constructed based on the nnUNet convolutional neural network. The nnUNet network includes a 2D UNet, a 3D UNet, and two cascaded 3D UNets. The 2D UNet is used to generate a coarse segmentation result for the medical image to be segmented, and the 3D UNet is used to further refine the coarse segmentation result. Compared to traditional UNet convolutional neural networks, the nnUNet convolutional neural network places greater emphasis on the preprocessing of the medical image to be segmented during pre-training. Furthermore, nnUNet can automatically set hyperparameters, such as training batch size, image block size, and downsampling times. Finally, cross-entropy loss and Dice loss are used as loss functions during training until the segmentation network achieves the preset training objective.
[0057] In this embodiment of the invention, the target organ is taken as an example of the adrenal gland. The segmentation mask of the preset organ includes, but is not limited to, the segmentation mask of the target organ and the masks of the surrounding organs of the target organ, such as the kidney segmentation mask, spleen segmentation mask, and pancreas segmentation mask. Further, the target organ can be further subdivided into the left adrenal gland and the right adrenal gland, corresponding to the left adrenal gland mask and the right adrenal gland mask, respectively.
[0058] For details, please refer to Figure 2 As shown, the step of using a pre-trained segmentation network to identify the target organ in the medical image to be segmented based on a preset organ segmentation mask, and obtaining the target organ image, includes:
[0059] S11. Using the segmentation network, the bounding box of the target organ is obtained from the medical image to be segmented according to the preset segmentation mask of the target organ.
[0060] S12. Using the segmentation network, based on the preset segmentation mask of the peripheral organs of the target organ, identify the bounding box of each peripheral organ in the medical image to be segmented, remove the bounding box of each peripheral organ from the medical image to be segmented, and obtain the reference bounding box of the target organ.
[0061] S13. Based on the spatial relationship between the target organ and surrounding organs, adjust the size of the overlapping area between the target organ's recognition frame and the target organ's reference recognition frame, and use the image corresponding to the adjusted overlapping area as the target organ image.
[0062] In this embodiment of the invention, the segmentation mask of the preset organ can be used to block non-monitored, non-target organs in the medical image to be segmented, thereby controlling the area and processing process that need to be processed in the medical image to be segmented.
[0063] S2. Obtain the lesion image of the target organ as a reference image, use the segmentation network to extract features from the reference image to form a reference feature image, and use the segmentation network to extract features from the target organ image to form a target feature image.
[0064] In this embodiment of the invention, the lesion images of the target organ include lesion images of various functional adrenal tumors of different sizes, such as primary aldosteronism, Cushing's syndrome, and pheochromocytoma.
[0065] For details, please refer to Figure 3 As shown, the step of extracting features from the reference image using the segmentation network to form a reference feature image includes:
[0066] S21. Sequentially use each convolutional layer of the segmentation network to perform convolution calculations on the reference image to obtain the convolutional feature image corresponding to each layer;
[0067] S22. Calculate the attention weights for each of the convolutional feature images using the normalization layer of the segmentation network;
[0068] S23. Multiply the convolutional feature image output by each convolutional layer with the corresponding attention weight to obtain the weighted convolutional feature image corresponding to each convolutional layer.
[0069] S24. Merge the weighted convolutional feature images corresponding to each convolutional layer to obtain the reference feature image.
[0070] In this embodiment of the invention, the encoder of the segmentation network includes multiple convolutional layers. The kernel size, number of channels, and stride of each convolutional layer can be the same or different. By setting different convolutional kernels, the perceptual field of feature extraction can be effectively improved, which is beneficial to improving the accuracy of adrenal lesion identification. The feature images extracted by each convolutional layer may be of different sizes. Feature images of different sizes can be feature images of different pixels. For example, a feature image with 500×500 pixels and a feature image with 1000×1000 pixels are feature images of different sizes. Therefore, on the original architecture of nnUNet, an attention mechanism is introduced. Through the normalization layer, the attention weights of each convolutional layer are normalized, which preserves the differences of each convolutional layer and facilitates the fusion of the convolutional feature images output by each convolutional layer in the subsequent process.
[0071] In another optional embodiment of the present invention, reference feature images of different sizes of the reference image can be extracted by performing upsampling or downsampling operations on the reference image.
[0072] Specifically, the step of extracting features from the reference image using the segmentation network to compose a reference feature image includes:
[0073] The reference image is downsampled using a predetermined number of convolutional layers of the segmentation network to obtain a downsampled feature image;
[0074] The remaining convolutional layers of the segmentation network are used to perform an upsampling operation on the downsampled feature image to obtain an upsampled feature image;
[0075] The downsampled feature images and upsampled feature images of the same size are merged to generate at least one reference feature image of different sizes.
[0076] In this embodiment of the invention, the preset number can be one or more, or it can be determined according to the actual number of convolutional layers in the segmentation network. For example, the preset number can be half the number of convolutional layers.
[0077] In this embodiment of the invention, the method for extracting features from the target organ image using the segmentation network to form a target feature image can be the same as the method for extracting features from the reference image using the segmentation network to form a reference feature image, and will not be described again here.
[0078] S3. Determine the tumor lesion region in the target organ image based on the target feature image and the reference feature image;
[0079] In this embodiment of the invention, by spatially comparing the target feature image and the reference feature image, the tumor lesion area in the target organ image can be quickly located.
[0080] For details, please refer to Figure 4 The step of determining the tumor lesion region in the target organ image based on the target feature image and the reference feature image includes:
[0081] S31. Identify the anatomical point corresponding to the reference feature image, and use the anatomical point as the origin of the spatial coordinates;
[0082] S32. Spatial registration is performed on the target feature image and the reference feature image based on the spatial coordinate origin;
[0083] S33. Use a pre-trained tumor lesion detection model to determine the reference tumor lesion identification box from the reference feature image;
[0084] S34. The overlapping area between the reference tumor lesion identification box and the target feature image is taken as the tumor lesion area in the target organ image.
[0085] It is understood that the target feature image and the reference feature image are of different sizes. Therefore, they can be registered using the dissecting points to ensure that they achieve spatial consistency and equal voxel spacing at the same dissecting point.
[0086] In this embodiment of the invention, the pre-trained tumor lesion detection model is determined by training multiple adrenal images with labeled adrenal lesions using a 2D convolutional neural network. The regions selected by the adrenal lesion identification boxes determined from the reference feature images do not necessarily all contain adrenal lesions. Therefore, it is necessary to filter each adrenal lesion identification box based on its adrenal lesion probability, deleting those with a probability less than a preset threshold. The adrenal lesion probability is the probability that the region selected by the adrenal lesion identification box contains an adrenal lesion.
[0087] Furthermore, after the segmentation of the lesion region of the medical image to be segmented is completed, the segmentation network can be used to extract the tumor voxel features of the tumor lesion region, calculate the probability value between the tumor voxel features and each preset tumor classification label, and select the tumor classification label corresponding to the highest probability value as the tumor classification result of the patient.
[0088] In this embodiment of the invention, the encoder and decoder of the segmentation network can be used to extract tumor voxel features of the tumor lesion region.
[0089] Specifically, the extraction of tumor voxel features from the tumor lesion region using the segmentation network includes:
[0090] The encoder of the segmentation network is used to perform a preset number of downsampling operations on the tumor lesion area to obtain a downsampled image;
[0091] The voxel feature values of the downsampled image are calculated sequentially using convolutional layers of different preset depths of the segmentation network to obtain the downsampled voxel feature matrix corresponding to each convolutional layer.
[0092] The decoder of the segmentation network performs the preset number of upsampling operations on the downsampled voxel feature matrix from the deepest convolutional layer, and connects it with the downsampled voxel feature matrix corresponding to the convolutional layer of the same depth to obtain the fused voxel feature matrix of the corresponding convolutional layer.
[0093] The fusion voxel feature matrices corresponding to each convolutional layer of the segmentation network are concatenated using the fully connected layers of the segmentation network to obtain the tumor voxel features of the tumor lesion region.
[0094] In this embodiment of the invention, the preset convolutional layers of different depths refer to convolutional layers that constitute the encoder in the segmentation network with different kernel sizes, number of channels, and strides.
[0095] In this embodiment of the invention, the activation function can be a common linear activation function, such as Sigmoid, Tanh, or ReLU.
[0096] In an optional embodiment of the present invention, the probability value can be calculated using the following activation function:
[0097]
[0098] in, tumor voxel characteristics and tumor classification labels The probability values between Tumor classification labels The weight vector, To find the transpose operator, To find the expected operator, The number of preset tumor classification labels.
[0099] In this embodiment of the invention, the preset tumor classification labels include, but are not limited to, primary aldosteronism labels, Cushing's syndrome labels, and pheochromocytoma labels.
[0100] This invention enables rapid identification of target organs in medical images to be segmented using a pre-defined organ segmentation mask. Furthermore, by utilizing the feature image of the target organ and a reference feature image composed of lesion images of the target organ, the tumor lesion region in the target organ image is accurately identified. For tumors with relatively small volumes, the above method can precisely limit the area of the medical image to be segmented that needs to be analyzed, reducing the segmentation difficulty and ensuring the accuracy of subsequent tumor classification results for the tumor lesion region.
[0101] like Figure 5 The diagram shown is a functional block diagram of a tumor image segmentation device provided in an embodiment of the present invention.
[0102] The tumor image segmentation device 100 of the present invention can be installed in an electronic device. Depending on the functions it performs, the tumor image segmentation device 100 includes: a target organ image segmentation module 101, a lesion feature image generation module 102, and a lesion image segmentation module 103. The module described in this invention can also be referred to as a unit, which refers to a series of computer program segments that can be executed by the processor of an electronic device and can perform a fixed function, and which are stored in the memory of the electronic device.
[0103] In this embodiment, the functions of each module / unit are as follows:
[0104] The target organ image segmentation module 101 is used to acquire the patient's medical image to be segmented, and to locate the target organ in the medical image to be segmented by using a pre-trained segmentation network according to the preset segmentation mask of the target organ and the preset segmentation mask of the surrounding organs of the target organ, so as to obtain the target organ image.
[0105] The lesion feature image generation module 102 is used to acquire the lesion image of the target organ as a reference image, extract the features of the reference image using the segmentation network to form a reference feature image, and extract the features of the target organ image using the segmentation network to form a target feature image.
[0106] The lesion image segmentation module 103 is used to determine the tumor lesion region in the target organ image based on the target feature image and the reference feature image.
[0107] In detail, each module in the tumor image segmentation device 100 described in this embodiment of the invention employs the same methods as described above during use. Figures 1 to 4 The method uses the same techniques as the tumor image segmentation method described above and can produce the same technical effects, so it will not be repeated here.
[0108] like Figure 6 The diagram shown is a schematic diagram of an electronic device for implementing a tumor image segmentation method according to an embodiment of the present invention.
[0109] The electronic device 1 may include a processor 10, a memory 11 and a bus, and may also include a computer program, such as a tumor image segmentation program, stored in the memory 11 and capable of running on the processor 10.
[0110] The memory 11 includes at least one type of readable storage medium, including flash memory, portable hard drive, multimedia card, card-type memory (e.g., SD or DX memory), magnetic memory, magnetic disk, optical disk, etc. In some embodiments, the memory 11 can be an internal storage unit of the electronic device 1, such as a portable hard drive. In other embodiments, the memory 11 can be an external storage device of the electronic device 1, such as a plug-in portable hard drive, smart media card (SMC), secure digital card (SD), flash card, etc., equipped on the electronic device 1. Furthermore, the memory 11 can include both internal and external storage units of the electronic device 1. The memory 11 can be used not only to store application software and various types of data installed on the electronic device 1, such as the code of a tumor image segmentation program, but also to temporarily store data that has been output or will be output.
[0111] In some embodiments, the processor 10 may be composed of integrated circuits, such as a single packaged integrated circuit or multiple integrated circuits with the same or different functions, including combinations of one or more central processing units (CPUs), microprocessors, digital processing chips, graphics processors, and various control chips. The processor 10 is the control unit of the electronic device, connecting various components of the entire electronic device through various interfaces and lines. It executes programs or modules (such as tumor image segmentation programs) stored in the memory 11, and calls data stored in the memory 11 to perform various functions of the electronic device 1 and process data.
[0112] The bus can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. This bus can be divided into an address bus, a data bus, a control bus, etc. The bus is configured to enable communication between the memory 11 and at least one processor 10, etc.
[0113] Figure 6 Only electronic devices with components are shown; it will be understood by those skilled in the art that... Figure 6The structure shown does not constitute a limitation on the electronic device 1, and may include fewer or more components than shown, or combine certain components, or have different component arrangements.
[0114] For example, although not shown, the electronic device 1 may also include a power supply (such as a battery) to power the various components. Preferably, the power supply can be logically connected to the at least one processor 10 through a power management device, thereby enabling functions such as charging management, discharging management, and power consumption management. The power supply may also include one or more DC or AC power supplies, recharging devices, power fault detection circuits, power converters or inverters, power status indicators, and other arbitrary components. The electronic device 1 may also include various sensors, Bluetooth modules, Wi-Fi modules, etc., which will not be described in detail here.
[0115] Furthermore, the electronic device 1 may also include a network interface. Optionally, the network interface may include a wired interface and / or a wireless interface (such as a Wi-Fi interface, a Bluetooth interface, etc.), which is typically used to establish communication connections between the electronic device 1 and other electronic devices.
[0116] Optionally, the electronic device 1 may further include a user interface, which may be a display, an input unit (such as a keyboard), and optionally, a standard wired interface or a wireless interface. Optionally, in some embodiments, the display may be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, or an OLED (Organic Light-Emitting Diode) touchscreen, etc. The display may also be appropriately referred to as a screen or display unit, used to display information processed in the electronic device 1 and to display a visual user interface.
[0117] It should be understood that the embodiments described are for illustrative purposes only and are not limited to this structure in the scope of the patent application.
[0118] The tumor image segmentation program stored in the memory 11 of the electronic device 1 is a combination of multiple instructions, which, when run in the processor 10, can achieve the following:
[0119] Acquire the patient's medical image to be segmented, and use a pre-trained segmentation network to locate the target organ in the medical image to be segmented according to the preset segmentation mask of the target organ and the preset segmentation mask of the surrounding organs of the target organ, so as to obtain the target organ image.
[0120] The lesion image of the target organ is obtained as a reference image. The features of the reference image are extracted using the segmentation network to form a reference feature image. The features of the target organ image are extracted using the segmentation network to form a target feature image.
[0121] Based on the target feature image and the reference feature image, the tumor lesion region in the target organ image is determined.
[0122] Furthermore, if the modules / units integrated in the electronic device 1 are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. The computer-readable storage medium can be volatile or non-volatile. For example, the computer-readable medium may include: any entity or device capable of carrying the computer program code, a recording medium, a USB flash drive, a portable hard drive, a magnetic disk, an optical disk, a computer memory, or a read-only memory (ROM).
[0123] The present invention also provides a computer-readable storage medium storing a computer program, which, when executed by a processor of an electronic device, can perform the following:
[0124] Acquire the patient's medical image to be segmented, and use a pre-trained segmentation network to locate the target organ in the medical image to be segmented according to the preset segmentation mask of the target organ and the preset segmentation mask of the surrounding organs of the target organ, so as to obtain the target organ image.
[0125] The lesion image of the target organ is obtained as a reference image. The features of the reference image are extracted using the segmentation network to form a reference feature image. The features of the target organ image are extracted using the segmentation network to form a target feature image.
[0126] Based on the target feature image and the reference feature image, the tumor lesion region in the target organ image is determined.
[0127] Furthermore, the functional modules in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or in the form of hardware plus software functional modules.
[0128] It will be apparent to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above, and that the present invention can be implemented in other specific forms without departing from the spirit or essential characteristics of the present invention.
[0129] Therefore, the embodiments should be considered exemplary and non-limiting in all respects, and the scope of the invention is defined by the appended claims rather than the foregoing description. Thus, all variations falling within the meaning and scope of equivalents of the claims are intended to be embraced within the invention. No appended diagram markings in the claims should be construed as limiting the scope of the claims.
[0130] The blockchain referred to in this invention is a novel application model of computer technologies such as distributed data storage, peer-to-peer transmission, consensus mechanisms, and encryption algorithms. Essentially, a blockchain is a decentralized database, a chain of data blocks linked together using cryptographic methods. Each data block contains information about a batch of network transactions, used to verify the validity of the information (anti-counterfeiting) and generate the next block. A blockchain can include an underlying blockchain platform, a platform product service layer, and an application service layer.
[0131] The embodiments of this application can acquire and process relevant data based on holographic projection technology. Artificial Intelligence (AI) is the theory, method, technology, and application system that uses digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use that knowledge to obtain optimal results.
[0132] Furthermore, it is clear that the word "comprising" does not exclude other units or steps, and the singular does not exclude the plural. Multiple units or devices recited in a system claim may also be implemented by a single unit or device through software or hardware. The term "second class" is used to indicate names and does not indicate any specific order.
[0133] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention.
Claims
1. A tumor image segmentation method, characterized in that, The method includes: Acquire the patient's medical image to be segmented, and use a pre-trained segmentation network to locate the target organ in the medical image to be segmented according to the preset segmentation mask of the target organ and the preset segmentation mask of the surrounding organs of the target organ, so as to obtain the target organ image. The lesion image of the target organ is obtained as a reference image. The features of the reference image are extracted using the segmentation network to form a reference feature image. The features of the target organ image are extracted using the segmentation network to form a target feature image. Based on the target feature image and the reference feature image, the tumor lesion region in the target organ image is determined; The step of locating the target organ in the medical image to be segmented based on a preset segmentation mask for the target organ and a preset segmentation mask for the surrounding organs of the target organ includes: Using the segmentation network, the bounding box of the target organ is obtained from the medical image to be segmented based on the preset segmentation mask of the target organ; Using the segmentation network, based on the preset segmentation mask of the surrounding organs of the target organ, the identification box of each surrounding organ in the medical image to be segmented is identified, and the identification box of each surrounding organ is removed from the medical image to be segmented to obtain the reference identification box of the target organ. Based on the spatial relationship between the target organ and surrounding organs, the size of the overlapping area between the target organ's recognition frame and its reference recognition frame is adjusted, and the image corresponding to the adjusted overlapping area is used as the image of the target organ.
2. The tumor image segmentation method as described in claim 1, characterized in that, The step of extracting features from the reference image using the segmentation network to form a reference feature image includes: The reference image is convolved sequentially using each convolutional layer of the segmentation network to obtain the convolutional feature image corresponding to each layer; The attention weights for each of the convolutional feature images are calculated using the normalization layer of the segmentation network; Multiply the convolutional feature image output by each convolutional layer by the corresponding attention weight to obtain the weighted convolutional feature image corresponding to each convolutional layer; The weighted convolutional feature images corresponding to each convolutional layer are merged to obtain the reference feature image.
3. The tumor image segmentation method as described in claim 1, characterized in that, The step of extracting features from the reference image using the segmentation network to form a reference feature image includes: The reference image is downsampled using a predetermined number of convolutional layers of the segmentation network to obtain a downsampled feature image; The remaining convolutional layers of the segmentation network are used to perform an upsampling operation on the downsampled feature image to obtain an upsampled feature image; The downsampled feature images and upsampled feature images of the same size are merged to generate at least one reference feature image of different sizes.
4. The tumor image segmentation method as described in claim 1, characterized in that, The step of determining the tumor lesion region in the target organ image based on the target feature image and the reference feature image includes: Identify the anatomical points corresponding to the reference feature image, and use the anatomical points as the origin of the spatial coordinates; Spatial registration is performed on the target feature image and the reference feature image based on the spatial coordinate origin; A pre-trained tumor lesion detection model is used to determine the reference tumor lesion identification box from the reference feature image; The overlapping area between the reference tumor lesion identification box and the target feature image is taken as the tumor lesion area in the target organ image.
5. The tumor image segmentation method as described in claim 1, characterized in that, After determining the tumor lesion region in the target organ image based on the target feature image and the reference feature image, the method further includes: The segmentation network is used to extract tumor voxel features from the tumor lesion region, and the probability value between the tumor voxel features and each preset tumor classification label is calculated. The tumor classification label corresponding to the highest probability value is selected as the tumor classification result of the patient.
6. The tumor image segmentation method as described in claim 5, characterized in that, The step of extracting tumor voxel features from the tumor lesion region using the segmentation network includes: The encoder of the segmentation network is used to perform a preset number of downsampling operations on the tumor lesion area to obtain a downsampled image; The voxel feature values of the downsampled image are calculated sequentially using convolutional layers of different preset depths of the segmentation network to obtain the downsampled voxel feature matrix corresponding to each convolutional layer. The decoder of the segmentation network performs the preset number of upsampling operations on the downsampled voxel feature matrix from the deepest convolutional layer, and connects it with the downsampled voxel feature matrix corresponding to the convolutional layer of the same depth to obtain the fused voxel feature matrix of the corresponding convolutional layer. The fusion voxel feature matrices corresponding to each convolutional layer of the segmentation network are concatenated using the fully connected layers of the segmentation network to obtain the tumor voxel features of the tumor lesion region.
7. A tumor image segmentation apparatus for implementing the tumor image segmentation method as described in any one of claims 1 to 6, characterized in that, The device includes: The target organ image segmentation module is used to acquire the patient's medical image to be segmented. Using a pre-trained segmentation network, the target organ in the medical image to be segmented is located according to the preset segmentation mask of the target organ and the preset segmentation mask of the surrounding organs of the target organ, so as to obtain the target organ image. The lesion feature image generation module is used to acquire the lesion image of the target organ as a reference image, extract the features of the reference image using the segmentation network to form a reference feature image, and extract the features of the target organ image using the segmentation network to form a feature image. The lesion image segmentation module is used to determine the tumor lesion region in the target organ image based on the target feature image and the reference feature image.
8. An electronic device, characterized in that, The electronic device includes: At least one processor; and, A memory communicatively connected to the at least one processor; wherein, The memory stores a computer program that can be executed by the at least one processor to enable the at least one processor to perform the tumor image segmentation method as described in any one of claims 1 to 6.
9. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements the tumor image segmentation method as described in any one of claims 1 to 6.