Segmentation method and application of organ lesion region of interest

Abstract

The invention relates to the field of radiomics, in particular to an organ focus ROI segmentation method. According to the method, multiple organs are segmented at the same time based on a deep learning segmentation algorithm, clearer and more reasonable boundaries exist among the multiple organs through boundary constraint, and target detection and a segmentation network are used for accurately segmenting a focus part; then expanding a certain width according to a region growing algorithm to obtain a region of interest around the target focus, performing false positive suppression through a Resnet classification network in the expansion process so as to constrain the contour of the target organ, and automatically removing regions outside the target organ or non-target organ tissues or foreign matters; the lesion edge obtained through segmentation is more accurate, and a certain degree of false positive can be reduced.

Description

Technical Field

[0001] The present invention relates to the field of radiomics, and particularly to a method for segmenting regions of interest in organ lesions and its applications. Background Art

[0002] With the development of technologies such as radiomics, more and more quantitative information in images has been mined for disease diagnosis, treatment, and prediction. Before obtaining quantitative image information, it is necessary to segment the region of interest, and then extract the quantitative information of the image after segmentation. In order to obtain the image information of a certain fixed location in the target region, image segmentation is mostly based on the contour of the target lesion and uses a region growing algorithm for expansion and contraction. One of the disadvantages of this method is that during the region growing process, it may exceed the boundary of the target region organ, and it is necessary to manually erase the region that exceeds the boundary. Another disadvantage of this method is that it cannot bypass the regions in the organ that are not relevant to the target region, such as blood vessels, metal shadows, etc. If the researcher does not remove the region outside the target organ or extracts the information of non-target organ tissues or foreign objects (such as metal), the extracted feature information will have a large deviation and cannot truly reflect the biological information of the region of interest.

[0003] Therefore, there is an urgent need to develop a method for segmenting the ROI region of lesions. Summary of the Invention

[0004] The present invention aims to solve at least one of the technical problems in the related art to some extent.

[0005] To this end, an object of the present invention is to provide a method for segmenting regions of interest in lesions within an organ, and the segmentation method includes:

[0006] (a) Obtain a medical image containing the organ;

[0007] (b) Use image segmentation method A to obtain the boundary between the target organ and adjacent organs, so as to obtain the target organ region;

[0008] (c) Use a predicted lesion tool to obtain a three-dimensional stereoscopic rectangular box 1 of the lesion in the target organ region;

[0009] (d) Perform false positive suppression on the three-dimensional stereoscopic rectangular box 1 of the lesion to obtain a three-dimensional stereoscopic rectangular box 2 of the lesion;

[0010] (e) Use image segmentation method B to segment the three-dimensional stereoscopic rectangular box 2 of the lesion to obtain a three-dimensional stereoscopic rectangular box 3 of the lesion;

[0011] (f) Use a region growing algorithm to expand the three-dimensional stereoscopic rectangular box 3 of the lesion to obtain the region of interest in the lesion within the organ.

[0012] The present invention first simultaneously segments multiple organs based on a deep learning segmentation algorithm, and makes the boundaries between multiple organs clearer and more reasonable through boundary constraints. At the same time, considering the irregular shapes of organs in patients who have undergone partial surgical resection or organ atrophy, targeted data augmentation is performed to make the segmentation results of the segmentation model for irregular target organs more robust. It can ensure that the segmentation results of the target organs are not affected by lesions, atrophy, surgery, etc. After segmenting the target organ and the surrounding organ regions, a target detection and segmentation network is used to accurately segment the lesion part, and then an interested region around the target lesion is obtained by expanding a certain width according to the region growing algorithm. During the expansion process, the contour of the target organ is used for constraint, and the expansion regions outside the contour of the target organ are automatically removed. For non-target organ tissues such as blood vessels or metal shadows inside the target organ encountered during the expansion process, they are automatically removed by setting a density difference threshold. During this process, as long as it is inside the target organ, the expansion process is not interfered by the metal shadow and stops, so as to truly reflect the biological information of the interested region.

[0013] In some embodiments of the present invention, the medical image includes at least one selected from CT images, MRI images, and ultrasound images.

[0014] In some embodiments of the present invention, the image segmentation method A and the image segmentation method B include at least one of a threshold-based segmentation method, a region-based segmentation method, an edge-based segmentation method, and a deep learning-based segmentation method.

[0015] In some embodiments of the present invention, the image segmentation method A and the image segmentation method B are the same.

[0016] In some embodiments of the present invention, the image segmentation method A and the image segmentation method B are different.

[0017] In some embodiments of the present invention, the image segmentation method A and the image segmentation method B are deep learning-based segmentation methods.

[0018] In some embodiments of the present invention, the deep learning segmentation algorithm includes at least one selected from 3DUnet and nnUnet.

[0019] In some embodiments of the present invention, the deep learning segmentation algorithms adopted by the image segmentation method A and the image segmentation method B are both 3DUnet.

[0020] In some embodiments of the present invention, in step (b), before training the deep learning segmentation algorithm, the proportion of training irregular target organs is increased by flipping, adding noise, intensity normalization, and rotating the target organs in the medical images in the training set, and data augmentation processing is performed.

[0021] In some embodiments of the present invention, both the image segmentation method A and / or the image segmentation method B use a loss function for optimization during the training process of the segmentation method.

[0022] In some embodiments of the present invention, the loss function includes at least one selected from BoundaryLoss and FocolLoss, and preferably the loss function BoundaryLoss.

[0023] In some embodiments of the present invention, in step (b), the method for obtaining the boundary between the target organ and the adjacent organ includes: marking the region of the target organ and the adjacent organ regions, and using the loss function for optimization during the training process of the deep learning segmentation algorithm to obtain the boundary between the target organ and the adjacent organ.

[0024] In some embodiments of the present invention, the predicted lesion tool is a prediction model capable of predicting a rectangular bounding box of the lesion.

[0025] In some embodiments of the present invention, the prediction model includes at least one selected from the Anchor-Free model and the Anchor-Based model.

[0026] In some embodiments of the present invention, the Anchor-Free model includes at least one selected from FCOS, YOLO, and FSAF.

[0027] In some embodiments of the present invention, the Anchor-Based model includes at least one selected from RetinaNet, SSD, and Faster RCNN.

[0028] In some embodiments of the present invention, in step (d), the method for false positive suppression includes: classifying whether the three-dimensional rectangular box 1 of the lesion is a lesion through a classification network, and removing non-lesion regions.

[0029] In some embodiments of the present invention, the classification network includes at least one selected from the Resnet, DenseNet, and ResNeXt classification networks.

[0030] In some embodiments of the present invention, in step (f), when expanding the three-dimensional rectangular box 3 of the lesion using the region growing algorithm, the contour of the target organ is used for constraint, and the regions outside the contour of the target organ are removed.

[0031] In some embodiments of the present invention, in step (f), when expanding the three-dimensional rectangular box 3 of the lesion using the region growing algorithm, the density of the parenchymal part and the non-parenchymal part of the target organ is compared, and a density difference threshold is set to remove the non-parenchymal part within the region of interest of the lesion obtained after expansion.

[0032] In some embodiments of the present invention, in step (f), when expanding the three-dimensional rectangular box 3 of the lesion using the region growing algorithm, the contour of the target organ is used for constraint to remove the region outside the contour of the target organ. At the same time, the density of the parenchymal part and the non-parenchymal part of the target organ is compared, and a density difference threshold is set to remove the non-parenchymal part within the region of interest of the lesion obtained after expansion.

[0033] In some embodiments of the present invention, step (g) is further included after step (f): expanding the region of interest of the lesion within the organ in step (f) using the morphological dilation algorithm to obtain the lesion edge.

[0034] On the other hand, the present invention provides a segmentation system for the region of interest of the lesion within an organ. The segmentation system includes the following modules:

[0035] An image acquisition module for acquiring the medical image of the organ;

[0036] A target organ region acquisition module connected to the image acquisition module. The target organ region acquisition module uses the image segmentation method A to obtain the boundary between the target organ and the adjacent organs so as to obtain the target organ region;

[0037] A first three-dimensional rectangular box acquisition module for the lesion, connected to the target organ region acquisition module. The first three-dimensional rectangular box acquisition module for the lesion uses a predicted lesion tool to obtain the three-dimensional rectangular box 1 of the lesion in the target organ region;

[0038] A second three-dimensional rectangular box acquisition module for the lesion, connected to the first three-dimensional rectangular box acquisition module for the lesion. The second three-dimensional rectangular box acquisition module for the lesion performs false positive suppression on the three-dimensional rectangular box 1 of the lesion to obtain the three-dimensional rectangular box 2 of the lesion;

[0039] A third three-dimensional rectangular box acquisition module for the lesion, connected to the second three-dimensional rectangular box acquisition module for the lesion. The third three-dimensional rectangular box acquisition module for the lesion uses the image segmentation method B to segment the three-dimensional rectangular box 2 of the lesion to obtain the three-dimensional rectangular box 3 of the lesion;

[0040] A result output module, which is connected to the third lesion three-dimensional rectangular box acquisition module. The result output module uses a region growing algorithm to expand the lesion three-dimensional rectangular box 3 to obtain the region of interest of the lesion within the organ.

[0041] In some embodiments of the present invention, the medical image includes at least one selected from CT images, MRI images, and ultrasound images.

[0042] In some embodiments of the present invention, the image segmentation method A and the image segmentation method B include at least one of a threshold-based segmentation method, a region-based segmentation method, an edge-based segmentation method, and a deep learning-based segmentation method.

[0043] In some embodiments of the present invention, the image segmentation method A and the image segmentation method B are the same.

[0044] In some embodiments of the present invention, the image segmentation method A and the image segmentation method B are different.

[0045] In some embodiments of the present invention, the image segmentation method A and the image segmentation method B are deep learning-based segmentation methods.

[0046] In some embodiments of the present invention, the deep learning segmentation algorithm includes at least one selected from 3DUnet and nnUnet.

[0047] In some embodiments of the present invention, the deep learning segmentation algorithms used by the image segmentation method A and the image segmentation method B are both 3DUnet.

[0048] In some embodiments of the present invention, in the target organ region acquisition module, before training the deep learning segmentation algorithm, the proportion of training irregular target organs is increased by flipping, adding noise, intensity normalization, and rotating the target organs in the medical images in the training set for data augmentation processing.

[0049] In some embodiments of the present invention, the image segmentation method A and / or the image segmentation method B use a loss function for optimization during the training process of the segmentation method.

[0050] In some embodiments of the present invention, the loss function includes at least one selected from BoundaryLoss and FocolLoss, and preferably the loss function BoundaryLoss.

[0051] In some embodiments of the present invention, in the target organ region acquisition module, the method for acquiring the boundary between the target organ and the adjacent organ includes: marking the region of the target organ and the adjacent organ regions, and using the loss function for optimization during the training process of the deep learning segmentation algorithm to obtain the boundary between the target organ and the adjacent organ.

[0052] In some embodiments of the present invention, the predicted lesion tool is a prediction model capable of predicting a rectangular bounding box of the lesion.

[0053] In some embodiments of the present invention, the prediction model includes at least one selected from the Anchor-Free model and the Anchor-Based model.

[0054] In some embodiments of the present invention, the Anchor-Free model includes at least one selected from FCOS, YOLO, and FSAF.

[0055] In some embodiments of the present invention, the Anchor-Based model includes at least one selected from RetinaNet, SSD, and Faster RCNN.

[0056] In some embodiments of the present invention, in the second lesion three-dimensional rectangular box acquisition module, the method for false positive suppression includes: classifying whether the lesion three-dimensional rectangular box 1 is a lesion through a classification network, and removing non-lesion regions.

[0057] In some embodiments of the present invention, the classification network includes at least one selected from the Resnet, DenseNet, and ResNeXt classification networks.

[0058] In some embodiments of the present invention, in the result output module, when expanding the lesion three-dimensional rectangular box 3 using the region growing algorithm, the contour of the target organ is used for constraint, and the region outside the contour of the target organ is removed.

[0059] In some embodiments of the present invention, in the result output module, when expanding the lesion three-dimensional rectangular box 3 using the region growing algorithm, by comparing the density of the parenchymal part and the non-parenchymal part of the target organ, a density difference threshold is set to remove the non-parenchymal part within the lesion region of interest obtained after expansion.

[0060] In some embodiments of the present invention, in the result output module, when using the region growing algorithm to expand the three-dimensional rectangular box 3 of the lesion, the contour of the target organ is used for constraint to remove the region outside the contour of the target organ. At the same time, by comparing the density of the parenchymal part and the non-parenchymal part of the target organ, a density difference threshold is set to remove the non-parenchymal part in the region of interest of the lesion obtained after expansion.

[0061] In some embodiments of the present invention, the result output module further includes a lesion expansion module, which is connected to the result output module. The lesion expansion module is used to expand the region of interest of the lesion in the organ in the result output module using the morphological dilation algorithm to obtain the lesion edge.

[0062] Another aspect of the present invention provides an electronic device for a method of segmenting a region of interest of a lesion in an organ, including a memory and a processor. Wherein, the processor runs a program corresponding to the executable program code by reading the executable program code stored in the memory to implement the above-mentioned method of segmenting the region of interest of the lesion in the organ.

[0063] Another aspect of the present invention provides a computer-readable storage medium, which stores a computer program. When the program is executed by a processor, it implements the above-mentioned method of segmenting the region of interest of the lesion in the organ.

[0064] The additional aspects and advantages of the present invention will be partially given in the following description, partially become obvious from the following description, or be understood through the practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0065] The above and / or additional aspects and advantages of the present invention will become obvious and easy to understand from the description of the embodiments in conjunction with the following drawings, wherein:

[0066] Figure 1 Shows a schematic flowchart of a method for segmenting a region of interest (ROI) of an organ lesion in an embodiment of the present invention;

[0067] Figure 2 Shows a flowchart of segmenting the ROI region of a liver lesion in an embodiment of the present invention;

[0068] Figure 3 Shows a flowchart of obtaining the liver region by multi-organ segmentation in an embodiment of the present invention;

[0069] Figure 4 Shows a flowchart of segmenting a lesion in the liver in an embodiment of the present invention;

[0070] Figure 5Shows the schematic diagram of the extended liver region in an embodiment of the present invention;

[0071] Figure 6 Shows the flowchart of the segmentation of the region of interest in the liver in Example 1 of the present invention. Detailed implementation manners

[0072] The embodiments of the present invention will be described in detail below. The following described embodiments are exemplary and are only used to explain the present invention, and should not be construed as a limitation to the present invention.

[0073] It should be noted that the terms "first" and "second" are only used for descriptive purposes and should not be construed as indicating or implying relative importance or implicitly specifying the quantity of the indicated technical features. Thus, the features defined with "first" and "second" may explicitly or implicitly include one or more of such features. Further, in the description of the present invention, unless otherwise specified, the meaning of "plurality" is two or more.

[0074] In the ranges disclosed herein, the endpoints and any value are not limited to the exact range or value, and these ranges or values should be understood to include values close to these ranges or values. For numerical ranges, between the endpoint values of each range, between the endpoint values of each range and a single point value, and between single point values, they can be combined with each other to obtain one or more new numerical ranges, and these numerical ranges should be regarded as specifically disclosed herein.

[0075] To make the present invention easier to understand, certain technical and scientific terms are specifically defined below. Unless otherwise clearly defined elsewhere in this document, all other technical and scientific terms used herein have the meaning commonly understood by those of ordinary skill in the art to which the present invention pertains.

[0076] In this document, the term "comprising" or "including" is an open expression, that is, it includes the content specified by the present invention, but does not exclude other aspects.

[0077] In this document, the terms "optionally", "optional" or "option" generally mean that the subsequent described event or condition may or may not occur, and this description includes the case where the event or condition occurs and the case where the event or condition does not occur.

[0078] In this article, the term "ROI" (region of interest) refers to the region of interest. In machine vision and image processing, the region that needs to be processed is outlined in the form of a square, circle, ellipse, irregular polygon, etc. from the processed image, which is called the region of interest, ROI. Various operators and functions are commonly used in machine vision software such as Halcon, OpenCV, and Matlab to obtain the region of interest ROI and perform the next step of image processing.

[0079] In this article, the term "boundary constraint" means that the boundaries between multiple organs cannot overlap with each other, that is, a voxel point can only belong to one organ category. Another boundary constraint refers to the integrity of the boundary contour and that the contour curvature does not change suddenly.

[0080] In this article, the term "false positive" means that some non-lesion tissues present images similar to lesions during scanning.

[0081] In this article, the term "lesion" refers to the site where the disease is concentrated or the main site of a comprehensive disease or infection, including but not limited to tumors, hydatid disease, etc.

[0082] In the term "morphological dilation algorithm" in this article, "morphology" refers to simplifying the data of the image, removing unimportant structures, and only maintaining the basic shape characteristics of the image. The region of interest is called the foreground pixel points, and the uninteresting part is called the background pixel points; "dilation" refers to the process of merging the background points of the target region into the target object and expanding the boundary of the target object to the outside.

[0083] In this article, the term "image segmentation" refers to the technology and process of dividing an image into several specific regions with unique properties and extracting the target of interest. It is a key step from image processing to image analysis. The existing image segmentation methods are mainly divided into the following categories: threshold-based segmentation methods, region-based segmentation methods, edge-based segmentation methods, and segmentation methods based on specific theories, etc. From a mathematical perspective, image segmentation is the process of dividing a digital image into non-overlapping regions. The process of image segmentation is also a labeling process, that is, pixels belonging to the same region are assigned the same number.

[0084] In this article, the term "region growing" refers to the process of developing a group of pixels or regions into a larger region. Starting from a set of seed points, the region growth from these points is achieved by merging adjacent pixels with similar attributes such as intensity, gray level, texture color, etc. to this region.

[0085] In this text, the term "CT value" is a unit of measurement for determining the density of a local tissue or organ in the human body. It is usually called the Hounsfield unit (HU). In fact, the CT value is the corresponding value in the CT image that is equivalent to the X-ray attenuation coefficient of each tissue. Whether it is a matrix image or matrix numbers, they are all representatives of the CT value, and the CT value is converted from the μ value of human tissues and organs. The CT value is not an absolutely constant value. It is related not only to internal factors in the human body such as breathing and blood flow, but also to external factors such as the X-ray tube voltage, CT device, and indoor temperature.

[0086] In the present invention, the term "loss function" is an operation function used to measure the degree of difference between the predicted value f(x) of the model and the true value Y. It is a non-negative real-valued function, usually denoted by L(Y, f(x)). The smaller the loss function, the better the robustness of the model. The loss function is mainly used in the training stage of the model. After each batch of training data is fed into the model, the predicted value is output through forward propagation. Then the loss function calculates the difference value between the predicted value and the true value, that is, the loss value. After obtaining the loss value, the model updates each parameter through backpropagation to reduce the loss between the true value and the predicted value, so that the predicted value generated by the model approaches the true value, thereby achieving the purpose of learning.

[0087] Segmentation method of organ lesion ROI

[0088] There are certain natural gaps between organs. Although some of the gaps are very subtle, clinicians can still accurately distinguish them based on experience.

[0089] According to an embodiment of the present invention, the present invention provides a segmentation method of an organ lesion ROI, and the method is as Figure 1 shown, including:

[0090] S100. Obtain a medical image containing the organ;

[0091] S120. Use image segmentation method A to obtain the boundary between the target organ and adjacent organs, so as to obtain the target organ region;

[0092] S130. Use a predicted lesion tool to obtain a three-dimensional stereoscopic rectangular box 1 of the lesion in the target organ region;

[0093] S140. Perform false positive suppression on the three-dimensional stereoscopic rectangular box 1 of the lesion to obtain a three-dimensional stereoscopic rectangular box 2 of the lesion;

[0094] S150. Use image segmentation method B to segment the three-dimensional stereoscopic rectangular box 2 of the lesion to obtain a three-dimensional stereoscopic rectangular box 3 of the lesion;

[0095] S160. Expand the three-dimensional rectangular box 3 of the lesion using the region growing algorithm to obtain the region of interest of the lesion within the organ.

[0096] According to some specific embodiments of the present invention, the method for segmenting the ROI region of the organ lesion proposed by the present invention, the organ includes all organs within the abdominal cavity, such as the liver, kidney, spleen, heart, and lung.

[0097] According to some specific embodiments of the present invention, considering that the target organ may have irregular resection situations, such as lesions, atrophy, surgery, individual differences, etc., data augmentation is performed before the segmentation algorithm training. The specific method is to use the method of random sampling to simulate the intraoperative organ resection after marking the target organ region, increasing the proportion of irregular organs in the dataset, so that the segmentation algorithm can better adapt to various irregular algorithms.

[0098] According to some specific embodiments of the present invention, the method for segmenting the ROI region of the organ lesion proposed by the present invention is based on the deep learning segmentation algorithm, and the algorithms include 3D U-Net and nnUnet.

[0099] According to some more specific embodiments of the present invention, for the method for segmenting the ROI region of the organ lesion proposed by the present invention, when the organ is the liver, the main segmentation method includes three stages: multi-organ segmentation to obtain the accurate liver region, liver lesion segmentation and ROI expansion, and lesion ROI region correction based on contour and non-parenchymal region constraints, as Figure 2 shown, the specific steps are as follows:

[0100] (1) Obtain the images of multiple organs;

[0101] (2) Segment the multiple organs to obtain the accurate liver region;

[0102] (3) Perform preliminary detection and false positive suppression on the liver lesions;

[0103] (4) Segment and expand the liver lesions;

[0104] (5) Obtain the ROI parenchymal region of the liver lesions;

[0105] Among them, the images in step (1) include at least one of CT images, MRI images, and ultrasound images; in steps (3) and (4), the process of preliminary detection and segmentation of the liver lesions requires contour constraints so that the segmented lesion area is only within the liver rather than other organs; after step (4), the CT threshold is further defined to segment the non-parenchymal region in the liver, thereby obtaining the ROI parenchymal region of the liver lesions.

[0106] According to some specific embodiments of the present invention, those skilled in the art can set different CT thresholds according to different situations. The CT threshold refers to a measurement unit for determining the density of a local tissue or organ in the human body, and is related to various factors such as the size of the lesion and different tissues and organs.

[0107] According to some specific embodiments of the present invention, the method for obtaining the accurate liver region in step (2) of the above organ lesion ROI region segmentation method is as Figure 3 described. Different from the method of directly segmenting the liver region by other methods, the segmentation method adopted by the present invention is to simultaneously label the liver region and the surrounding organ regions (spleen, gallbladder, kidney, etc.), and use a loss function for optimization during the training process of the segmentation algorithm. The loss function includes BoundaryLoss and FocolLoss. At the same time, the contour parts of other adjacent organs in the region adjacent to the liver are used for constraint, so that the boundary between the liver and the surrounding organs is clearer. Before segmentation, considering the possible irregular resection of the liver, such as lesions, atrophy, surgery, individual differences, etc., data augmentation is performed before the training of the segmentation algorithm. The preferred method is to use the method of random sampling to simulate the intraoperative liver resection after labeling the liver region, and increase the proportion of irregular livers in the dataset, so that the segmentation algorithm can better adapt to various irregular algorithms.

[0108] According to some more specific embodiments of the present invention, the specific method for liver lesion segmentation and expansion in step (4) of the above organ lesion ROI region segmentation is as Figure 4 shown, including:

[0109] ① Obtain the CT liver region image;

[0110] ② Based on FCOS, predict the rectangular bounding box of the lesion;

[0111] ③ Based on 3DUnet, segment and amplify the lesion within the detection box;

[0112] ④ Obtain the lesion ROI region within the liver;

[0113] After obtaining the specific liver region in the above step (2), lesion detection is performed inside the liver, and then the detected lesions are segmented and the ROI is expanded. The rectangular bounding box of the lesion can be predicted by the FCOS or anchor algorithm. Here, FCOS is preferred. FCOS is a single-stage fully convolutional object detection network based on pixel-level prediction. By removing the pre-defined anchors similar to those in two-stage object detection networks, FCOS can avoid complex operations related to anchors and the constraints on anchor sizes. After predicting the three-dimensional rectangular box of the lesion through the detection network, the obtained rectangular bounding box of the lesion is subjected to false positive suppression through the Resnet classification network to reduce false positives caused by regions such as cysts and blood vessels. Then, similarly, the actual lesion region is segmented from the three-dimensional rectangular box of the lesion through the 3DU-Net segmentation network. Different from directly segmenting the lesion, the strategy of detecting first and then segmenting can make the edge of the segmented lesion more accurate and reduce false positives to a certain extent.

[0114] According to some specific embodiments of the present invention, after obtaining the lesion region in the above step ③, the morphological dilation algorithm is used to expand the lesion part to obtain the edge region of interest of the research target. The accurate liver contour boundary can be obtained through step (2), and at the same time, the non-liver parenchyma parts such as blood vessels and metal shadows in the liver can be segmented through density contrast. These two results can simultaneously constrain the ROI region of the lesion in the liver obtained in step ④ to ensure that the expanded lesion ROI region will neither go outside the liver nor contain non-liver parenchyma parts, as Figure 5 shown.

[0115] The application of the present invention to the liver can standardize the liver parenchyma region of the lesion, reduce the interference caused by non-parenchyma regions and non-liver regions during subsequent calculation of lesion features, and achieve as much as possible the measurement unity at the lesion feature level.

[0116] The present invention first simultaneously segments multiple organs based on a deep learning segmentation algorithm, and makes the boundaries between multiple organs clearer and more reasonable through boundary constraints. At the same time, considering the irregular shapes of organs of patients who have undergone partial surgical resection or organ atrophy, targeted data augmentation is used to make the segmentation results of the segmentation model for irregular target organs more robust. It can ensure that the segmentation results of the target organs are not affected by lesions, atrophy, surgery, etc. After segmenting the target organ and the surrounding organ regions, a target detection and segmentation network is used to accurately segment the lesion part, and then an interested region around the target lesion is obtained by expanding a certain width according to the region growing algorithm. During the expansion process, the contour of the target organ is used for constraint, and the expanded regions outside the contour of the target organ are automatically removed. For non-target organ tissues such as blood vessels or metal shadows inside the target organ encountered during the expansion process, they are automatically removed by setting a density difference threshold. During this process, as long as it is inside the target organ, the expansion process is not interrupted by the metal shadow, so as to truly reflect the biological information of the interested region.

[0117] The following will explain the solution of the present disclosure in conjunction with embodiments. Those skilled in the art will understand that the following embodiments are only used to illustrate the present disclosure and should not be regarded as limiting the scope of the present disclosure. For those not specified in the embodiments regarding specific technologies or conditions, they shall be carried out according to the technologies or conditions described in the literature in this field or according to the product specifications.

[0118] Example 1: Segmentation of the Parenchymal Region of the Lesion ROI in the Liver

[0119] Taking the liver as an example, the inventor segmented the parenchymal region of the lesion ROI in the liver, and the segmentation steps are as Figure 6 shown:

[0120] 1) Obtain CT images;

[0121] 2) Use the 3DUnet segmentation network to obtain the boundaries between the organ and adjacent organs, segment the liver region, and use the 3DFCOS algorithm to obtain the candidate boxes of the lesions in the liver region;

[0122] 3) Suppress false positives for the obtained candidate boxes of the lesions through the Resnet classification network to obtain the true three-dimensional rectangular boxes of the lesions;

[0123] 4) Use the 3DUnet segmentation network to segment the actual lesion region in the three-dimensional rectangular box of the lesion;

[0124] 5) Expand the segmented lesion region using the region growing algorithm and constrain it with the liver contour to obtain the parenchymal region of the lesion ROI in the liver.

[0125] It is known in the art that adaptively expanding the outer diameter of a tumor can better obtain the region of interest of the tumor. As can be seen from Figure (6), without the contour constraint of the present invention, the expanded tumor easily exceeds the boundary of the liver organ. By density contrast, non-liver parenchymal parts such as blood vessels and metal shadows in the liver can be removed. Moreover, due to the contour constraint of the liver, the ROI region obtained by the region growth algorithm is only inside the liver organ and is not affected by the non-parenchymal parts of the liver, thus truly reflecting the biological information of the region of interest of the liver.

[0126] The segmentation method for the region of interest of a lesion within an organ proposed by the present invention can standardize the lesion region, reduce the interference brought by non-target regions when calculating lesion features subsequently, and achieve the measurement unification at the lesion feature level as much as possible. The organ includes, but is not limited to, abdominal organs such as the liver, kidney, spleen, heart, and lung.

[0127] In the description of this specification, the description with reference to terms such as "one embodiment", "some embodiments", "example", "specific example", "some implementation schemes" or "some examples" means that the specific features, structures, materials or characteristics described in connection with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic expressions of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in a suitable manner in any one or more embodiments or examples. In addition, without contradiction, those skilled in the art can combine and combine the different embodiments or examples described in this specification and the features of different embodiments or examples.

[0128] Although the embodiments of the present invention have been shown and described above, it can be understood that the above embodiments are exemplary and should not be construed as limiting the present invention. Those of ordinary skill in the art can make changes, modifications, substitutions and variations to the above embodiments within the scope of the present invention.

Claims

1. A method for segmenting a region of interest of a lesion in an organ, characterized in that, The segmentation method includes: (a) Obtain a medical image containing the organ; (b) Use image segmentation method A to obtain the boundary between the target organ and adjacent organs, so as to obtain the target organ region; (c) Use a predicted lesion tool to obtain a three-dimensional rectangular box 1 of the lesion in the target organ region; (d) Perform false positive suppression on the three-dimensional rectangular box 1 of the lesion to obtain a three-dimensional rectangular box 2 of the lesion; (e) Use image segmentation method B to segment the three-dimensional rectangular box 2 of the lesion to obtain a three-dimensional rectangular box 3 of the lesion; (f) Use a region growing algorithm to expand the three-dimensional rectangular box 3 of the lesion to obtain the region of interest of the lesion in the organ.

2. The segmentation method according to claim 1, wherein The medical image includes at least one selected from CT images, MRI images, and ultrasound images.

3. The segmentation method according to claim 1, wherein The image segmentation method A and the image segmentation method B include at least one of a threshold-based segmentation method, a region-based segmentation method, an edge-based segmentation method, and a deep learning-based segmentation method; Optionally, the image segmentation method A and the image segmentation method B are the same or different; Optionally, the image segmentation method A and the image segmentation method B are deep learning-based segmentation methods; Optionally, the deep learning segmentation algorithm includes at least one selected from 3D Unet and nnUnet; Optionally, the deep learning segmentation algorithms used by the image segmentation method A and the image segmentation method B are both 3D Unet; Optionally, in step (b), before training the deep learning segmentation algorithm, the proportion of irregular target organs in the training is increased by flipping, adding noise, intensity normalization, and rotation of the target organs in the medical images in the training set for data augmentation processing; Optionally, the image segmentation method A and / or the image segmentation method B use a loss function for optimization during the training process of the segmentation method; Optionally, the loss function includes at least one selected from Boundary Loss and Focol Loss, and preferably the loss function Boundary Loss; Optionally, in step (b), the method for obtaining the boundary between the target organ and adjacent organs includes: marking the regions of the target organ and adjacent organ regions, and using the loss function for optimization during the training process of the deep learning segmentation algorithm to obtain the boundary between the target organ and adjacent organs.

4. The splitting method according to claim 1, wherein The predicted lesion tool is a prediction model capable of predicting a rectangular bounding box of a lesion; Optionally, the prediction model includes at least one selected from Anchor-Free models and Anchor-Based models; Optionally, the Anchor-Free model includes at least one selected from FCOS, YOLO, and FSAF; Optionally, the Anchor-Based model includes at least one selected from RetinaNet, SSD, and Faster RCNN.

5. The segmentation method according to claim 1, wherein In step (d), the method for performing false positive suppression includes: The three-dimensional rectangular box 1 of the lesion is classified as a lesion or not through a classification network, and non-lesion areas are removed; Optionally, the classification network includes at least one selected from Resnet, DenseNet, and ResNeXt classification networks.

6. The segmentation method according to claim 1, characterized in that, In step (f), when expanding the three-dimensional rectangular box 3 of the lesion using the region growing algorithm, it is constrained by the contour of the target organ, and the area outside the contour of the target organ is removed; Optionally, in step (f), when expanding the three-dimensional rectangular box 3 of the lesion using the region growing algorithm, by comparing the densities of the parenchymal part and the non-parenchymal part of the target organ, a density difference threshold is set to remove the non-parenchymal part within the region of interest of the lesion obtained after expansion; Optionally, in step (f), when expanding the three-dimensional rectangular box 3 of the lesion using the region growing algorithm, it is constrained by the contour of the target organ, and the area outside the contour of the target organ is removed. At the same time, by comparing the densities of the parenchymal part and the non-parenchymal part of the target organ, a density difference threshold is set to remove the non-parenchymal part within the region of interest of the lesion obtained after expansion.

7. The segmentation method according to claim 1, wherein, It further includes step (g), The region of interest of the lesion within the organ described in step (f) is expanded using the morphological dilation algorithm to obtain the lesion edge.

8. A segmentation system for regions of interest of lesions within an organ, characterized in that, The segmentation system includes the following modules: An image acquisition module for acquiring the medical image of the organ; A target organ region acquisition module connected to the image acquisition module. The target organ region acquisition module uses image segmentation method A to obtain the boundary between the target organ and adjacent organs, so as to obtain the target organ region; A first three-dimensional rectangular box acquisition module of the lesion connected to the target organ region acquisition module. The first three-dimensional rectangular box acquisition module of the lesion uses a predicted lesion tool to obtain the three-dimensional rectangular box 1 of the lesion in the target organ region; A second three-dimensional rectangular box acquisition module of the lesion connected to the first three-dimensional rectangular box acquisition module of the lesion. The second three-dimensional rectangular box acquisition module of the lesion suppresses false positives of the three-dimensional rectangular box 1 of the lesion to obtain the three-dimensional rectangular box 2 of the lesion; A third three-dimensional rectangular box acquisition module of the lesion connected to the second three-dimensional rectangular box acquisition module of the lesion. The third three-dimensional rectangular box acquisition module of the lesion uses image segmentation method B to segment the three-dimensional rectangular box 2 of the lesion to obtain the three-dimensional rectangular box 3 of the lesion; A result output module connected to the third three-dimensional rectangular box acquisition module of the lesion. The result output module uses the region growing algorithm to expand the three-dimensional rectangular box 3 of the lesion to obtain the region of interest of the lesion within the organ.

9. The splitting system according to claim 8, wherein The medical image includes at least one selected from CT images, MRI images, and ultrasound images; Optionally, the image segmentation method A and the image segmentation method B include at least one of a threshold-based segmentation method, a region-based segmentation method, an edge-based segmentation method, and a deep learning-based segmentation method; Optionally, the image segmentation method A and the image segmentation method B are the same or different; Optionally, the image segmentation method A and the image segmentation method B are deep learning-based segmentation methods; Optionally, the deep learning segmentation algorithm includes at least one selected from 3D Unet and nnUnet; Optionally, the deep learning segmentation algorithms adopted by the image segmentation method A and the image segmentation method B are both 3D Unet; Optionally, in the target organ region acquisition module, before training the deep learning segmentation algorithm, the proportion of irregular target organs in the training is increased by flipping, adding noise, intensity normalization, and rotating the target organs in the medical images in the training set, and data augmentation processing is performed; Optionally, during the training process of the segmentation method, both the image segmentation method A and / or the image segmentation method B use a loss function for optimization; Optionally, the loss function includes at least one selected from Boundary Loss and Focol Loss, and preferably the loss function Boundary Loss; Optionally, in the target organ region acquisition module, the method for obtaining the boundary between the target organ and adjacent organs includes: Marking the region of the target organ and the regions of adjacent organs, and using the loss function for optimization during the training process of the deep learning segmentation algorithm to obtain the boundary between the target organ and adjacent organs.

10. The splitting system according to claim 8, characterized in that, The predicted lesion tool is a prediction model capable of predicting a rectangular bounding box of a lesion; Optionally, the prediction model includes at least one selected from an Anchor-Free model and an Anchor-Based model; Optionally, the Anchor-Free model includes at least one selected from FCOS, YOLO, and FSAF; Optionally, the Anchor-Based model includes at least one selected from RetinaNet, SSD, and Faster RCNN; Optionally, in the second lesion three-dimensional rectangular box acquisition module, the method for false positive suppression includes: classifying whether the lesion three-dimensional rectangular box 1 is a lesion through a classification network, and removing non-lesion regions; Optionally, the classification network includes at least one selected from Resnet, DenseNet, and ResNeXt classification networks; Optionally, in the result output module, when expanding the lesion three-dimensional rectangular box 3 using the region growing algorithm, the contour of the target organ is used for constraint to remove the regions outside the contour of the target organ; Optionally, in the result output module, when expanding the lesion three-dimensional rectangular box 3 using the region growing algorithm, by comparing the density of the parenchymal part and non-parenchymal part of the target organ, a density difference threshold is set to remove the non-parenchymal part within the lesion region of interest obtained after expansion; Optionally, in the result output module, when expanding the three-dimensional rectangular box 3 of the lesion using the region growing algorithm, the contour of the target organ is used for constraint to remove the area outside the contour of the target organ. At the same time, by comparing the density of the parenchymal part and the non-parenchymal part of the target organ, a density difference threshold is set to remove the non-parenchymal part within the region of interest of the lesion obtained after expansion.

11. The segmentation system according to claim 8, wherein Further comprising a lesion expansion module, the lesion expansion module is connected to the result output module, and the lesion expansion module is used to expand the region of interest of the lesion within the organ in the result output module using the morphological dilation algorithm to obtain the lesion edge.

12. An electronic device for a method of segmenting a region of interest of a lesion in an organ, characterized in that, Comprising a memory and a processor; Wherein, the processor runs a program corresponding to the executable program code by reading the executable program code stored in the memory, so as to implement the method for segmenting the region of interest of the lesion within the organ according to any one of claims 1-7.

13. A computer-readable storage medium storing a computer program, characterized in that, When the program is executed by the processor, it implements the method for segmenting the region of interest of the lesion within the organ according to any one of claims 1-7.

Images