Liver and gall tumor three-dimensional positioning method and system based on detection data and medium

Abstract

The invention provides a liver and gall tumor three-dimensional positioning method and system based on detection data and a medium, and the method comprises the steps: obtaining the multi-modal medical image data of a patient, and carrying out the three-dimensional reconstruction, and obtaining a corresponding three-dimensional image; performing image recognition on each three-dimensional image to obtain a liver and gall area and a corresponding tumor area; fusing the liver and gall areas of the plurality of three-dimensional images and the corresponding tumor areas to obtain the liver and gall areas of the patient and the corresponding tumor areas; the method comprises the following steps: acquiring multi-modal medical image data, respectively reconstructing three-dimensional images, carrying out image identification on the three-dimensional images to obtain a corresponding liver and gall region and a tumor region, fusing a plurality of three-dimensional images in multiple modals to obtain a final liver and gall region and a tumor region of a patient, and fusing image advantages in multiple modals to obtain the liver and gall region of the patient. And the relatively accurate liver and gall region and the corresponding tumor region are obtained, so that the identification accuracy of the liver and gall tumor can be improved, and a relatively accurate data basis is provided for subsequent operations.

Description

Technical Field

[0001] This application relates to the technical field of detection of hepatobiliary tumors, and particularly relates to a three-dimensional localization method, system and medium for hepatobiliary tumors based on detection data. Background Art

[0002] The liver and gallbladder refer to the liver and the gallbladder. The liver is the main organ for metabolic functions, and its main functions are to store glycogen, secrete synthetic proteins, produce bile, and assist in food digestion; the gallbladder is a pear-shaped sac structure, and its main functions are to concentrate and store bile, secrete mucus, and protect the biliary mucosa. Hepatobiliary tumors are relatively common diseases. For hepatobiliary tumors, the direct and effective treatment method is to remove the tumor, and the prerequisite for removing the tumor is to accurately obtain the location information and size information of the tumor, etc.

[0003] Currently, for the surgical resection method of hepatobiliary tumors, it is mostly to obtain medical images by equipment and then medical staff judge its location and size based on experience, and perform resection based on the on-site situation during the operation. Obviously, such a method is relatively dependent on the doctor's experience and the adaptability during the operation. Based on some deep learning algorithms, it has been able to determine the coordinate position of the lesion in medical images. However, the clarity of medical images is not particularly high, and there are certain interference shadows in them. At the same time, the accuracy of deep learning algorithms is also limited, resulting in low recognition accuracy for the regions of interest and lesion regions in medical images, and even misguidance problems.

[0004] Therefore, there is an urgent need for a solution that can accurately obtain the location information and size information of hepatobiliary tumors before surgery. Summary of the Invention

[0005] To solve the above technical problems, this application is proposed. Embodiments of this application provide a three-dimensional localization method, system and medium for hepatobiliary tumors based on detection data.

[0006] According to one aspect of this application, a three-dimensional localization method for hepatobiliary tumors based on detection data is provided, including: obtaining multi-modal medical image data of a patient; wherein, the medical image data of each modality includes multiple stacked two-dimensional images; performing three-dimensional reconstruction on the medical image data of each modality to obtain corresponding three-dimensional images; respectively performing image recognition on each three-dimensional image to obtain the hepatobiliary region and the corresponding tumor region; fusing the hepatobiliary regions and the corresponding tumor regions of multiple three-dimensional images to obtain the hepatobiliary region and the corresponding tumor region of the patient.

[0007] In one embodiment, the image recognition of each three-dimensional image to obtain the liver and gall region and the corresponding tumor region includes: for each three-dimensional image, dividing the three-dimensional image into multiple layered images according to a set rule; respectively recognizing the liver and gall region and the corresponding tumor region in each layer of the layered images; for each three-dimensional image, fusing the liver and gall regions and the corresponding tumor regions in all the layered images of the three-dimensional image to obtain the liver and gall region and the corresponding tumor region of the three-dimensional image.

[0008] In one embodiment, the dividing the three-dimensional image into multiple layered images according to a set rule includes: dividing the three-dimensional image into multiple layered images according to a set number of layers; wherein, each layer of the layered images includes multiple consecutive two-dimensional images; the fusing the liver and gall regions and the corresponding tumor regions in all the layered images of the three-dimensional image to obtain the liver and gall region and the corresponding tumor region of the three-dimensional image includes: splicing the liver and gall regions and the corresponding tumor regions in all the layered images of the three-dimensional image to obtain a spliced region image; performing post-processing on the spliced region image to obtain the liver and gall region and the corresponding tumor region of the three-dimensional image.

[0009] In one embodiment, the performing post-processing on the spliced region image to obtain the liver and gall region and the corresponding tumor region of the three-dimensional image includes: forming any path on the boundary of the liver and gall region of the spliced region image from the first layer of the layered image to the last layer of the layered image; if the curvature change rate between two adjacent points on the path is greater than a first preset value, then obtaining the curvature change rate between two symmetric-side points on the layered image where the two adjacent points are located; if the curvature change rate between the symmetric-side points is less than a second preset value, then adjusting the relative position between the layered images where the two adjacent points are located to reduce the curvature change rate between the two adjacent points and increase the curvature change rate between the symmetric-side points to obtain the liver and gall region of the three-dimensional image; wherein, the second preset value has the opposite sign to the first preset value.

[0010] In one embodiment, the performing post-processing on the spliced region image to obtain the liver and gall region and the corresponding tumor region of the three-dimensional image further includes: if the curvature change rate between the symmetric-side points is greater than or equal to the second preset value, then performing smoothing processing on the two adjacent points.

[0011] In one embodiment, the separately identifying the liver and gallbladder regions and the corresponding tumor regions in each layer of the layered images includes: identifying a reference region in each layer of the layered images; wherein the reference region represents a specific region in the layered image; determining an interval range where the liver and gallbladder regions are located based on the relative positional relationship between the reference region and the liver and gallbladder regions; performing binarization processing on the interval range to obtain a binarized image; identifying the liver and gallbladder regions in the binarized image; and identifying the tumor regions corresponding to the liver and gallbladder regions in the layered image based on the liver and gallbladder regions.

[0012] In one embodiment, the fusing the liver and gallbladder regions and the corresponding tumor regions of multiple three-dimensional images to obtain the liver and gallbladder regions and the corresponding tumor regions of the patient includes: aligning the liver and gallbladder regions and the corresponding tumor regions of multiple three-dimensional images, and merging the aligned liver and gallbladder regions and the corresponding tumor regions of multiple three-dimensional images to obtain the liver and gallbladder regions and the corresponding tumor regions of the patient.

[0013] In one embodiment, the aligning the liver and gallbladder regions and the corresponding tumor regions of multiple three-dimensional images includes: calculating the variance of differences between the liver and gallbladder regions of the medical image data of all modalities in each layer of the layered images; selecting the layered image with the smallest variance of differences as the landmark layer image; and aligning the liver and gallbladder regions and the corresponding tumor regions of multiple three-dimensional images based on the landmark layer image.

[0014] According to another aspect of the present application, there is provided a three-dimensional positioning system for liver and gallbladder tumors based on detection data, including: a data acquisition module for acquiring multi-modal medical image data of a patient; wherein each modality of medical image data includes multiple stacked two-dimensional images; a three-dimensional reconstruction module for performing three-dimensional reconstruction on each modality of medical image data to obtain corresponding three-dimensional images; an image recognition module for performing image recognition on each three-dimensional image respectively to obtain the liver and gallbladder regions and the corresponding tumor regions; and an image fusion module for fusing the liver and gallbladder regions and the corresponding tumor regions of multiple three-dimensional images to obtain the liver and gallbladder regions and the corresponding tumor regions of the patient.

[0015] According to another aspect of the present application, there is provided a computer-readable storage medium storing a computer program for executing any of the above methods.

[0016] A three-dimensional localization method, system and medium for hepatobiliary tumors based on detection data provided by the present application obtain multi-modal medical image data of a patient; wherein, the medical image data of each modality includes multiple stacked two-dimensional images; perform three-dimensional reconstruction on the medical image data of each modality to obtain corresponding three-dimensional images; perform image recognition on each three-dimensional image respectively to obtain the hepatobiliary region and the corresponding tumor region; fuse the hepatobiliary regions and the corresponding tumor regions of multiple three-dimensional images to obtain the hepatobiliary region and the corresponding tumor region of the patient; by obtaining multi-modal medical image data, reconstructing three-dimensional images respectively, and performing image recognition on the three-dimensional images to obtain the corresponding hepatobiliary region and tumor region, and fusing multiple three-dimensional images under multiple modalities to obtain the final hepatobiliary region and tumor region of the patient, the advantages of images under multiple modalities are fused to obtain a relatively accurate hepatobiliary region and the corresponding tumor region, thereby improving the recognition accuracy of hepatobiliary tumors and providing a relatively accurate data basis for subsequent surgeries. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] The above and other objects, features and advantages of the present application will become more obvious by describing the embodiments of the present application in more detail with reference to the accompanying drawings. The accompanying drawings are used to provide a further understanding of the embodiments of the present application and constitute a part of the specification. Together with the embodiments of the present application, they are used to explain the present application and do not constitute a limitation to the present application. In the accompanying drawings, the same reference numerals generally represent the same components or steps.

[0018] Figure 1 is a schematic flowchart of a three-dimensional localization method for hepatobiliary tumors based on detection data provided by an exemplary embodiment of the present application.

[0019] Figure 2 is a schematic structural diagram of a three-dimensional localization system for hepatobiliary tumors based on detection data provided by an exemplary embodiment of the present application.

[0020] Figure 3 is a structural diagram of an electronic device provided by an exemplary embodiment of the present application. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0021] Next, exemplary embodiments of the present application will be described in detail with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments of the present application. It should be understood that the present application is not limited by the exemplary embodiments described herein.

[0022] Figure 1 is a schematic flowchart of a three-dimensional localization method for hepatobiliary tumors based on detection data provided by an exemplary embodiment of the present application. As Figure 1 shown, the three-dimensional localization method for hepatobiliary tumors based on detection data includes the following steps: Step 110: Obtain the multi-modal medical image data of the patient.

[0023] Among them, the medical image data of each modality includes multiple two-dimensional images arranged in layers. In this application, multiple modalities of medical image data of the patient are obtained through multiple medical imaging devices, such as CT images, MRI images, ultrasound images, etc.

[0024] Step 120: Perform three-dimensional reconstruction on the medical image data of each modality to obtain the corresponding three-dimensional image.

[0025] After obtaining the medical image data of multiple modalities, three-dimensional reconstruction is performed on the medical image data of each modality respectively to obtain the three-dimensional image corresponding to each modality, that is, a three-dimensional image including the liver and gall region and the tumor region is obtained.

[0026] Step 130: Perform image recognition on each three-dimensional image respectively to obtain the liver and gall region and the corresponding tumor region.

[0027] In this application, image recognition is performed on each three-dimensional image to obtain the liver and gall region and the corresponding tumor region in each three-dimensional image.

[0028] Step 140: Fuse the liver and gall regions and the corresponding tumor regions of multiple three-dimensional images to obtain the liver and gall region and the corresponding tumor region of the patient.

[0029] After obtaining the liver and gall region and the corresponding tumor region in each three-dimensional image in this application, by fusing the liver and gall regions and the corresponding tumor regions in multiple three-dimensional images, the final liver and gall region and the corresponding tumor region are obtained, that is, the advantages of multiple three-dimensional images are fused to mutually verify and assist, so as to improve the accuracy of identifying the liver and gall region and the corresponding tumor region of the patient.

[0030] A three-dimensional positioning method for liver and gall tumors based on detection data provided by this application, by obtaining the multi-modal medical image data of the patient; among them, the medical image data of each modality includes multiple two-dimensional images arranged in layers; performing three-dimensional reconstruction on the medical image data of each modality to obtain the corresponding three-dimensional image; performing image recognition on each three-dimensional image respectively to obtain the liver and gall region and the corresponding tumor region; fusing the liver and gall regions and the corresponding tumor regions of multiple three-dimensional images to obtain the liver and gall region and the corresponding tumor region of the patient; by obtaining multi-modal medical image data, reconstructing three-dimensional images respectively, and performing image recognition on the three-dimensional images to obtain the corresponding liver and gall region and tumor region, and fusing multiple three-dimensional images under multiple modalities to obtain the final liver and gall region and tumor region of the patient, fusing the advantages of images under multiple modalities to obtain a relatively accurate liver and gall region and the corresponding tumor region, so as to improve the recognition accuracy of liver and gall tumors and provide a relatively accurate data basis for subsequent surgeries.

[0031] In one embodiment, the specific implementation of the above step 130 may be: for each three-dimensional image, dividing the three-dimensional image into multiple layers of layered images according to a set rule; respectively identifying the liver and gallbladder regions and the corresponding tumor regions in each layer of layered image; for each three-dimensional image, fusing the liver and gallbladder regions and the corresponding tumor regions in all the layered images of the three-dimensional image to obtain the liver and gallbladder regions and the corresponding tumor regions of the three-dimensional image.

[0032] In this application, the three-dimensional image is divided into multiple layers of layered images, and each layer of layered image is identified to obtain the liver and gallbladder regions and the corresponding tumor regions of the layered image. After obtaining the liver and gallbladder regions and the corresponding tumor regions of the layered image, by fusing the liver and gallbladder regions and the corresponding tumor regions of all the layered images of each three-dimensional image, the liver and gallbladder regions and the corresponding tumor regions of the three-dimensional image are obtained.

[0033] In one embodiment, the specific implementation of the above step 130 may be: dividing the three-dimensional image into multiple layers of layered images according to a set number of layers; wherein, each layer of layered image includes multiple layers of continuous two-dimensional images; splicing the liver and gallbladder regions and the corresponding tumor regions in all the layered images of the three-dimensional image to obtain a spliced region image; performing post-processing on the spliced region image to obtain the liver and gallbladder regions and the corresponding tumor regions of the three-dimensional image.

[0034] Specifically, this application may use multiple layers of continuous two-dimensional images as one layer of layered image, and utilize the continuity between the continuous layers of two-dimensional images to mutually verify the liver and gallbladder regions and the corresponding tumor regions of each layer of two-dimensional image, thereby improving the accuracy of identifying the liver and gallbladder regions and the corresponding tumor regions of the layered image. After identifying the relatively accurate liver and gallbladder regions and the corresponding tumor regions of the layered image, splicing each layer of layered image to obtain the liver and gallbladder regions and the corresponding tumor regions of the overall three-dimensional image, and performing post-processing on the spliced image after splicing to further improve the accuracy of identifying the liver and gallbladder regions and the corresponding tumor regions.

[0035] In one embodiment, the specific implementation of the above step 130 may be: forming any path located on the boundary of the liver and gallbladder region of the spliced region image from the first layer of layered image to the last layer of layered image of the spliced region image; if the curvature change rate between two adjacent points on the path is greater than a first preset value, then obtaining the curvature change rate between two symmetric points on the layered images where the two adjacent points are located; if the curvature change rate between the two symmetric points is less than a second preset value, then adjusting the relative positions between the layered images where the two adjacent points are located to reduce the curvature change rate between the two adjacent points and increase the curvature change rate between the two symmetric points to obtain the liver and gallbladder region of the three-dimensional image; wherein, the second preset value has the opposite sign to the first preset value.

[0036] Specifically, the present application constructs a path formed by starting from the first hierarchical image, passing through each hierarchical image to the last hierarchical image. For example, starting from any point on the boundary of the liver and gall region of the first hierarchical image, verify the tangent direction of this point and connect a point on the boundary of the liver and gall region of each hierarchical image in sequence to form a path. It should be understood that there can be many paths in the present application. To simplify the solution, the present application can select multiple starting points corresponding to a specific angle to form a finite number of paths. After obtaining a path, the present application determines whether the curvature change rate between adjacent two points on the path is greater than a first preset value. If it exists, obtain the curvature change rate between two symmetric points on the hierarchical images where the adjacent two points are located. For example, with a fixed section plane as a reference, if the curvature change rate between adjacent two points on the left path is greater than the first preset value, it indicates that there is a mutation in the positions of the adjacent two points on the left path. At this time, by further determining whether the curvature change rate between the corresponding two points on the right path is less than a second preset value (with the opposite sign to the first preset value), if it is less, it indicates that one of the hierarchical images is misaligned with other hierarchical images. Then, adjust the relative positions between the hierarchical images where the adjacent two points are located to reduce the curvature change rate between the adjacent two points and increase the curvature change rate between the symmetric two points, so as to obtain the liver and gall region of the three-dimensional image, thereby improving the accuracy of splicing and then improving the accuracy of recognition.

[0037] In an embodiment, the specific implementation manner of the above step 130 may be: If the curvature change rate between the symmetric two points is greater than or equal to the second preset value, perform smoothing processing on the adjacent two points.

[0038] If there is a mutation in the positions of the adjacent two points and the curvature change rate between the symmetric two points is greater than or equal to the second preset value, it indicates that the mutation in the positions of the adjacent two points is not caused by misalignment. At this time, smooth the adjacent two points to solve the problem of the mutation in the positions of the adjacent two points and improve the recognition accuracy.

[0039] In an embodiment, the specific implementation manner of the above step 130 may be: Identify the reference region in each hierarchical image; where the reference region represents a specific region in the hierarchical image; based on the relative position relationship between the reference region and the liver and gall region, determine the interval range where the liver and gall region is located; perform binarization processing on the interval range to obtain a binarized image; identify the liver and gall region in the binarized image; based on the liver and gall region, identify the tumor region corresponding to the liver and gall region in the hierarchical image.

[0040] This application improves the recognition accuracy of the hepatobiliary region by identifying the reference region (such as the region where the ribs are located, etc.) in the layered image and excluding regions that are located in other positions and are similar to the hepatobiliary region based on the relative positional relationship between the reference region and the hepatobiliary region; after determining the range of the interval where the hepatobiliary region is located, binarize this range of the interval, and identify the hepatobiliary region in the binarized image, and based on the identified hepatobiliary region, identify the tumor region of the hepatobiliary region in the layered image, that is, identify the hepatobiliary region under the condition of excluding background interference, and then identify the tumor region within the defined hepatobiliary region in the layered image to exclude interference factors outside the range of the hepatobiliary region, thereby improving the recognition accuracy of the hepatobiliary region and the tumor region.

[0041] In one embodiment, the specific implementation manner of the above step 140 may be: align the hepatobiliary regions and the corresponding tumor regions of multiple three-dimensional images, and merge the hepatobiliary regions and the corresponding tumor regions of the aligned multiple three-dimensional images to obtain the hepatobiliary region and the corresponding tumor region of the patient.

[0042] This application aligns the hepatobiliary regions and the corresponding tumor regions of multiple three-dimensional images thereof, and merges the hepatobiliary regions and the corresponding tumor regions of the aligned multiple three-dimensional images to improve the accuracy during merging, and then improve the recognition accuracy.

[0043] In one embodiment, the specific implementation manner of the above step 140 may be: calculate the variance of the differences between the hepatobiliary regions of the medical image data of all modalities in each layer of the layered image; select the layered image with the smallest variance of differences as the landmark layer image; align the hepatobiliary regions and the corresponding tumor regions of multiple three-dimensional images based on the landmark layer image.

[0044] This application calculates the variance of the differences between the hepatobiliary regions of the medical image data of all modalities in each layer of the layered image to determine the differences between the hepatobiliary regions in each layered image, and selects the layered image with the smallest variance of differences as the landmark layer image, and realizes the alignment operation of the three-dimensional images by adjusting the positions of other layered images, so as to reduce the differences between the hepatobiliary regions and the corresponding tumor regions of each three-dimensional image after alignment, thereby improving the final recognition accuracy.

[0045] Figure 2 It is a schematic structural diagram of a three-dimensional positioning system for hepatobiliary tumors based on detection data provided by an exemplary embodiment of this application. As Figure 2As shown in the figure, the three-dimensional liver and gallbladder tumor localization system 20 based on detection data includes: a data acquisition module 21 for acquiring multi-modal medical image data of a patient; wherein, the medical image data of each modality includes multiple two-dimensional images arranged in layers; a three-dimensional reconstruction module 22 for performing three-dimensional reconstruction on the medical image data of each modality to obtain corresponding three-dimensional images; an image recognition module 23 for respectively performing image recognition on each three-dimensional image to obtain the liver and gallbladder region and the corresponding tumor region; and an image fusion module 24 for fusing the liver and gallbladder regions and the corresponding tumor regions of multiple three-dimensional images to obtain the liver and gallbladder region and the corresponding tumor region of the patient.

[0046] A three-dimensional liver and gallbladder tumor localization system provided by the present application acquires multi-modal medical image data of a patient through the data acquisition module 21; wherein, the medical image data of each modality includes multiple two-dimensional images arranged in layers; the three-dimensional reconstruction module 22 performs three-dimensional reconstruction on the medical image data of each modality to obtain corresponding three-dimensional images; the image recognition module 23 respectively performs image recognition on each three-dimensional image to obtain the liver and gallbladder region and the corresponding tumor region; the image fusion module 24 fuses the liver and gallbladder regions and the corresponding tumor regions of multiple three-dimensional images to obtain the liver and gallbladder region and the corresponding tumor region of the patient; by acquiring multi-modal medical image data, respectively reconstructing three-dimensional images, and performing image recognition on the three-dimensional images to obtain the corresponding liver and gallbladder regions and tumor regions, and fusing multiple three-dimensional images under multiple modalities to obtain the final liver and gallbladder region and tumor region of the patient, the advantages of images under multiple modalities are fused to obtain a relatively accurate liver and gallbladder region and the corresponding tumor region, thereby improving the recognition accuracy of liver and gallbladder tumors and providing a relatively accurate data basis for subsequent surgeries.

[0047] In one embodiment, the above image recognition module 23 is further configured to: for each three-dimensional image, divide the three-dimensional image into multiple layered images according to a set rule; respectively recognize the liver and gallbladder region and the corresponding tumor region in each layered image; for each three-dimensional image, fuse the liver and gallbladder regions and the corresponding tumor regions in all the layered images of the three-dimensional image to obtain the liver and gallbladder region and the corresponding tumor region of the three-dimensional image.

[0048] In one embodiment, the above image recognition module 23 is further configured to: divide the three-dimensional image into multiple layered images according to a set number of layers; wherein, each layered image includes multiple consecutive two-dimensional images; splice the liver and gallbladder regions and the corresponding tumor regions in all the layered images of the three-dimensional image to obtain a spliced region image; perform post-processing on the spliced region image to obtain the liver and gallbladder region and the corresponding tumor region of the three-dimensional image.

[0049] In one embodiment, the above-mentioned image recognition module 23 is further configured to: form a path of any point located on the boundary of the liver and gall region of the spliced region image from the first-layer hierarchical image to the last-layer hierarchical image of the spliced region image; if the curvature change rate between two adjacent points on the path is greater than a first preset value, obtain the curvature change rate between two symmetric-side points on the hierarchical images where the two adjacent points are located; if the curvature change rate between the two symmetric-side points is less than a second preset value, adjust the relative position between the hierarchical images where the two adjacent points are located to reduce the curvature change rate between the two adjacent points and increase the curvature change rate between the two symmetric-side points, so as to obtain the liver and gall region of the three-dimensional image; wherein, the second preset value has the opposite sign to the first preset value.

[0050] In one embodiment, the above-mentioned image recognition module 23 is further configured to: if the curvature change rate between two symmetric-side points is greater than or equal to the second preset value, perform a smoothing process on the two adjacent points.

[0051] In one embodiment, the above-mentioned image recognition module 23 is further configured to: recognize a reference region in each hierarchical image; wherein, the reference region represents a specific region in the hierarchical image; based on the relative position relationship between the reference region and the liver and gall region, determine the interval range where the liver and gall region is located; perform binarization processing on the interval range to obtain a binarized image; recognize the liver and gall region in the binarized image; based on the liver and gall region, recognize the tumor region corresponding to the liver and gall region in the hierarchical image.

[0052] In one embodiment, the above-mentioned image fusion module 24 is further configured to: align the liver and gall regions and the corresponding tumor regions of multiple three-dimensional images, and merge the liver and gall regions and the corresponding tumor regions of the aligned multiple three-dimensional images to obtain the liver and gall region and the corresponding tumor region of the patient.

[0053] In one embodiment, the above-mentioned image fusion module 24 is further configured to: calculate the variance of the differences between the liver and gall regions of the medical image data of all modalities in each hierarchical image; select the hierarchical image with the smallest variance of differences as the marker layer image; align the liver and gall regions and the corresponding tumor regions of multiple three-dimensional images based on the marker layer image.

[0054] Next, refer to Figure 3 to describe the electronic device according to an embodiment of the present application. The electronic device can be any one or both of the first device and the second device, or a stand-alone device independent of them, and the stand-alone device can communicate with the first device and the second device to receive the input signals collected from them.

[0055] Figure 3 The block diagram of the electronic device according to an embodiment of the present application is illustrated.

[0056] As Figure 3As shown, the electronic device 10 includes one or more processors 11 and a memory 12.

[0057] The processor 11 can be a central processing unit (CPU) or other forms of processing units with data processing capabilities and / or instruction execution capabilities, and can control other components in the electronic device 10 to perform desired functions.

[0058] The memory 12 can include one or more computer program products, and the computer program products can include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. The volatile memory can include, for example, random access memory (RAM) and / or cache memory, etc. The non-volatile memory can include, for example, read-only memory (ROM), hard disk, flash memory, etc. One or more computer program instructions can be stored on the computer-readable storage media, and the processor 11 can run the program instructions to implement the methods of the various embodiments of the present application described above and / or other desired functions. Various contents such as input signals, signal components, noise components, etc. can also be stored in the computer-readable storage media.

[0059] In one example, the electronic device 10 can further include: an input device 13 and an output device 14, and these components are interconnected through a bus system and / or other forms of connection mechanisms (not shown).

[0060] When the electronic device is a stand-alone device, the input device 13 can be a communication network connector for receiving the collected input signals from the first device and the second device.

[0061] In addition, the input device 13 can further include, for example, a keyboard, a mouse, etc.

[0062] The output device 14 can output various information to the outside, including the determined distance information, direction information, etc. The output device 14 can include, for example, a display, a speaker, a printer, and a communication network and its connected remote output devices, etc.

[0063] Of course, for simplicity, Figure 3 only some of the components related to the present application in the electronic device 10 are shown, and components such as buses, input / output interfaces, etc. are omitted. In addition, according to specific application scenarios, the electronic device 10 can further include any other appropriate components.

[0064] In addition to the above methods and devices, an embodiment of the present application may also be a computer program product, which includes computer program instructions that, when run by a processor, cause the processor to execute the steps in the methods according to various embodiments of the present application described in the "Exemplary Methods" section above in this specification.

[0065] The computer program product may be written in any combination of one or more programming languages for programming code to perform the operations of the embodiments of the present application. The programming languages include object-oriented programming languages such as Java, C++, etc., and also include conventional procedural programming languages such as the "C" language or similar programming languages. The programming code may be executed entirely on the user's computing device, partially on the user's device, executed as an independent software package, partially on the user's computing device and partially on a remote computing device, or entirely on a remote computing device or server.

[0066] In addition, an embodiment of the present application may also be a computer-readable storage medium, on which computer program instructions are stored, and the computer program instructions, when run by a processor, cause the processor to execute the steps in the methods according to various embodiments of the present application described in the "Exemplary Methods" section above in this specification.

[0067] The computer-readable storage medium may adopt any combination of one or more readable media. The readable medium may be a readable signal medium or a readable storage medium. The readable storage medium may, for example, include but is not limited to an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the above. More specific examples (a non-exhaustive list) of the readable storage medium include: an electrical connection having one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0068] The basic principles of the present application have been described above in conjunction with specific embodiments. However, it should be noted that the advantages, benefits, effects, etc. mentioned in the present application are only examples and not limitations, and it cannot be considered that these advantages, benefits, effects, etc. are essential for each embodiment of the present application. In addition, the above-disclosed specific details are only for the purposes of illustration and facilitating understanding, and are not limitations. The above details do not limit the present application to necessarily adopt the above specific details for implementation.

[0069] The block diagrams of the devices, apparatuses, equipment, and systems involved in this application are only illustrative examples and are not intended to require or imply that they must be connected, arranged, and configured in the manner shown in the block diagrams. As those skilled in the art will recognize, these devices, apparatuses, equipment, and systems can be connected, arranged, and configured in any manner. Words such as "including", "comprising", "having", etc. are open-ended terms, meaning "including but not limited to", and can be used interchangeably with each other. The word "or" and "and" used herein refer to the phrase "and / or", and can be used interchangeably with it, unless the context clearly indicates otherwise. The phrase "such as" used herein refers to the phrase "such as but not limited to", and can be used interchangeably with it.

[0070] It should also be noted that in the devices, equipment, and methods of this application, each component or each step can be decomposed and / or recombined. These decompositions and / or recombinations should be regarded as equivalent solutions of this application.

[0071] The above description of the disclosed aspects is provided to enable any person skilled in the art to make or use this application. Various modifications to these aspects will be readily apparent to those skilled in the art, and the general principles defined herein can be applied to other aspects without departing from the scope of this application. Therefore, this application is not intended to be limited to the aspects shown herein, but rather to the broadest scope consistent with the principles and novel features disclosed herein.

[0072] The above description has been given for purposes of illustration and description. In addition, this description is not intended to limit the embodiments of this application to the forms disclosed herein. Although multiple example aspects and embodiments have been discussed above, those skilled in the art will recognize certain variations, modifications, changes, additions, and sub-combinations thereof.

Claims

1. A three-dimensional positioning method for hepatobiliary tumors based on detection data, characterized in that: include: Acquire multimodal medical image data of a patient; wherein each modality of medical image data includes a plurality of two-dimensional images stacked in layers; Perform three-dimensional reconstruction on the medical image data of each modality to obtain a corresponding three-dimensional image; Perform image recognition on each three-dimensional image to obtain the liver and gallbladder area and the corresponding tumor area; The hepatobiliary region and the corresponding tumor region of the multiple three-dimensional images are fused to obtain the hepatobiliary region and the corresponding tumor region of the patient.

2. The three-dimensional positioning method for hepatobiliary tumors based on detection data according to claim 1, characterized in that: The image recognition is performed on each three-dimensional image to obtain the liver and gallbladder regions and the corresponding tumor regions, including: For each three-dimensional image, the three-dimensional image is divided into multiple layers of layered images according to a set rule; Respectively identifying the hepatobiliary region and the corresponding tumor region in each layer of the layered image; For each three-dimensional image, the hepatobiliary region and the corresponding tumor region in all layered images of the three-dimensional image are fused to obtain the hepatobiliary region and the corresponding tumor region of the three-dimensional image.

3. The three-dimensional positioning method for hepatobiliary tumors based on detection data according to claim 2, characterized in that: The step of dividing the three-dimensional image into multiple layers of layered images according to a set rule comprises: Dividing the three-dimensional image into multiple layers of layered images according to a set number of layers; wherein each layer of the layered image includes multiple layers of continuous two-dimensional images; The step of fusing the hepatobiliary region and the corresponding tumor region in all layered images of the three-dimensional image to obtain the hepatobiliary region and the corresponding tumor region of the three-dimensional image comprises: splicing the liver and gallbladder regions and the corresponding tumor regions in all layered images of the three-dimensional image to obtain a spliced ​​region image; The stitched area image is post-processed to obtain the liver and gallbladder area and the corresponding tumor area of ​​the three-dimensional image.

4. The method for three-dimensional positioning of hepatobiliary tumors based on detection data according to claim 3, characterized in that: The post-processing of the stitched area image to obtain the hepatobiliary area and the corresponding tumor area of ​​the three-dimensional image includes: Forming any path located at the boundary of the hepatobiliary region of the stitched region image from the first layered image to the last layered image of the stitched region image; If the curvature change rate between two adjacent points on the path is greater than a first preset value, obtaining the curvature change rate between two points on the symmetric side of the layered image where the two adjacent points are located; If the curvature change rate between the two points on the symmetrical side is less than a second preset value, the relative position between the layered images where the two adjacent points are located is adjusted to reduce the curvature change rate between the two adjacent points and increase the curvature change rate between the two points on the symmetrical side to obtain the liver and gallbladder area of ​​the three-dimensional image; wherein the second preset value has an opposite sign to the first preset value.

5. The method for three-dimensional positioning of hepatobiliary tumors based on detection data according to claim 4, characterized in that: The post-processing of the stitched area image to obtain the hepatobiliary area and the corresponding tumor area of ​​the three-dimensional image further includes: If the curvature change rate between the two points on the symmetric side is greater than or equal to the second preset value, smoothing processing is performed on the two adjacent points.

6. The method for three-dimensional positioning of hepatobiliary tumors based on detection data according to claim 2, characterized in that: The step of respectively identifying the hepatobiliary region and the corresponding tumor region in each layer of the layered image comprises: Identifying a reference region in each layer of the layered image; wherein the reference region represents a specific region in the layered image; Based on the relative position relationship between the reference area and the hepatobiliary area, determining the interval range where the hepatobiliary area is located; Performing binarization processing on the interval range to obtain a binarized image; Identifying the liver and gallbladder region in the binary image; Based on the hepatobiliary region, a tumor region corresponding to the hepatobiliary region is identified in the layered image.

7. The three-dimensional positioning method for hepatobiliary tumors based on detection data according to claim 2, characterized in that: The fusing of the hepatobiliary region and the corresponding tumor region of the plurality of three-dimensional images to obtain the hepatobiliary region and the corresponding tumor region of the patient comprises: The hepatobiliary regions and the corresponding tumor regions of the multiple three-dimensional images are aligned, and the hepatobiliary regions and the corresponding tumor regions of the multiple three-dimensional images after alignment are merged to obtain the hepatobiliary regions and the corresponding tumor regions of the patient.

8. The method for three-dimensional positioning of hepatobiliary tumors based on detection data according to claim 7, characterized in that: The aligning of the hepatobiliary regions and the corresponding tumor regions of the plurality of three-dimensional images comprises: Calculating the difference variance between the hepatobiliary regions of the medical image data of all modalities in each layer of the layered image; Selecting the layered image with the smallest difference variance as the marker layer image; The hepatobiliary regions and corresponding tumor regions of the plurality of three-dimensional images are aligned based on the marker layer image.

9. A three-dimensional positioning system for hepatobiliary tumors based on detection data, characterized in that: include: A data acquisition module, used to acquire multimodal medical image data of a patient; wherein each modality of medical image data includes a plurality of stacked two-dimensional images; A three-dimensional reconstruction module is used to perform three-dimensional reconstruction on the medical image data of each modality to obtain a corresponding three-dimensional image; An image recognition module is used to perform image recognition on each three-dimensional image to obtain the liver and gallbladder area and the corresponding tumor area; The image fusion module is used to fuse the hepatobiliary region and the corresponding tumor region of the multiple three-dimensional images to obtain the hepatobiliary region and the corresponding tumor region of the patient.

10. A computer-readable storage medium, characterized in that: The storage medium stores a computer program, and the computer program is used to execute the method according to any one of claims 1 to 8.

Images