A hepatobiliary tumor three-dimensional positioning method and system based on detection data and a medium

Abstract

This application provides a method, system, and medium for three-dimensional localization of hepatobiliary tumors based on detection data. It acquires multimodal medical image data of the patient and performs three-dimensional reconstruction to obtain corresponding three-dimensional images. Image recognition is performed on each three-dimensional image to obtain the hepatobiliary region and the corresponding tumor region. The hepatobiliary region and the corresponding tumor region from multiple three-dimensional images are then fused to obtain the patient's hepatobiliary region and the corresponding tumor region. By acquiring multimodal medical image data, reconstructing three-dimensional images separately, and performing image recognition on the three-dimensional images to obtain the corresponding hepatobiliary region and tumor region, and fusing multiple three-dimensional images from multiple modalities, the final hepatobiliary region and tumor region of the patient are obtained. By leveraging the advantages of images from multiple modalities, a more accurate hepatobiliary region and the corresponding tumor region are obtained, thereby improving the accuracy of hepatobiliary tumor identification and providing a more accurate data foundation for subsequent surgery.

Description

Technical Field

[0001] This application relates to the field of hepatobiliary tumor detection technology, specifically to a method, system, and medium for three-dimensional localization of hepatobiliary tumors based on detection data. Background Technology

[0002] The liver and gallbladder refer to the liver and gallbladder. The liver is the main organ for metabolic functions, primarily storing glycogen, secreting synthetic proteins, producing bile, and aiding in food digestion. The gallbladder is a pear-shaped sac structure whose main functions are concentrating and storing bile, secreting mucus, and protecting the bile duct mucosa. Hepatobiliary tumors are relatively common diseases. The most direct and effective treatment for hepatobiliary tumors is tumor removal, which requires accurate knowledge of the tumor's location and size.

[0003] Currently, surgical resection of hepatobiliary tumors mostly involves acquiring medical images using equipment, then medical staff judging the location and size of the tumor based on experience, and performing the resection during surgery based on the situation on site. This method clearly relies heavily on the surgeon's experience and adaptability during the operation. Some deep learning algorithms have been able to determine the coordinates of lesions in medical images. However, the clarity of medical images is not particularly high, and there are often interference shadows and other issues. Furthermore, the accuracy of deep learning algorithms is limited, resulting in low accuracy in identifying regions of interest and lesion areas in medical images, and even potential misleading errors.

[0004] Therefore, there is an urgent need for a solution that can accurately obtain the location and size information of hepatobiliary tumors before surgery. Summary of the Invention

[0005] To address the aforementioned technical problems, this application is proposed. Embodiments of this application provide a method, system, and medium for three-dimensional localization of hepatobiliary tumors based on detection data.

[0006] According to one aspect of this application, a method for three-dimensional localization of hepatobiliary tumors based on detection data is provided, comprising: acquiring multimodal medical image data of a patient; wherein, each modality of medical image data includes multiple stacked two-dimensional images; performing three-dimensional reconstruction on each modality of medical image data to obtain a corresponding three-dimensional image; performing image recognition on each three-dimensional image to obtain a hepatobiliary region and a corresponding tumor region; and fusing the hepatobiliary region and the corresponding tumor region of multiple three-dimensional images to obtain the hepatobiliary region and the corresponding tumor region of the patient.

[0007] In one embodiment, the step of performing image recognition on each three-dimensional image to obtain the hepatobiliary region and the corresponding tumor region includes: for each three-dimensional image, dividing the three-dimensional image into multiple layers according to a set rule; identifying the hepatobiliary region and the corresponding tumor region in each layer of the layered image; and for each three-dimensional image, fusing the hepatobiliary region and the corresponding tumor region in all layers of the three-dimensional image to obtain the hepatobiliary region and the corresponding tumor region of the three-dimensional image.

[0008] In one embodiment, dividing the three-dimensional image into multiple layers according to a set rule includes: dividing the three-dimensional image into multiple layers according to a set number of layers; wherein each layer of the layered image includes multiple consecutive two-dimensional images; fusing the hepatobiliary region and the corresponding tumor region in all the layered images of the three-dimensional image to obtain the hepatobiliary region and the corresponding tumor region of the three-dimensional image includes: stitching the hepatobiliary region and the corresponding tumor region in all the layered images of the three-dimensional image to obtain a stitched region image; and performing post-processing on the stitched region image to obtain the hepatobiliary region and the corresponding tumor region of the three-dimensional image.

[0009] In one embodiment, the post-processing of the stitched region image to obtain the hepatobiliary region and the corresponding tumor region of the three-dimensional image includes: forming any path located at the boundary of the hepatobiliary region of the stitched region image from the first layer image to the last layer image; if the rate of curvature change between two adjacent points on the path is greater than a first preset value, then obtaining the rate of curvature change between two points on the symmetrical side of the layer image where the two adjacent points are located; if the rate of curvature change between the two points on the symmetrical side is less than a second preset value, then adjusting the relative position between the layer images where the two adjacent points are located to reduce the rate of curvature change between the two adjacent points and increase the rate of curvature change between the two points on the symmetrical side, thereby obtaining the hepatobiliary region of the three-dimensional image; wherein the second preset value has the opposite sign to the first preset value.

[0010] In one embodiment, the post-processing of the stitched region image to obtain the hepatobiliary region and the corresponding tumor region of the three-dimensional image further includes: if the rate of change of curvature between the two points on the symmetrical side is greater than or equal to the second preset value, then the two adjacent points are smoothed.

[0011] In one embodiment, identifying the hepatobiliary region and the corresponding tumor region in each layer of the layered image includes: identifying a reference region in each layer of the layered image; wherein the reference region represents a specific region in the layered image; determining the interval range where the hepatobiliary region is located based on the relative positional relationship between the reference region and the hepatobiliary region; performing binarization processing on the interval range to obtain a binarized image; identifying the hepatobiliary region in the binarized image; and identifying the tumor region corresponding to the hepatobiliary region in the layered image based on the hepatobiliary region.

[0012] In one embodiment, fusing the hepatobiliary region and the corresponding tumor region of multiple three-dimensional images to obtain the patient's hepatobiliary region and the corresponding tumor region includes: aligning the hepatobiliary region and the corresponding tumor region of multiple three-dimensional images, and merging the aligned hepatobiliary region and the corresponding tumor region of the multiple three-dimensional images to obtain the patient's hepatobiliary region and the corresponding tumor region.

[0013] In one embodiment, aligning the hepatobiliary region and the corresponding tumor region of multiple three-dimensional images includes: calculating the variance between the hepatobiliary regions of all modalities of medical image data in each layer of the layered image; selecting the layered image with the smallest variance as the marker layer image; and aligning the hepatobiliary region and the corresponding tumor region of multiple three-dimensional images based on the marker layer image.

[0014] According to another aspect of this application, a three-dimensional localization system for hepatobiliary tumors based on detection data is provided, comprising: a data acquisition module for acquiring multimodal medical image data of a patient; wherein each modality of medical image data includes multiple stacked two-dimensional images; a three-dimensional reconstruction module for performing three-dimensional reconstruction on each modality of medical image data to obtain a corresponding three-dimensional image; an image recognition module for performing image recognition on each three-dimensional image to obtain the hepatobiliary region and the corresponding tumor region; and an image fusion module for fusing the hepatobiliary region and the corresponding tumor region of the multiple three-dimensional images to obtain the hepatobiliary region and the corresponding tumor region of the patient.

[0015] According to another aspect of this application, a computer-readable storage medium is provided, the storage medium storing a computer program for performing any of the methods described above.

[0016] This application provides a method, system, and medium for three-dimensional localization of hepatobiliary tumors based on detection data. The method involves acquiring multimodal medical image data from a patient. Each modality of medical image data includes multiple stacked two-dimensional images. Three-dimensional reconstruction is performed on each modality to obtain a corresponding three-dimensional image. Image recognition is performed on each three-dimensional image to obtain the hepatobiliary region and the corresponding tumor region. The hepatobiliary region and the corresponding tumor region from multiple three-dimensional images are then fused to obtain the patient's hepatobiliary region and the corresponding tumor region. By acquiring multimodal medical image data, reconstructing three-dimensional images, and performing image recognition on the three-dimensional images to obtain the corresponding hepatobiliary region and tumor region, and fusing multiple three-dimensional images from multiple modalities to obtain the patient's final hepatobiliary region and tumor region, the method leverages the advantages of fusing images from multiple modalities to obtain a more accurate hepatobiliary region and the corresponding tumor region. This improves the accuracy of hepatobiliary tumor identification and provides a more accurate data foundation for subsequent surgery. Attached Figure Description

[0017] The above and other objects, features, and advantages of this application will become more apparent from the more detailed description of the embodiments of this application in conjunction with the accompanying drawings. The drawings are provided to further illustrate the embodiments of this application and form part of the specification. They are used together with the embodiments of this application to explain this application and do not constitute a limitation thereof. In the drawings, the same reference numerals generally represent the same components or steps.

[0018] Figure 1 This is a flowchart illustrating an exemplary embodiment of the present application of a method for three-dimensional localization of hepatobiliary tumors based on detection data.

[0019] Figure 2 This is a schematic diagram of the structure of a three-dimensional localization system for hepatobiliary tumors based on detection data provided in an exemplary embodiment of this application.

[0020] Figure 3 This is a structural diagram of an electronic device provided in an exemplary embodiment of this application. Detailed Implementation

[0021] Hereinafter, exemplary embodiments according to this application will be described in detail with reference to the accompanying drawings. Obviously, the described embodiments are merely some embodiments of this application, and not all embodiments of this application. It should be understood that this application is not limited to the exemplary embodiments described herein.

[0022] Figure 1 This is a flowchart illustrating an exemplary embodiment of the present application of a method for three-dimensional localization of hepatobiliary tumors based on detection data. Figure 1 As shown, the three-dimensional localization method for hepatobiliary tumors based on detection data includes the following steps:

[0023] Step 110: Acquire the patient's multimodal medical image data.

[0024] Each modality of medical image data includes multiple stacked two-dimensional images. This application acquires medical image data of patients in multiple modalities using various medical imaging devices, such as CT images, MRI images, and ultrasound images.

[0025] Step 120: Perform 3D reconstruction on the medical image data of each modality to obtain the corresponding 3D image.

[0026] After acquiring medical image data of multiple modalities, three-dimensional reconstruction was performed on the medical image data of each modality to obtain a three-dimensional image corresponding to each modality, that is, a three-dimensional image containing the liver and gallbladder region and the tumor region.

[0027] Step 130: Perform image recognition on each 3D image to obtain the liver and gallbladder region and the corresponding tumor region.

[0028] This application performs image recognition on each three-dimensional image to obtain the hepatobiliary region and the corresponding tumor region in each three-dimensional image.

[0029] Step 140: Fuse the hepatobiliary region and the corresponding tumor region from multiple three-dimensional images to obtain the patient's hepatobiliary region and the corresponding tumor region.

[0030] After obtaining the hepatobiliary region and the corresponding tumor region in each three-dimensional image, this application fuses the hepatobiliary region and the corresponding tumor region from multiple three-dimensional images to obtain the final hepatobiliary region and the corresponding tumor region. That is, the advantages of fusing all three-dimensional images are mutually verified and assisted to improve the accuracy of identifying the patient's hepatobiliary region and the corresponding tumor region.

[0031] This application provides a three-dimensional localization method for hepatobiliary tumors based on detection data. The method acquires multimodal medical image data of the patient. Each modality of medical image data includes multiple stacked two-dimensional images. Three-dimensional reconstruction is performed on each modality of medical image data to obtain a corresponding three-dimensional image. Image recognition is performed on each three-dimensional image to obtain the hepatobiliary region and the corresponding tumor region. The hepatobiliary region and the corresponding tumor region from multiple three-dimensional images are fused to obtain the patient's hepatobiliary region and the corresponding tumor region. By acquiring multimodal medical image data, reconstructing three-dimensional images separately, and performing image recognition on the three-dimensional images to obtain the corresponding hepatobiliary region and tumor region, and fusing multiple three-dimensional images from multiple modalities to obtain the patient's final hepatobiliary region and tumor region, the method leverages the advantages of fusing images from multiple modalities to obtain a more accurate hepatobiliary region and the corresponding tumor region, thereby improving the accuracy of hepatobiliary tumor identification and providing a more accurate data foundation for subsequent surgery.

[0032] In one embodiment, the specific implementation of step 130 above may be as follows: for each three-dimensional image, the three-dimensional image is divided into multiple layers according to a set rule; the hepatobiliary region and the corresponding tumor region in each layer are identified respectively; for each three-dimensional image, the hepatobiliary region and the corresponding tumor region in all layers of the three-dimensional image are fused to obtain the hepatobiliary region and the corresponding tumor region of the three-dimensional image.

[0033] This application divides a three-dimensional image into multiple layers and identifies each layer to obtain the hepatobiliary region and the corresponding tumor region of the layered image. After obtaining the hepatobiliary region and the corresponding tumor region of the layered image, the hepatobiliary region and the corresponding tumor region of the three-dimensional image are obtained by fusing the hepatobiliary region and the corresponding tumor region of all layers of each three-dimensional image.

[0034] In one embodiment, step 130 can be implemented as follows: the three-dimensional image is divided into multiple layered images according to a set number of layers; wherein each layered image includes multiple consecutive two-dimensional images; the hepatobiliary region and the corresponding tumor region in all the layered images of the three-dimensional image are stitched together to obtain a stitched region image; the stitched region image is post-processed to obtain the hepatobiliary region and the corresponding tumor region of the three-dimensional image.

[0035] Specifically, this application can use multiple layers of continuous two-dimensional images as a single layer image. By utilizing the continuity between the continuous layers of two-dimensional images, the hepatobiliary region and the corresponding tumor region of each layer of two-dimensional images can be mutually verified, thereby improving the accuracy of identifying the hepatobiliary region and the corresponding tumor region of the layered images. After identifying the hepatobiliary region and the corresponding tumor region of the layered images with relatively accurate identification, the layered images are stitched together to obtain the overall three-dimensional image of the hepatobiliary region and the corresponding tumor region. Furthermore, the stitched image is post-processed after stitching to further improve the accuracy of identifying the hepatobiliary region and the corresponding tumor region.

[0036] In one embodiment, step 130 can be implemented as follows: a path is formed from the first layer of the stitched region image to the last layer of the stitched region image, which is located at the boundary of the hepatobiliary region of the stitched region image; if the rate of curvature change between two adjacent points on the path is greater than a first preset value, the rate of curvature change between two symmetrical points on the layered image where the two adjacent points are located is obtained; if the rate of curvature change between two symmetrical points is less than a second preset value, the relative position between the layered images where the two adjacent points are located is adjusted to reduce the rate of curvature change between the two adjacent points and increase the rate of curvature change between the two symmetrical points, thereby obtaining the hepatobiliary region of the three-dimensional image; wherein the second preset value has the opposite sign to the first preset value.

[0037] Specifically, this application constructs a path starting from the first layer image, passing through each layer image, and ending at the last layer image. For example, starting from any point on the boundary of the hepatobiliary region of the first layer image, the tangent direction of that point is verified to connect points on the boundaries of the hepatobiliary regions of each layer image to form a path. It should be understood that the path in this application can include many paths. To simplify the scheme, this application can select multiple starting points corresponding to specific angles to form a limited number of paths. After obtaining a path, this application determines whether there is a curvature change rate between two adjacent points on the path that is greater than a first preset value. If so, it obtains the curvature change rate between two points on the symmetrical side of the layered image where the two adjacent points are located. For example, taking a fixed cross-section as a reference, if there is a curvature change rate between two adjacent points on the left path that is greater than the first preset value, it indicates that there is a sudden change in the position of the two adjacent points on the left path. At this time, it determines whether the curvature change rate between the corresponding two points on the right path is less than a second preset value (the sign is opposite to the first preset value). If it is less, it indicates that one layer of the layered image is misaligned with other layered images. Then, it adjusts the relative position between the layered images where the two adjacent points are located to reduce the curvature change rate between the two adjacent points and increase the curvature change rate between the two points on the symmetrical side, thereby obtaining the liver and gallbladder region of the three-dimensional image, thereby improving the accuracy of stitching and thus improving the accuracy of recognition.

[0038] In one embodiment, step 130 can be implemented as follows: if the rate of change of curvature between two points on the symmetrical side is greater than or equal to a second preset value, then the two adjacent points are smoothed.

[0039] If there is a sudden change in the position of the two adjacent points and the rate of change of curvature between the two points on the symmetrical side is greater than or equal to the second preset value, it indicates that the sudden change in the position of the two adjacent points is not caused by splicing misalignment. In this case, the two adjacent points are smoothed to solve the problem of the sudden change in the position of the two adjacent points and improve the recognition accuracy.

[0040] In one embodiment, step 130 can be implemented as follows: identifying a reference region in each layered image; wherein the reference region represents a specific region in the layered image; determining the range of the liver and gallbladder region based on the relative positional relationship between the reference region and the liver and gallbladder region; performing binarization processing on the range to obtain a binarized image; identifying the liver and gallbladder region in the binarized image; and identifying the tumor region corresponding to the liver and gallbladder region in the layered image based on the liver and gallbladder region.

[0041] This application improves the accuracy of liver and gallbladder region identification by identifying reference regions (such as the rib area) in layered images and excluding regions similar to the liver and gallbladder region based on the relative positional relationship between the reference region and the liver and gallbladder region. After determining the range of the liver and gallbladder region, the range is binarized, and the liver and gallbladder region is identified in the binarized image. Based on the identified liver and gallbladder region, the tumor region of the liver and gallbladder region is identified in the layered image, that is, the liver and gallbladder region is identified after eliminating background interference. Then, the tumor region is identified within the liver and gallbladder region in the layered image to eliminate interference factors outside the range of the liver and gallbladder region, thereby improving the identification accuracy of the liver and gallbladder region and the tumor region.

[0042] In one embodiment, step 140 can be implemented by aligning the hepatobiliary region and the corresponding tumor region of multiple three-dimensional images, and merging the aligned hepatobiliary region and the corresponding tumor region of the multiple three-dimensional images to obtain the patient's hepatobiliary region and the corresponding tumor region.

[0043] This application improves the accuracy of merging by combining the hepatobiliary region and the corresponding tumor region from multiple three-dimensional images and merging the aligned multiple three-dimensional images of the hepatobiliary region and the corresponding tumor region, thereby improving the accuracy of identification.

[0044] In one embodiment, step 140 can be implemented as follows: calculate the variance of the difference between the hepatobiliary regions of all modal medical image data in each layer of the layered image; select the layered image with the smallest variance as the marker layer image; and align the hepatobiliary regions and corresponding tumor regions of multiple three-dimensional images based on the marker layer image.

[0045] This application calculates the variance between the hepatobiliary regions of all modalities in the medical image data of each layer to determine the differences between the hepatobiliary regions in each layer image, and selects the layer image with the smallest variance as the marker layer image. By adjusting the position of other layer images, the alignment operation of the three-dimensional images is achieved, thereby reducing the differences between the hepatobiliary regions and the corresponding tumor regions in each aligned three-dimensional image, and thus improving the final recognition accuracy.

[0046] Figure 2 This is a schematic diagram of the structure of a three-dimensional localization system for hepatobiliary tumors based on detection data, provided in an exemplary embodiment of this application. Figure 2As shown, the hepatobiliary tumor three-dimensional localization system 20 based on detection data includes: a data acquisition module 21, used to acquire multimodal medical image data of the patient; wherein, the medical image data of each modality includes multiple stacked two-dimensional images; a three-dimensional reconstruction module 22, used to perform three-dimensional reconstruction on the medical image data of each modality to obtain the corresponding three-dimensional image; an image recognition module 23, used to perform image recognition on each three-dimensional image to obtain the hepatobiliary region and the corresponding tumor region; and an image fusion module 24, used to fuse the hepatobiliary region and the corresponding tumor region of multiple three-dimensional images to obtain the patient's hepatobiliary region and the corresponding tumor region.

[0047] This application provides a three-dimensional localization system for hepatobiliary tumors based on detection data. The system acquires multimodal medical image data of the patient through a data acquisition module 21. Each modality of medical image data includes multiple stacked two-dimensional images. A three-dimensional reconstruction module 22 performs three-dimensional reconstruction on each modality of medical image data to obtain the corresponding three-dimensional image. An image recognition module 23 performs image recognition on each three-dimensional image to obtain the hepatobiliary region and the corresponding tumor region. An image fusion module 24 fuses the hepatobiliary region and the corresponding tumor region from multiple three-dimensional images to obtain the patient's hepatobiliary region and the corresponding tumor region. By acquiring multimodal medical image data, reconstructing three-dimensional images, and performing image recognition on the three-dimensional images to obtain the corresponding hepatobiliary region and tumor region, and fusing multiple three-dimensional images from multiple modalities to obtain the patient's final hepatobiliary region and tumor region, the system leverages the advantages of fusing images from multiple modalities to obtain a more accurate hepatobiliary region and the corresponding tumor region, thereby improving the accuracy of hepatobiliary tumor identification and providing a more accurate data foundation for subsequent surgery.

[0048] In one embodiment, the image recognition module 23 is further configured to: divide each three-dimensional image into multiple layers according to a set rule; identify the hepatobiliary region and the corresponding tumor region in each layer; and fuse the hepatobiliary region and the corresponding tumor region in all layers of the three-dimensional image for each three-dimensional image to obtain the hepatobiliary region and the corresponding tumor region of the three-dimensional image.

[0049] In one embodiment, the image recognition module 23 is further configured to: divide the three-dimensional image into multiple layered images according to a set number of layers; wherein each layered image includes multiple consecutive two-dimensional images; stitch together the hepatobiliary region and the corresponding tumor region in all layered images of the three-dimensional image to obtain a stitched region image; and perform post-processing on the stitched region image to obtain the hepatobiliary region and the corresponding tumor region of the three-dimensional image.

[0050] In one embodiment, the image recognition module 23 is further configured to: form any path located at the boundary of the hepatobiliary region of the stitched region image from the first layer of the stitched region image to the last layer of the stitched region image; if the rate of curvature change between two adjacent points on the path is greater than a first preset value, then obtain the rate of curvature change between two points on the symmetrical side of the layered image where the two adjacent points are located; if the rate of curvature change between two points on the symmetrical side is less than a second preset value, then adjust the relative position between the layered images where the two adjacent points are located to reduce the rate of curvature change between the two adjacent points and increase the rate of curvature change between the two points on the symmetrical side, thereby obtaining the hepatobiliary region of the three-dimensional image; wherein, the second preset value has the opposite sign to the first preset value.

[0051] In one embodiment, the image recognition module 23 is further configured to: if the rate of change of curvature between two points on the symmetrical side is greater than or equal to a second preset value, then smooth the two adjacent points.

[0052] In one embodiment, the image recognition module 23 is further configured to: identify a reference region in each layered image; wherein the reference region represents a specific region in the layered image; determine the range of the liver and gallbladder region based on the relative positional relationship between the reference region and the liver and gallbladder region; perform binarization processing on the range to obtain a binarized image; identify the liver and gallbladder region in the binarized image; and identify the tumor region corresponding to the liver and gallbladder region in the layered image based on the liver and gallbladder region.

[0053] In one embodiment, the image fusion module 24 is further configured to: align the hepatobiliary region and the corresponding tumor region of multiple three-dimensional images, and merge the aligned hepatobiliary region and the corresponding tumor region of the multiple three-dimensional images to obtain the patient's hepatobiliary region and the corresponding tumor region.

[0054] In one embodiment, the image fusion module 24 is further configured to: calculate the variance between the hepatobiliary regions of all modalities of medical image data in each layered image; select the layered image with the smallest variance as the marker layer image; and align the hepatobiliary regions and corresponding tumor regions of multiple three-dimensional images based on the marker layer image.

[0055] Below, for reference Figure 3 This application describes an electronic device according to embodiments thereof. The electronic device may be either or both of a first device and a second device, or a standalone device independent of them, which may communicate with the first device and the second device to receive acquired input signals from them.

[0056] Figure 3 A block diagram of an electronic device according to an embodiment of this application is illustrated.

[0057] like Figure 3As shown, the electronic device 10 includes one or more processors 11 and memory 12.

[0058] The processor 11 may be a central processing unit (CPU) or other form of processing unit with data processing capabilities and / or instruction execution capabilities, and may control other components in the electronic device 10 to perform desired functions.

[0059] The memory 12 may include one or more computer program products, which may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. The volatile memory may include, for example, random access memory (RAM) and / or cache memory. The non-volatile memory may include, for example, read-only memory (ROM), hard disk, flash memory, etc. One or more computer program instructions may be stored on the computer-readable storage medium, and the processor 11 may execute the program instructions to implement the methods of the various embodiments of this application described above and / or other desired functions. Various contents such as input signals, signal components, and noise components may also be stored in the computer-readable storage medium.

[0060] In one example, the electronic device 10 may also include an input device 13 and an output device 14, which are interconnected via a bus system and / or other forms of connection mechanism (not shown).

[0061] When the electronic device is a standalone device, the input device 13 can be a communication network connector for receiving the collected input signals from the first device and the second device.

[0062] In addition, the input device 13 may also include, for example, a keyboard, a mouse, etc.

[0063] The output device 14 can output various information to the outside, including determined distance information, direction information, etc. The output device 14 may include, for example, a display, a speaker, a printer, and a communication network and its connected remote output devices, etc.

[0064] Of course, for the sake of simplicity, Figure 3 Only some of the components of the electronic device 10 relevant to this application are shown in this illustration; components such as buses, input / output interfaces, etc., are omitted. In addition, the electronic device 10 may include any other suitable components depending on the specific application.

[0065] In addition to the methods and apparatus described above, embodiments of this application may also be computer program products, which include computer program instructions that, when executed by a processor, cause the processor to perform the steps in the methods according to various embodiments of this application described in the "Exemplary Methods" section above.

[0066] The computer program product can be written in any combination of one or more programming languages ​​to perform the operations of the embodiments of this application. The programming languages ​​include object-oriented programming languages ​​such as Java and C++, as well as conventional procedural programming languages ​​such as C or similar languages. The program code can be executed entirely on the user's computing device, partially on the user's computing device, as a standalone software package, partially on the user's computing device and partially on a remote computing device, or entirely on a remote computing device or server.

[0067] Furthermore, embodiments of this application may also be computer-readable storage media storing computer program instructions thereon, which, when executed by a processor, cause the processor to perform the steps in the methods according to various embodiments of this application described in the "Exemplary Methods" section above.

[0068] The computer-readable storage medium may be any combination of one or more readable media. A readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may be, for example, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of readable storage media (a non-exhaustive list) include: an electrical connection having one or more wires, a portable disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof.

[0069] The basic principles of this application have been described above with reference to specific embodiments. However, it should be noted that the advantages, benefits, and effects mentioned in this application are merely examples and not limitations, and should not be considered as essential features of each embodiment of this application. Furthermore, the specific details disclosed above are for illustrative and facilitative purposes only, and are not limitations. These details do not limit the application to the necessity of employing the aforementioned specific details for implementation.

[0070] The block diagrams of devices, apparatuses, devices, and systems involved in this application are merely illustrative examples and are not intended to require or imply that they must be connected, arranged, or configured in the manner shown in the block diagrams. As those skilled in the art will recognize, these devices, apparatuses, devices, and systems can be connected, arranged, and configured in any manner. Words such as “comprising,” “including,” “having,” etc., are open-ended terms meaning “including but not limited to,” and are used interchangeably with them. The terms “or” and “and” as used herein refer to the terms “and / or,” and are used interchangeably with them unless the context clearly indicates otherwise. The term “such as” as used herein refers to the phrase “such as but not limited to,” and is used interchangeably with it.

[0071] It should also be noted that in the apparatus, equipment, and methods of this application, the components or steps can be disassembled and / or recombined. These disassemblies and / or recombinations should be considered as equivalent solutions of this application.

[0072] The above description of the disclosed aspects is provided to enable any person skilled in the art to make or use this application. Various modifications to these aspects will be readily apparent to those skilled in the art, and the general principles defined herein can be applied to other aspects without departing from the scope of this application. Therefore, this application is not intended to be limited to the aspects shown herein, but rather to be accorded the widest scope consistent with the principles and novel features disclosed herein.

[0073] The above description has been given for purposes of illustration and description. Furthermore, this description is not intended to limit the embodiments of this application to the forms disclosed herein. Although numerous exemplary aspects and embodiments have been discussed above, those skilled in the art will recognize certain variations, modifications, alterations, additions, and sub-combinations thereof.

Claims

1. A three-dimensional localization method for hepatobiliary tumors based on detection data, characterized in that, include: Acquire multimodal medical image data of patients; wherein, the medical image data of each modality includes multiple stacked two-dimensional images; Three-dimensional reconstruction is performed on the medical image data of each modality to obtain the corresponding three-dimensional image; Image recognition was performed on each 3D image to obtain the hepatobiliary region and the corresponding tumor region; By fusing the hepatobiliary region and the corresponding tumor region from multiple three-dimensional images, the hepatobiliary region and the corresponding tumor region of the patient are obtained. The step of performing image recognition on each three-dimensional image to obtain the hepatobiliary region and the corresponding tumor region includes: For each 3D image, the 3D image is divided into multiple layers according to a set rule; Identify the hepatobiliary region and the corresponding tumor region in each layer of the layered image; For each three-dimensional image, the hepatobiliary region and the corresponding tumor region in all layered images of the three-dimensional image are fused to obtain the hepatobiliary region and the corresponding tumor region of the three-dimensional image. The step of dividing the three-dimensional image into multiple layers according to a set rule includes: The three-dimensional image is divided into multiple layered images according to a set number of layers; wherein each layered image includes multiple consecutive two-dimensional images; The step of fusing the hepatobiliary region and the corresponding tumor region in all layered images of the three-dimensional image to obtain the hepatobiliary region and the corresponding tumor region of the three-dimensional image includes: The hepatobiliary region and the corresponding tumor region in all the layered images of the three-dimensional image are stitched together to obtain the stitched region image. Post-processing is performed on the stitched region image to obtain the hepatobiliary region and the corresponding tumor region of the three-dimensional image; The post-processing of the stitched region image to obtain the hepatobiliary region and corresponding tumor region of the three-dimensional image includes: A path can be formed from the first layer of the stitched region image to the last layer of the stitched region image, which is located at the boundary of the hepatobiliary region of the stitched region image. If the rate of curvature change between two adjacent points on the path is greater than a first preset value, then the rate of curvature change between two points on the symmetrical side of the layered image where the two adjacent points are located is obtained. If the rate of curvature change between the two points on the symmetrical side is less than the second preset value, then the relative position between the layered images where the two adjacent points are located is adjusted to reduce the rate of curvature change between the two adjacent points and increase the rate of curvature change between the two points on the symmetrical side, thereby obtaining the hepatobiliary region of the three-dimensional image; wherein, the second preset value has the opposite sign to the first preset value; The post-processing of the stitched region image to obtain the hepatobiliary region and corresponding tumor region of the three-dimensional image further includes: If the rate of change of curvature between the two points on the symmetrical side is greater than or equal to the second preset value, then the two adjacent points are smoothed.

2. The method for three-dimensional localization of hepatobiliary tumors based on detection data according to claim 1, characterized in that, The step of identifying the hepatobiliary region and the corresponding tumor region in each layer of the layered image includes: Identify a reference region in each layer of the layered image; wherein the reference region represents a specific region in the layered image; Based on the relative positional relationship between the reference region and the hepatobiliary region, the range of the hepatobiliary region is determined; The interval range is binarized to obtain a binarized image; The liver and gallbladder region is identified in the binarized image; Based on the hepatobiliary region, the tumor region corresponding to the hepatobiliary region is identified in the layered image.

3. The method for three-dimensional localization of hepatobiliary tumors based on detection data according to claim 1, characterized in that, The method of fusing multiple three-dimensional images of the hepatobiliary region and the corresponding tumor region to obtain the patient's hepatobiliary region and corresponding tumor region includes: The liver and gallbladder regions and corresponding tumor regions of multiple three-dimensional images are aligned, and the aligned liver and gallbladder regions and corresponding tumor regions of the multiple three-dimensional images are merged to obtain the liver and gallbladder regions and corresponding tumor regions of the patient.

4. The three-dimensional localization method for hepatobiliary tumors based on detection data according to claim 3, characterized in that, The alignment of the hepatobiliary region and the corresponding tumor region of the multiple three-dimensional images includes: Calculate the variance of the differences between the hepatobiliary regions of medical image data of all modalities in each layer of the layered image; The layered image with the smallest variance is selected as the marker layer image; Based on the marker layer image, the hepatobiliary region and the corresponding tumor region of multiple three-dimensional images are aligned.

5. A three-dimensional localization system for hepatobiliary tumors based on detection data, characterized in that, include: The data acquisition module is used to acquire multimodal medical image data of patients; wherein, the medical image data of each modality includes multiple stacked two-dimensional images; The 3D reconstruction module is used to perform 3D reconstruction on medical image data of each modality to obtain the corresponding 3D image; The image recognition module is used to perform image recognition on each 3D image to obtain the liver and gallbladder region and the corresponding tumor region. An image fusion module is used to fuse the hepatobiliary region and the corresponding tumor region of multiple three-dimensional images to obtain the hepatobiliary region and the corresponding tumor region of the patient. The image recognition module is configured as follows: For each 3D image, the 3D image is divided into multiple layers according to a set rule; Identify the hepatobiliary region and the corresponding tumor region in each layer of the layered image; For each three-dimensional image, the hepatobiliary region and the corresponding tumor region in all layered images of the three-dimensional image are fused to obtain the hepatobiliary region and the corresponding tumor region of the three-dimensional image. The image recognition module is configured as follows: The three-dimensional image is divided into multiple layered images according to a set number of layers; wherein each layered image includes multiple consecutive two-dimensional images; The hepatobiliary region and the corresponding tumor region in all the layered images of the three-dimensional image are stitched together to obtain the stitched region image. Post-processing is performed on the stitched region image to obtain the hepatobiliary region and the corresponding tumor region of the three-dimensional image; The image recognition module is configured as follows: A path can be formed from the first layer of the stitched region image to the last layer of the stitched region image, which is located at the boundary of the hepatobiliary region of the stitched region image. If the rate of curvature change between two adjacent points on the path is greater than a first preset value, then the rate of curvature change between two points on the symmetrical side of the layered image where the two adjacent points are located is obtained. If the rate of curvature change between the two points on the symmetrical side is less than the second preset value, then the relative position between the layered images where the two adjacent points are located is adjusted to reduce the rate of curvature change between the two adjacent points and increase the rate of curvature change between the two points on the symmetrical side, thereby obtaining the hepatobiliary region of the three-dimensional image; wherein, the second preset value has the opposite sign to the first preset value; If the rate of change of curvature between the two points on the symmetrical side is greater than or equal to the second preset value, then the two adjacent points are smoothed.

6. A computer-readable storage medium, characterized in that, The storage medium stores a computer program for performing the method described in any one of claims 1-4.

Images