Bifurcation point detection method, image registration method, image segmentation method and system

Abstract

This invention provides a branching point detection method, an image registration method, an image segmentation method, and a system. The tree-structure branching point detection method uses a deep learning network to detect branching points in a tree structure. It is characterized by the following steps: a training step, where a model for detecting branching points in the tree structure is trained using the deep learning network based on a branching point topology atlas of the tree structure; and an inference step, where the branching points detected by the model trained in the training step are reinforced using the branching point topology atlas.

Description

Technical Field

[0001] This invention relates to a tree branching point detection method and system for detecting branching points of a tree structure in an image, an image registration method and system using the method, and an image segmentation method and system. Background Technology

[0002] The trachea and blood vessels inside the lungs or liver are vital tissues in the human body. These internal blood vessels or trachea typically have a tree-like structure, and the detection or segmentation of these tree-like structures is an important research topic. Furthermore, these tree-like structures are also significant for the segmentation of organs with such structures and for multimodal image registration.

[0003] Patent Document 1 (WO2020 / 044840) discloses a device for region segmentation using tracheal structure, used to segment lung regions contained in medical images. Regarding the detection and localization of landmarks in the trachea, which has a tree-like structure, Patent Document 1 describes extracting bronchial structures including bifurcation points to segment regions based on the locations of multiple bifurcation points within the bronchial region. Furthermore, Patent Document 1 describes using machine learning to extract bronchial structures. Summary of the Invention

[0004] Problems in the prior art

[0005] As mentioned above, for a tree-like structure, the bifurcation point is an important key point. Patent Document 1 proposes to use the locations of multiple bifurcation points in the bronchial region for region division. However, the technical solution of Patent Document 1 only involves region division rather than image segmentation. Specifically, Patent Document 1 uses the location of the bifurcation point to divide the lung region into multiple regions in the vertical direction, or divides the lung region into regions within a specific distance from a specific bifurcation point and regions outside the specific distance (see paragraph 0042 of Patent Document 1 specification).

[0006] Furthermore, most existing segmentation methods extract the centerline and branching points of the tree structure from the segmented image after segmenting the tree structure. These traditional methods often perform tree structure segmentation directly from the image based on image intensity, geometry, etc., or directly perform tree structure image segmentation based on deep learning. The branching points, as key points, are not fully utilized in tree structure segmentation.

[0007] In addition, as a branching point detection technology, although some machine learning-based tree structures or tree structure branching point detection methods have been proposed in existing technologies such as Patent Document 1, these existing branching point detection methods do not consider the correspondence between branching points, resulting in insufficient detection precision and accuracy.

[0008] Means for solving technical problems

[0009] This invention addresses the problems of the prior art by proposing a tree-structured method and system for detecting branching points that considers the correspondence between branching points. Furthermore, it proposes an image registration method, an image segmentation method, and a system based on this branching point detection method.

[0010] According to one aspect of the present invention, a method for detecting branching points in a tree structure is provided, which detects branching points in a tree structure using a deep learning network. The method comprises the following steps: a training step, in which a model for detecting branching points in the tree structure is trained using the deep learning network through a branching point topology atlas of the tree structure; and an inference step, in which the branching points detected by the model trained through the training step are reinforced using the branching point topology atlas.

[0011] Therefore, this invention utilizes a tree-structured keypoint detection model trained by a deep learning network to detect branching points in the tree structure. During training, the deep learning network can leverage the topological information of multiple keypoints in the tree structure. During inference, it combines the branching point topology atlas with the detected branching points in the tree structure to enhance the detection results. This allows the corresponding information between the various branching points in the tree structure to be used for branching point detection, thereby improving the accuracy and precision of branching point detection.

[0012] In addition, the present invention provides an image registration method and an image segmentation method based on the above-mentioned bifurcation point detection method.

[0013] Specifically, according to another aspect of the present invention, an image registration method is provided, which registers images of multiple modalities of an organ using the branching points of the tree structure. The method is characterized by comprising the following steps: an acquisition step, acquiring images of the multiple modalities of the organ; a tree branching point detection step, detecting the branching points of the tree structure in the acquired images of the organ using the aforementioned tree branching point detection method; and a registration step, registering the images of the multiple modalities using the detected branching points.

[0014] According to another aspect of the present invention, an image segmentation method is provided for segmenting organs with a tree-like structure in a medical image, characterized by comprising the following steps: an acquisition step, acquiring the medical image; a tree-like structure branching point detection step, detecting the branching points of the tree-like structure in the acquired medical image using the above-described tree-like structure branching point detection method; a centerline generation step, generating a centerline of the tree-like structure using the detected branching points; and a segmentation step, segmenting the tree-like structure according to the generated centerline of the tree-like structure.

[0015] The present invention can also be implemented as a tree-structured branch point detection system, an image registration system, or an image segmentation system that has functional modules capable of implementing the steps of the above methods; it can also be implemented as a computer program that enables a computer to execute the steps included in the above methods; or it can be implemented as a recording medium that records the above computer program.

[0016] According to the present invention, the corresponding information between the various branching points of the tree structure is used for branching point detection, thereby improving the accuracy and precision of branching point detection.

[0017] Furthermore, according to the present invention, the bifurcation point detection results, which improve the accuracy and precision of bifurcation point detection, are used for image registration such as multimodal image registration, thereby improving the efficiency and precision of registration.

[0018] Furthermore, according to the present invention, the bifurcation point detection results, which improve the accuracy and precision of bifurcation point detection, are used for tree-structured image segmentation, thereby improving the efficiency and precision of segmentation. Attached Figure Description

[0019] Figure 1 This is a flowchart illustrating the training and inference process of the tree-structured branching point detection model according to the first embodiment of the present invention;

[0020] Figure 2 This is a schematic diagram illustrating the specific process of training the tree-structured branching point detection model according to the first embodiment of the present invention;

[0021] Figure 3 This is a schematic diagram illustrating the specific flow of the reasoning process of the tree-structured branching point detection model according to the first embodiment of the present invention;

[0022] Figure 4 This is a flowchart illustrating the image registration method according to a second embodiment of the present invention;

[0023] Figure 5 This is an explanatory diagram illustrating the process of correcting the detection results in the image registration method according to the second embodiment of the present invention;

[0024] Figure 6 This is a flowchart illustrating an image segmentation method according to a third embodiment of the present invention;

[0025] Figure 7 This is a functional block diagram illustrating the tree-structure branching point detection system according to the first embodiment of the present invention. Detailed Implementation

[0026] This invention relates to methods and systems for bifurcation point detection, image registration, and image segmentation. These methods can be implemented by executing software programs using a standalone computer or other device with a CPU (central processing unit). These systems can be implemented as the aforementioned standalone computer or as hardware circuits capable of executing the various steps of the aforementioned methods. Furthermore, the system of this invention can also be pre-installed as part of a medical image acquisition device such as a magnetic resonance imaging (MRI) device.

[0027] Hereinafter, preferred embodiments of the present invention will be described with reference to the accompanying drawings.

[0028] Furthermore, in different embodiments, the same reference numerals are used for the same components, and repeated descriptions are omitted where appropriate.

[0029] <First Implementation Method>

[0030] According to the first embodiment, the present invention is a method and system for detecting branching points in a tree structure, which utilizes a key point detection model for a tree structure trained by a deep learning network to detect branching points in the tree structure.

[0031] A complete deep learning framework for detection models comprises two main parts: training and inference. The training process inputs a labeled training dataset (or ground truth, GT) into the model, calculates the objective function (loss function) between the output detection results and the ground truth, and corrects the network parameters using methods such as gradient descent and stochastic gradient descent to minimize the loss function. This process is repeated until the error between the network's output detection results and the ground truth meets the specified accuracy, indicating that the model has converged and the error in the model's predictions has been reduced. The inference process involves inputting unlabeled live data into the trained model to obtain the actual detection values. The tree-structured branching point detection method of this invention consists of the above-described training and inference processes.

[0032] The first embodiment of the present invention will be described in detail below with reference to the accompanying drawings.

[0033] Figure 1This is a flowchart illustrating the training and inference process of the tree-structured branching point detection model according to the first embodiment of the present invention, wherein... Figure 1 (a) represents the training process. Figure 1 (b) represents the reasoning process.

[0034] like Figure 1 As shown, the training process of the tree-structured branch point detection model of the present invention includes the following steps: Step S100, inputting an image set and ground truth (GT) values ​​of key points; Step S200, generating sample and key point heatmaps; Step S300, training the key point detector using a deep learning network; Step S400, outputting the trained model. The inference process of the tree-structured branch point detection model of the present invention includes the following steps: Step S100', inputting an image set into the trained model; Step S200', detecting key points using the trained model; Step S300', enhancing the detection results using a key point topology atlas.

[0035] The following combination Figure 2 and Figure 3 Provide a detailed explanation of the training and reasoning processes.

[0036] Figure 2 This is a schematic diagram illustrating the specific process of training a tree-structured branching point detection model according to the first embodiment of the present invention.

[0037] like Figure 2 As shown, the input in step S100 includes an input of a tree-structured 3D image set (S101) and an input of branching points (i.e., keypoints) of the tree structure as GT tags (S102). Here, the GT tags input in step S102 can be annotated using known methods on the 3D image set input in step S101, and can be prepared in advance. That is, steps S101 and S102 can be as follows... Figure 2 The steps shown are performed sequentially, but they can also be performed in parallel. In this case, the arrow between steps S101 and S102 can be omitted.

[0038] Step S200 performs image preprocessing and transformation on the input image set. Specifically, for example, isotropic transformation, intensity transformation, elastic transformation, etc., can be used to generate samples from the input image set (S201). Step S200 also transforms the input keypoints that serve as GT labels and generates a GT heatmap, for example, a Gaussian kernel heatmap can be used to generate a heatmap of the keypoints (S202).

[0039] The description of step S200 above is merely an example. The preprocessing and transformation of the image set and the generation of the GT heatmap can be implemented using various methods known in the art.

[0040] In step S300, the bifurcation point detection model is trained. One feature of this invention is that a bifurcation point topology atlas is introduced into the training process when training the bifurcation point detection model, so that the deep learning network can utilize the topological information of multiple key points in the tree structure during training, thereby improving the accuracy and precision of bifurcation point detection.

[0041] Specifically, this invention first selects a network with characteristics suitable for the aforementioned features when constructing the network architecture. For example, during network design, a network type suitable for capturing the aforementioned topological information at each scale is selected. Since stacked networks with intermediate supervision can provide mechanisms for re-evaluating intermediate heatmaps and re-evaluating higher-order spatial relationships, the deep learning network of this invention preferably employs a stacked network. For example, the deep learning network of this invention can be a stacked hourglass network or a coupled U-network.

[0042] Secondly, this invention includes a term in the loss function for model training that represents the difference between the tree-structured branch point topology and the output value. The loss function is used to estimate the degree of inconsistency (error value) between the model's output value P and the true value G. A typical loss function can be represented as L(P,G). One feature of this invention is that the loss function during training is expressed by the following formula:

[0043] Loss total =λ1L(P, G) + λ2L(P, A)

[0044] Among them, Loss total L(P,G) represents the total error of the model; L(P,A) represents the difference between the output value and the true value; L(P,A) represents the difference between the output value and the topological graph of the tree-structured branch points; λ1 and λ2 are weight coefficients.

[0045] L can employ common loss functions in the field, such as LogLoss, squared loss, exponential loss, Hinge loss, etc. The details of these loss functions are common knowledge and will not be elaborated here. The evaluation process of L(P,G) can employ various methods in the prior art, which will not be elaborated here. Specifically, the value of L(P,A) is a comparison of the similarity between the model's output predicted point set P and the bifurcation point topology map A in terms of position and topology. The difference in topology between the two is calculated using the function L(P,A). The comparison of the similarity between position and topology can employ various methods in the prior art, which will not be elaborated here. The topology map A contains the connections and edge length relationships between bifurcation points at different levels. This invention calculates the difference between the position / topology of specific points in the predicted point set P and the position / topology in the topology map A, as part of the loss function.

[0046] The weighting coefficients λ1 and λ2 are numbers between 0 and 1, and their sum is 1. The specific values ​​of λ1 and λ2 can be selected in advance based on experience, or they can be optimized by adjusting the weighting coefficients λ1 and λ2 to calculate the distance between the output detection result and the true value, just like optimizing other parameters during training. The goal is to select the weighting coefficient values ​​that yield the best results.

[0047] The above describes the training process of the tree-structured branch point detection model according to the first embodiment of the present invention. As described above, the present invention introduces a tree-structured branch point topology atlas into the training of the tree-structured keypoint detection model, allowing the deep learning network to utilize the topological information of multiple key points in the tree structure during training.

[0048] Figure 3 This is a schematic diagram illustrating the specific flow of the reasoning process of the tree-structured branch point detection model according to the first embodiment of the present invention.

[0049] like Figure 3 As shown, in step S100', an image set is input into the trained tree-structured branching point detection model. In step S200', key points are detected using the trained model. Here, in step S200', a heatmap is output for each detected branching point. The bright spots in the heatmap correspond to the branching points in the tree structure. Ideally, each heatmap should have only one bright spot. However, due to the accuracy and precision of the detection model, as well as false detections caused by noise, there may be more than one bright spot in the obtained heatmap. To solve this problem, one feature of the present invention is that the detection results are enhanced using a branching point topology map in the following step S300'.

[0050] The topology atlas of bifurcation points can be used to enhance detection results because, for tubular organs with a tree-like structure, such as blood vessels or trachea, although the bifurcation points may be located in different places, they generally have a relatively stable topology. For example, liver blood vessels or lung airways / blood vessels basically have a standard or normalizable topology consisting of first-level bifurcation points, second-level bifurcation point 1, second-level bifurcation point 2, and so on. Therefore, by applying the topology atlas of bifurcation points to the bifurcation point detection results output by the bifurcation point detection model and filtering out detection results that do not conform to the topology atlas, the detection results can be enhanced. In particular, higher-level bifurcation points can be detected more stably and accurately through the above enhancement.

[0051] Therefore, in step S300', the detected bifurcation points are filtered using a bifurcation point topology map. Figure 3 Taking the portal vein of the liver as an example, this paper illustrates the process of enhancing the detection results during the inference of the tree-structured detection model. The output of the detection model includes N heatmaps corresponding to bifurcation points 1 through N, each containing candidate extreme points (bright spots) for that bifurcation point. The number of these extreme point candidates may be more than one. As shown in the figure, this invention filters extreme points in heatmaps containing multiple extreme point candidates based on the hierarchical and spatial relationships between bifurcation points in the liver's portal vein topology, such as the first-level bifurcation point, the second-level right bifurcation point, the second-level right bifurcation point, the second-level left bifurcation point, the second-level left bifurcation point, and so on. Extreme points that do not conform to the bifurcation point topology are removed, and only one extreme point is retained for each heatmap as the enhanced detection result. The specific method for the above removal can employ methods commonly used in the field, and will not be elaborated here.

[0052] The above describes the reasoning process of the tree-structure branching point detection model according to the first embodiment of the present invention. As described above, the present invention introduces a tree-structure branching point topology map into the reasoning of the tree-structure key point detection model. During the reasoning process, the branching point topology map is combined with the detected tree-structure branching points to enhance the detection results.

[0053] Therefore, in the tree structure branching point detection method of the present invention, which includes the above-mentioned training and inference processes, the accuracy and precision of branching point detection are improved by using the corresponding information between the various branching points of the tree structure for branching point detection.

[0054] This invention improves both the training and inference processes in the tree-structure branching point detection method, achieving the invention's objective through the improved training and inference processes. However, it is readily understood that improvements made only to either the training or inference process also fall within the scope of this invention and can achieve the same objective.

[0055] Furthermore, the present invention can also include various variations. For example, the topology of the branching points in a tree structure may be standard or may vary due to individual factors such as gender for different organs. Therefore, the topology atlas of branching points in the present invention can use a pre-prepared standard topology atlas, or it can be a non-standard topology atlas obtained, for example, through a training process to account for these individual factors.

[0056] The following describes the tree-structure branching point detection system of the first embodiment. Figure 7 This is a functional block diagram illustrating the tree-structure branching point detection system according to the first embodiment of the present invention. For example... Figure 7 As shown, the tree branch point detection system 1 of the first embodiment is a tree branch point detection system that detects branch points in a tree structure using a deep learning network. It includes: a training device 10, which trains a model for detecting branch points in a tree structure using a deep learning network by utilizing a branch point topology atlas of the tree structure; and an inference device 20, which enhances the branch points detected by the model trained by the training device using the branch point topology atlas.

[0057] The above describes the tree-structure branching point detection method and system according to the first embodiment. According to the present invention, a tree-structure keypoint detection model trained by a deep learning network is used to detect branching points in a tree structure. During training, the deep learning network can utilize the topological information of multiple keypoints in the tree structure, and during inference, it can enhance the detection results using a branching point topology atlas. Therefore, the present invention uses the correspondence information between various branching points in the tree structure for branching point detection, improving the accuracy and precision of branching point detection.

[0058] <Second Implementation Method>

[0059] According to the second embodiment of the present invention, the tree-structure branching point detection method of the first embodiment is applied to the registration of medical images, especially the registration of multimodal medical images.

[0060] The second embodiment of the present invention will be described in detail below with reference to the accompanying drawings.

[0061] Figure 4 This is a flowchart illustrating an image registration method according to a second embodiment of the present invention. For example... Figure 4As shown, the image registration method of the present invention includes the following steps.

[0062] In step S4100, images of multiple modalities of the organ are acquired. Taking the registration between a 3D liver image as a preoperative image and a 2D liver image as an intraoperative image as an example, this embodiment will be described. In this case, step S4100 may include a step of acquiring a set of 3D preoperative MR / CT images (step S4101) and a step of acquiring a set of 2D intraoperative US (ultrasound) images of the same organ (S4102).

[0063] In step S4200, the bifurcation points of liver blood vessels, which form a tree structure, are detected in the obtained organ image; in step S4300, the detection results of liver blood vessel bifurcation points are enhanced using a 3D topological atlas of liver blood vessels.

[0064] Here, steps S4200 and S4300 can be specifically implemented through the various steps of the tree-structure branching point detection method described in the first embodiment. In other words, Figure 4 The detection and enhancement of the tree-like branching points formed by steps S4200 and S4300, enclosed by the dashed line, can be performed using the tree-like branching point detection method of the present invention described in the first embodiment. For example, step S4200 corresponds to step S200' in the first embodiment, and step S4300 corresponds to step S300' in the first embodiment.

[0065] In the case of registering a 3D image as a preoperative image and a 2D image as an intraoperative image according to the present invention, step S4200 can be divided into two separate steps, S4201 and S4202, for the 3D image and the 2D image, respectively. The branching point detection steps S4201 and S4202 for the 3D image can be performed using the tree-structured branching point detection method of the present invention described in the first embodiment. On the other hand, since branching point detection for the 2D image is relatively simple, to speed up processing and reduce processing load, the branching point detection step S4202 for the 2D image can be performed manually, for example, without using the tree-structured branching point detection method of the present invention. In this case, step S4300 only needs to be performed on the portion of the 3D image using the tree-structured branching point detection method of the present invention (the output of step S4201).

[0066] In step S4400, the preoperative liver image and the intraoperative liver image are registered using, for example, liver vessel bifurcation points detected in steps S4200 and S4300. Image registration using vessel bifurcation points can be performed using any known existing technique, and therefore will not be elaborated further.

[0067] Here, in order to improve the registration accuracy and reduce the registration processing load, after detecting the branching points of the tree structure, before registration in step S4400, the present invention can also correct the detection results of the branching points of the tree structure by the organ segmentation results. Figure 5 This is an explanatory diagram illustrating the process of correcting the detection results before registration in step S4400 of the image registration method according to the second embodiment of the present invention. Figure 5 As shown, a segmented image of liver tissue is obtained by segmenting, for example, the liver. This segmented image is then used as a template for detecting bifurcation points of liver vessels. For instance, bright spots in the heatmap located within the liver tissue region of the segmented image are retained as bifurcation point detection results, while bright spots outside the liver tissue region are excluded. Therefore, bifurcation points are corrected before registration, thereby improving registration accuracy and reducing unnecessary processing load. Figure 5 In this context, PV and HV represent the portal vein and hepatic vein of the liver, respectively.

[0068] return Figure 4 To elaborate further, after registering the multimodal images through the bifurcation point in step S4400, further fine registration of the multimodal images can be performed in step S4500 using other structures. The fine registration process can obviously be omitted, therefore step S4500 is not mandatory.

[0069] Furthermore, according to the image registration method of the present invention, in the registration step S4400, in addition to using the branching points of the tree structure, the tree structure itself can also be used. For example, after detecting the branching points of the tree structure, the center line of the tree structure of the organ is generated using the detected branching points, and the tree structure of the organ is segmented according to the generated center line to generate a segmented image of the tree structure. Then, the registration step S4400 registers images of multiple modalities based on the generated tree structure image.

[0070] The registration methods described above, such as registration using branch points, registration using tree structures, and fine registration, can be implemented using techniques known in the art, and therefore will not be elaborated further.

[0071] The image registration system of the second embodiment is an image registration system that registers images of multiple modalities of an organ using the branching points of an organ with a tree-like structure. It includes: an acquisition device for acquiring images of multiple modalities of an organ; a tree-like branching point detection device for detecting branching points of a tree-like structure in the acquired images of the organ using the tree-like branching point detection method described in the first embodiment; and a registration device for registering the images of multiple modalities using the detected branching points.

[0072] The image registration method and system of the second embodiment have been described above. The above description only shows the registration between a 3D image as a preoperative image and a 2D image as an intraoperative image; however, the image registration method of the present invention can obviously be applied to images of other modalities, and can be applied to the registration between images of more modalities. Furthermore, the second embodiment has been described above using liver blood vessels as an example; however, the image registration method of the present invention can obviously be applied to the registration of other organs with a tree-like structure.

[0073] In addition to achieving the effects of the first embodiment, the second embodiment can also improve the registration accuracy of multimodal images, such as preoperative and intraoperative images.

[0074] <Third Implementation Method>

[0075] According to the third embodiment of the present invention, the tree-structure branch point detection method of the first embodiment is applied to image segmentation.

[0076] The third embodiment of the present invention will be described in detail below with reference to the accompanying drawings.

[0077] Figure 6 This is a flowchart illustrating an image segmentation method according to a third embodiment of the present invention.

[0078] like Figure 6 As shown, the image segmentation method of the present invention includes the following steps: Step S6100, acquiring a medical image; Step S6200, detecting the branching points of a tree structure in the acquired medical image, wherein Step S6200 can be implemented using the various steps of the tree structure branching point detection method of the present invention described in the first embodiment; Step S6300, generating the center line of the tree structure using the detected branching points; Step S6400, segmenting the tree structure according to the generated center line of the tree structure.

[0079] Taking a tree-like structure as a blood vessel as an example, step S6300 can be implemented by methods known in the art for extracting the center line using key points, such as the vessel tracking method. Step S6400 can be implemented by traditional blood vessel segmentation methods such as threshold-based segmentation, image intensity or geometric features, or deep learning-based segmentation methods. Therefore, the details will not be elaborated here.

[0080] The image segmentation system of the third embodiment is an image segmentation system for segmenting organs with a tree-like structure in medical images, comprising: an acquisition device for acquiring medical images; a tree-like structure branching point detection device for detecting branching points of a tree-like structure in the acquired medical images using the tree-like structure branching point detection method described in the first embodiment; a centerline generation device for generating a centerline of the tree-like structure using the detected branching points; and a segmentation device for segmenting the tree-like structure based on the generated centerline of the tree-like structure.

[0081] The image segmentation method and system according to the third embodiment have been described above. In addition to achieving the effects of the first embodiment, the third embodiment also improves the accuracy of tree-structure segmentation.

[0082] <Other variations>

[0083] The present invention is not limited to the embodiments described above, and can be modified in many ways.

[0084] For example, the above embodiments have been described using blood vessels as an example, but the present invention can also be used for other tubular organs such as the trachea.

[0085] The system of the present invention can also be installed in medical devices as a circuit capable of realizing the functions described in the various embodiments, or it can be distributed as a program that can be executed by a computer, stored on storage media such as disks (floppy disks, hard disks, etc.), optical disks (CD-ROMs, DVDs, BDs, etc.), optical disks (MOs), semiconductor memories, etc.

[0086] Furthermore, middleware such as an OS (operating system), database management software, and network software that runs on a computer based on instructions from a program installed on the computer from a storage medium can also execute a portion of the processes used to implement the above-described embodiments.

[0087] The foregoing has described several embodiments of the present invention, but these embodiments are provided as examples and are not intended to limit the scope of the invention. These new embodiments can be implemented in a wide variety of other ways, and various omissions, substitutions, and modifications can be made without departing from the spirit of the invention. These embodiments or variations thereof are included within the scope or spirit of the invention, and also within the scope of the invention and its equivalents as described in the claims.

Claims

1. A method for detecting branching points in a tree structure, which uses a deep learning network to detect branching points in a tree structure in an image, characterized in that, Includes the following steps: The training step involves using the branch point topology atlas of the tree structure to train a model for detecting the branch points of the tree structure through the deep learning network. as well as In the inference step, the bifurcation point topology atlas is used to enhance the bifurcation points detected by the model trained in the training step. In the training step, the loss function used for model training includes a term representing the difference between the bifurcation point topology map and the output value. In the reasoning step, the detected bifurcation points are filtered using the bifurcation point topology graph to perform the reinforcement. The loss function is expressed by the following formula: in, Loss total This represents the total error value of the model; L(P, G) represents the difference between the output value and the true value; L(P, A) represents the difference between the output value and the bifurcation point topology atlas; λ1 and λ2 are weighting coefficients.

2. The method for detecting branching points of a tree structure as described in claim 1, characterized in that, The deep learning network is an hourglass network or a coupled U-network.

3. An image registration method, which registers images of multiple modalities of an organ using the branching points of the tree-like structure of the organ, characterized in that, Includes the following steps: The acquisition step involves acquiring images of the multiple modalities of the organ; The tree-like structure branching point detection step involves detecting the branching points of the tree-like structure in the obtained image of the organ using the tree-like structure branching point detection method as described in claim 1 or 2. as well as The registration step involves registering the images of the multiple modalities using the detected bifurcation points.

4. The image registration method as described in claim 3, characterized in that, It also includes a tree structure image generation step, which uses the branching points of the tree structure detected in the tree structure branching point detection step to generate the tree structure centerline topology of the organ, and performs tree structure segmentation of the organ based on the tree structure centerline topology to generate a tree structure image. The registration step further registers the images of the multiple modalities based on the tree structure image.

5. The image registration method as described in claim 3, characterized in that, The multiple modal images include preoperative images and intraoperative images of the organ. In the tree-like structure branching point detection step, the tree-like structure branching points of the organ are detected in the preoperative image using the tree-like structure branching point detection method, and the tree-like structure branching points of the organ are detected in the intraoperative image using the same method, or the tree-like structure branching points of the organ are manually set in the intraoperative image. In the tree-like structure branching point detection step, the detection results of the tree-like structure branching points in the preoperative image are also corrected based on the segmentation results of the organ.

6. The image registration method as described in claim 5, characterized in that, The preoperative images are MR or CT images, and the intraoperative images are ultrasound images.

7. The image registration method according to any one of claims 3 to 6, characterized in that, The deep learning network is an hourglass network or a coupled U-network.

8. An image segmentation method for segmenting organs with a tree-like structure in medical images, characterized in that, Includes the following steps: The acquisition step involves acquiring the medical image; The tree-like structure branching point detection step involves detecting the branching points of the tree-like structure in the acquired medical image using the tree-like structure branching point detection method as described in claim 1 or 2. The centerline generation step involves generating the centerline of the tree structure using the detected branching points. as well as The segmentation step involves segmenting the tree structure according to the center line of the generated tree structure.

9. A tree-structure branching point detection system, which detects branching points in a tree structure in an image using a deep learning network, characterized in that, have: The training device uses the tree-like branch point topology map to train a model for detecting branch points of the tree-like structure through the deep learning network, wherein the loss function used for training the model includes a term representing the difference between the branch point topology map and the output value; as well as The inference device uses the bifurcation point topology atlas to enhance the bifurcation points detected by the model trained by the training device. The inference device uses the bifurcation point topology map to filter the detected bifurcation points, thereby performing the enhancement. The loss function is expressed by the following formula: in, Loss total This represents the total error value of the model; L(P, G) represents the difference between the output value and the true value; L(P, A) represents the difference between the output value and the bifurcation point topology atlas; λ1 and λ2 are weighting coefficients.

10. An image registration system that registers images of multiple modalities of an organ using the branching points of the tree-like structure of the organ, characterized in that, have: Acquisition device, acquiring images of the organ in the plurality of modalities; A tree-like structure branching point detection device, using the tree-like structure branching point detection method as described in claim 1 or 2, detects the branching points of the tree-like structure in the obtained image of the organ; as well as The registration device registers the images of the multiple modalities using the detected bifurcation points.

11. An image segmentation system for segmenting organs with a tree-like structure in medical images, characterized in that, have: Acquisition device, acquires the medical image; A tree-like structure branching point detection device, using the tree-like structure branching point detection method as described in claim 1 or 2, detects the branching points of the tree-like structure in the acquired medical image; A centerline generation device generates the centerline of the tree structure using the detected bifurcation points; as well as The segmentation device segments the tree structure according to the center line of the generated tree structure.

Images