Precise navigation method and system for endoscope debridement of pancreatic wrapped necrosis

Abstract

The invention provides a pancreatic wrapped necrosis endoscope debridement precise navigation method and system, and relates to the technical field of endoscope navigation. Comprising the following steps: acquiring a plurality of historical abdominal CT images before endoscopic drainage debridement, and identifying contour areas of a pancreatic head, a pancreatic neck, a gland, a pancreatic tail, a duodenum, a gastric antrum and a gastric body in the abdominal CT images; an endoscope image and an ultrasonic image in the endoscope drainage debridement process are obtained, a gastroscope part is recognized, and a pancreas area is recognized; constructing a feature vector, inputting the feature vector into a pre-trained CNN network model, and obtaining an abdominal CT image with the highest matching degree with a pancreas region recognition result in the current ultrasonic image from a plurality of historical abdominal CT images; and carrying out abdominal CT multi-plane reconstruction and display, so as to realize positioning and navigation of pancreatic wrapped necrosis endoscope debridement. According to the invention, through feature fusion processing of the endoscopic image, the ultrasonic image and the CT image, accuracy and safety required by an operation are ensured.

Description

Technical Field

[0001] The present invention belongs to the technical field of endoscopic navigation, and in particular relates to a precise navigation method and system for endoscopic debridement of pancreatic encapsulated necrosis. Background Art

[0002] Severe acute pancreatitis has a high mortality rate. Most deaths occur after 4 weeks, and WON (wall off pancreatic necrosis, WON) combined with infection is the most common cause of death at this stage. At present, the "upward ladder" therapy represented by minimally invasive technology such as digestive endoscopic drainage and debridement has become the first choice for the treatment of WON. Compared with traditional surgical treatment of trauma, the treatment cycle is short. The endoscopic debridement method is to make an incision or place a stent on the posterior wall of the stomach through an endoscope, and then clean the pancreas and surrounding necrotic tissue through an artificial sinus tract on the posterior wall of the stomach. It is currently a hot topic and development prospect in digestive endoscopy research.

[0003] Endoscopic WON debridement and drainage is a complex and high-risk endoscopic operation. One of the difficulties is that the endoscopic imaging range is limited, and only a local two-dimensional image is displayed. When the endoscope enters the pancreatic necrotic cavity with complex morphology and structure, there are problems such as narrow field of view, lack of sense of space, direction, depth, and easy obstruction. Doctors cannot accurately calculate the movement path of the endoscope to the designated inspection position in advance, and cannot make accurate judgments on the current position and posture of the endoscope in the anatomical structure and the movement trend. At the same time, there is a lack of intuitive understanding of the tissue structure around the endoscope position, and it is impossible to "pass through" the organ wall to observe the important tissue structure that may be hidden behind. Therefore, doctors are basically in a "blind" state, which may cause doctors to make mistakes in decision-making and increase the risk of surgery. Summary of the invention

[0004] In order to overcome the deficiencies of the above-mentioned prior art, the present invention provides a precise navigation method and system for endoscopic debridement of pancreatic encapsulated necrosis, which fully utilizes the real-time nature of endoscopic images and the globality of CT image three-dimensional volume data, constructs a custom image matching model through endoscopic image, ultrasound image and CT image feature fusion processing, and realizes the display of CT multi-plane image visualization information to doctors, realizes digestive endoscopic NOTES surgery under real-time navigation state, and ensures the accuracy and safety required for surgery.

[0005] To achieve the above objectives, one or more embodiments of the present invention provide the following technical solutions:

[0006] A first aspect of the present invention provides a precise navigation method for endoscopic debridement of encapsulated pancreatic necrosis.

[0007] The precise navigation method for endoscopic debridement of pancreatic encapsulated necrosis includes the following steps:

[0008] Obtain multiple historical abdominal CT images of the patient before endoscopic drainage and debridement, and identify the contour areas of the pancreatic head, pancreatic neck, gland, pancreatic tail, duodenum, gastric antrum, and gastric body in the abdominal CT images;

[0009] Obtain endoscopic images and ultrasound images during endoscopic drainage and debridement, identify the gastroscopic part in the endoscopic image, and identify the pancreatic area in the ultrasound image;

[0010] Based on the recognition results of the gastroscope part in the endoscopic image, the feature vector is constructed by combining the recognition results of the pancreatic region in the ultrasound image and the recognition results of the abdominal CT image;

[0011] The feature vector is input into the pre-trained CNN network model to obtain the abdominal CT image with the highest matching degree with the pancreatic region recognition result in the current ultrasound image from multiple historical abdominal CT images;

[0012] Based on the pancreatic region recognition results in the current ultrasound image and the abdominal CT image with the highest matching degree, abdominal CT multi-planar reconstruction and display are performed to achieve positioning navigation for endoscopic debridement of pancreatic encapsulated necrosis.

[0013] A second aspect of the present invention provides a precise navigation system for endoscopic debridement of pancreatic encapsulated necrosis.

[0014] Precision navigation system for endoscopic debridement of pancreatic encapsulated necrosis, including:

[0015] The CT image recognition module is configured to: obtain multiple historical abdominal CT images of the patient before endoscopic drainage and debridement, and recognize the contour areas of the pancreatic head, pancreatic neck, gland, pancreatic tail, duodenum, gastric antrum and gastric body in the abdominal CT images;

[0016] The ultrasound image recognition module is configured to: obtain an endoscopic image and an ultrasound image during the endoscopic drainage and debridement process, recognize the gastroscope part in the endoscopic image, and recognize the pancreatic area in the ultrasound image;

[0017] The feature vector construction module is configured to: construct a feature vector based on the recognition result of the gastroscope part in the endoscopic image, combined with the pancreatic region recognition result in the ultrasound image and the abdominal CT image recognition result;

[0018] The matching module is configured to: input the feature vector into a pre-trained CNN network model, and obtain an abdominal CT image with the highest matching degree with the pancreatic region recognition result in the current ultrasound image from multiple historical abdominal CT images;

[0019] The multi-planar reconstruction module is configured to: perform multi-planar reconstruction and display of abdominal CT based on the pancreatic region recognition result in the current ultrasound image and the abdominal CT image with the highest matching degree, so as to realize the positioning navigation of endoscopic debridement of pancreatic encapsulated necrosis.

[0020] One or more of the above technical solutions have the following beneficial effects:

[0021] The present invention provides a precise navigation method and system for endoscopic debridement of pancreatic encapsulated necrosis, which fully utilizes the real-time nature of endoscopic images and the global nature of CT image three-dimensional volume data, constructs a custom image matching model through endoscopic image, ultrasound image and CT image feature fusion processing, and presents CT multi-plane image visualization information to doctors, thereby realizing digestive endoscopic NOTES surgery under real-time navigation state and ensuring the accuracy and safety required for surgery.

[0022] The present invention identifies the pancreatic region in the ultrasound image of the endoscopic drainage and debridement process, and simultaneously identifies the contour regions of the pancreatic head, pancreatic neck, gland, pancreatic tail, duodenum, gastric antrum and gastric body in the abdominal CT image. Based on the recognition result of the gastroscopic part in the endoscopic image, combined with the pancreatic region recognition result in the ultrasound image and the abdominal CT image recognition result, a feature vector is constructed, a CNN network model is trained based on the feature vector, and an abdominal CT image with the highest matching degree with the pancreatic region recognition result in the current ultrasound image is obtained from multiple historical abdominal CT images, so as to facilitate subsequent multi-plane reconstruction and display in combination with the abdominal CT image with the highest matching degree.

[0023] The feature vector constructed by the present invention includes 20 feature values, which take into account the area size and center point X, Y coordinate values ​​of the pancreatic head, pancreatic neck, gland and / or pancreatic tail identified in the ultrasound image, and the area size and center point X, Y coordinate values ​​of the pancreatic head, pancreatic neck, gland and / or pancreatic tail identified in the abdominal CT image, and also take into account the stomach part identified in the ultrasound image and the stomach part identified in the abdominal CT image; through the designed feature vector, an accurate result of whether the ultrasound image and the CT image match can be obtained.

[0024] Advantages of additional aspects of the present invention will be given in part in the following description, and in part will become obvious from the following description, or will be learned through practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] The accompanying drawings in the specification, which constitute a part of the present invention, are used to provide a further understanding of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute improper limitations on the present invention.

[0026] Figure 1 This is a flow chart of the method of the first embodiment.

[0027] Figure 2 This is a cross-sectional CT image through the duodenal jejunal flexure.

[0028] Figure 3 These are the transverse, sagittal, and coronal images after MPR reconstruction.

[0029] Figure 4 It is a system structure diagram of the second embodiment.

[0030] Among them: 1. Left lobe of liver; 2. Hepatic flexure of colon; 3. Gastric antrum; 4. Gastric body; 5. Right lobe of liver; 6. Descending part of duodenum; 7. Head of pancreas; 8. Portal vein; 9. Superior mesenteric artery; 10. Body of pancreas; 11. Inferior vena cava; 12. Abdominal aorta; 13. Left adrenal gland; 14. Splenic vein; 15. Right kidney; 16. Left kidney; 17. Spleen. DETAILED DESCRIPTION

[0031] It should be noted that the following detailed descriptions are exemplary and are intended to provide further explanation of the present invention. Unless otherwise specified, all technical and scientific terms used herein have the same meanings as those commonly understood by those skilled in the art to which the present invention belongs.

[0032] It should be noted that the terms used herein are for describing specific embodiments only and are not intended to be limiting of exemplary embodiments according to the present invention.

[0033] In the absence of conflict, the embodiments of the present invention and the features of the embodiments may be combined with each other.

[0034] Embodiment 1

[0035] This embodiment discloses a precise navigation method for endoscopic debridement of encapsulated pancreatic necrosis.

[0036] like Figure 1 As shown in the figure, the precise navigation method for endoscopic debridement of pancreatic encapsulated necrosis includes the following steps:

[0037] Obtain multiple historical abdominal CT images of the patient before endoscopic drainage and debridement, and identify the contour areas of the pancreatic head, pancreatic neck, gland, pancreatic tail, duodenum, gastric antrum, and gastric body in the abdominal CT images;

[0038] Obtain endoscopic images and ultrasound images during endoscopic drainage and debridement, identify the gastroscopic part in the endoscopic image, and identify the pancreatic area in the ultrasound image;

[0039] Based on the recognition results of the gastroscope part in the endoscopic image, the feature vector is constructed by combining the recognition results of the pancreatic region in the ultrasound image and the recognition results of the abdominal CT image;

[0040] The feature vector is input into the pre-trained CNN network model to obtain the abdominal CT image with the highest matching degree with the pancreatic region recognition result in the current ultrasound image from multiple historical abdominal CT images;

[0041] Based on the pancreatic region recognition results in the current ultrasound image and the abdominal CT image with the highest matching degree, abdominal CT multi-planar reconstruction and display are performed to achieve positioning navigation for endoscopic debridement of pancreatic encapsulated necrosis.

[0042] The specific operations are as follows:

[0043] The CT images and ultrasound images collected and recognized and processed as described below are scaled to 512X512 resolution images and will not be summarized in detail.

[0044] 1. Endoscopic and Ultrasound Image Category Recognition Model

[0045] EUS ultrasound endoscope is an advanced medical device that integrates ultrasound and endoscopy. It is an upgraded version of gastroscopy. It can not only directly observe the digestive tract, but also perform real-time ultrasound scanning of the digestive tract and surrounding organs, and obtain endoscopic and ultrasound images at the same time. Therefore, there are endoscopic images and ultrasound images in the screen during the examination. By collecting endoscopic and ultrasound images, they are divided into two categories and trained a binary classification neural network model.

[0046] Since the screen of EUS ultrasound endoscope can be switched, the screen before switching to ultrasound is the endoscopic image, which can identify the gastroscopic part in real time. The endoscopic image is maintained when the endoscope goes deep into the body, and the screen is switched to the ultrasound probe screen when the pancreatic area is to be detected.

[0047] 2. Endoscopic Image Part Recognition Model

[0048] Collect historical endoscopic images, annotate gastric endoscopic images, divide them into the following categories: cardia, gastric body, pylorus, gastric fundus, gastric angle, gastric antrum, and duodenum, and train a gastroscopic site classification network.

[0049] 3. CT thin-layer image segmentation model

[0050] Abdominal CT images were collected, and mask labels of the pancreatic head contour area, pancreatic neck contour area, gland contour area, pancreatic tail contour area, duodenum contour area, gastric antrum contour area, and gastric body contour area were marked in the slice images to train the segmentation network model.

[0051] like Figure 2The figure shows a transverse CT image of the duodenal jejunal flexure. 1 is the left lobe of the liver; 2 is the hepatic flexure of the colon; 3 is the gastric antrum; 4 is the gastric body; 5 is the right lobe of the liver; 6 is the descending part of the duodenum; 7 is the head of the pancreas; 8 is the portal vein; 9 is the superior mesenteric artery; 10 is the body of the pancreas; 11 is the inferior vena cava; 12 is the abdominal aorta; 13 is the left adrenal gland; 14 is the splenic vein; 15 is the right kidney; 16 is the left kidney; 17 is the spleen.

[0052] 4. EUS ultrasound image segmentation model

[0053] Ultrasound images of historical endoscopic debridement of pancreatic encapsulated necrosis were collected, and mask labels of the pancreatic head contour area, pancreatic neck contour area, gland contour area, pancreatic tail contour area, duodenum contour area, gastric antrum contour area, and gastric body contour area were marked in the ultrasound images to train the segmentation network model.

[0054] 5. Construct CT image and ultrasound image matching module

[0055] Collect historical examination data of patients, including abdominal CT examinations and endoscopic and ultrasound images during endoscopic debridement of encapsulated pancreatic necrosis, and develop a correspondence between CT and ultrasound images: that is, given an ultrasound image, find the slice image that best meets the conditions from the CT slice sequence image. For example, if the ultrasound image is of the pancreatic head area viewed in the duodenum, the CT slice image should include the duodenal area and the pancreatic head area. Construct a custom feature vector to create training samples to ensure that each EUS ultrasound image has its corresponding matching CT image.

[0056] The construction of feature vectors utilizes the stomach part recognition of white light endoscopy, ultrasound images, and CT images. When the endoscope lens goes deep into the stomach, the white light endoscopy image is maintained. When the pancreas area is to be detected, the ultrasound probe image is switched. The numerical representation of the feature value of the stomach part recognized in the ultrasound image refers to the stomach part recognized by the white light endoscopy image before switching to the ultrasound image.

[0057] Construct feature vectors. The specific process is:

[0058] Different endoscopic gastric sites were represented by different characteristic values, including the cardia, gastric body, pylorus, gastric fundus, gastric angle, gastric antrum, and duodenum;

[0059] Different pancreatic sub-regions are represented by different eigenvalues, including the pancreatic head, pancreatic neck, gland, and pancreatic tail;

[0060] obtaining a numerical representation of a characteristic value of a stomach region identified in an ultrasound image;

[0061] Calculate the area size and X, Y coordinate values ​​of the center point of the pancreatic head, pancreatic neck, gland and / or pancreatic tail identified in the ultrasound image respectively;

[0062] Obtaining numerical representation of characteristic values ​​of the stomach part identified by the abdominal CT image, and calculating the area size of the identified stomach part and the coordinates of the center point of the stomach area;

[0063] Calculate the area size and center point X and Y coordinate values ​​of the pancreatic head, pancreatic neck, gland and / or pancreatic tail identified in each abdominal CT image;

[0064] The numerical representation of the characteristic value of the stomach part identified in the ultrasound image, the area size and the X, Y coordinate values ​​of the center point of the pancreatic head, pancreatic neck, gland and / or pancreatic tail identified in the ultrasound image, the numerical representation of the characteristic value of the stomach part identified in the abdominal CT image, the area size of the stomach part and the coordinates of the center point of the stomach area identified in the abdominal CT image, and the area size and the X, Y coordinate values ​​of the center point of the pancreatic head, pancreatic neck, gland and / or pancreatic tail identified in the abdominal CT image, a total of 20 characteristic values, are collectively used as characteristic vectors.

[0065] Further:

[0066] Endoscopic sites are indicated by numerical values, including the cardia, gastric body, pylorus, gastric fundus, gastric angle, gastric antrum, and duodenum, as shown in the following table:

[0067]

[0068]

[0069] The numerical values ​​of the pancreatic sub-regions are shown in the following table:

[0070] Pancreatic subregion names Eigenvalue numerical representation Head of Pancreas 0 Pancreatic neck 1 Glands 2 Tail of pancreas 3

[0071] The custom feature vector is as follows:

[0072]

[0073]

[0074] The recognition results corresponding to the feature vectors are shown in the following table:

[0075] Category Label Category Value 0 Ultrasound and CT images do not match 1 Ultrasound image and CT image matching

[0076] The area sizes of the pancreatic head, pancreatic neck, glandular area, and pancreatic tail in the table are normalized data, that is, the area calculated by pixels divided by the total area of ​​the image. If the area does not exist in the image, the area size is 0. The coordinates of the pancreatic head contour area, pancreatic neck contour area, glandular contour area, and pancreatic tail contour area are normalized data, that is, the coordinate value X and the coordinate value Y are divided by the width and height of the image respectively. Build a custom CNN network, input the feature vector composed of the above 20 eigenvalues, and the corresponding label is the category to which it belongs. The network structure is divided into: the input layer is a 20X1 floating point vector, including multiple fully connected layers, and the output layer of the fully connected network uses the Sigmoid activation function for binary classification. The output layer returns 2 classifications and probabilities, and uses the optimizer of the gradient descent algorithm.

[0077] 6. Build abdominal CT MPR (multi-planar reconstruction) display module

[0078] like Figure 3 As shown, in the MPR interface of the CT film reading PACS system, the unique identifier of the CT image section and the pancreatic contour area of ​​the current image are input, and the MPR interactive cross-line is positioned at the coordinate position of the center point of the pancreatic contour area as the cross-line coordinate point of the MPR, and the three cross-section images of the transverse, sagittal, and coronal positions after MPR reconstruction are displayed. At the same time, the CT thin-layer image segmentation model of step 3 is called to identify the gastric area: cardia, gastric body, pylorus, gastric fundus, gastric angle, gastric antrum, and duodenum.

[0079] 7. Real-time auxiliary display of CT images for gastroscopy

[0080] First, obtain the patient's abdominal CT image before endoscopic drainage and debridement treatment, and use the slice images to call the CT thin-layer image segmentation model in step 3 in sequence to identify the contour areas of the pancreatic head, pancreatic neck, gland, pancreatic tail, duodenum, gastric antrum and gastric body, and record the contour area set of each slice image.

[0081] During the endoscopic drainage and debridement process, the inspection screen is collected in real time, and the endoscopic and ultrasound image category recognition model in step 1 is called on the image to identify whether the current image is an endoscopic image or an ultrasound image;

[0082] The endoscopic image calls the endoscopic image part recognition model in step 2 to recognize the gastroscopic part and record the current gastroscopic part;

[0083] When the ultrasound image is identified, the EUS ultrasound image segmentation model of step 3 is called synchronously; when the pancreatic area is identified, such as the contour area of ​​the pancreatic head, pancreatic neck, gland, and pancreatic tail, the CT image and ultrasound image matching module of step 5 is called to match the current ultrasound image with the slice images in the patient's abdominal CT examination in sequence, and input the custom feature vector into the matching model, specifically:

[0084] [Numerical representation of the name of the stomach part corresponding to the ultrasound image,

[0085] The size of the pancreatic head area in ultrasound images,

[0086] The size of the pancreatic neck area in ultrasound images,

[0087] The size of the glandular area in the ultrasound image,

[0088] The size of the pancreatic tail region in ultrasound images,

[0089] The X and Y coordinates of the center point of the pancreatic head area in the ultrasound image,

[0090] The X and Y coordinates of the center point of the pancreatic neck area in the ultrasound image,

[0091] The X and Y coordinates of the center point of the pancreatic body area in the ultrasound image,

[0092] The X and Y coordinates of the center point of the pancreatic tail area in the ultrasound image,

[0093] Numerical representation of the stomach name corresponding to the CT image,

[0094] The size of the stomach area corresponding to the CT image,

[0095] The X and Y coordinate values ​​of the center point of the stomach area in the CT image,

[0096] The size of the pancreatic head in CT images,

[0097] The size of the pancreatic neck region in CT images,

[0098] The size of the pancreatic body in CT images,

[0099] The size of the pancreatic tail region in CT images,

[0100] The X and Y coordinates of the center point of the pancreatic head area in the CT image,

[0101] The X and Y coordinates of the center point of the pancreatic neck area in the CT image,

[0102] The X and Y coordinates of the center point of the pancreatic body in the CT image.

[0103] X, Y coordinate values ​​of the center point of the pancreatic tail region in the CT image];

[0104] Identify which CT slice image the current ultrasound image matches, and extract the unique identification information of the successfully matched CT slice image (SopInstUID identification information stored in the DICOM file);

[0105] The mark is input to the abdominal CT MPR (multi-planar reconstruction) display module in step 5, and the crosshair coordinates of the MPR are located at the center coordinates of the pancreatic body area. (The pancreatic body area here refers to the overall pancreatic area, such as identifying the pancreatic head contour area and the pancreatic neck contour area, and merging the two areas into one pancreatic overall area).

[0106] The above process enables real-time positioning and navigation through CT images during endoscopic examination, and displays the visualization information of CT multi-plane images to doctors, providing positioning of anatomical directions under the endoscopic field of view.

[0107] The following are some examples:

[0108] First, call the CT image segmentation model in step 3 for the CT image, and identify the contour areas of the pancreatic head, pancreatic neck, gland, pancreatic tail, duodenum, gastric antrum, and gastric body for each CT transverse image. For example, if there are 400 CT images, call the segmentation model to obtain the identified parts and areas on each image. If the pancreatic head is identified, calculate the circumscribed rectangle of the pancreatic head contour area, and obtain the X, Y coordinate values ​​of the center point of the pancreatic head area and the area size of the pancreatic head area; if the pancreatic head cannot be identified, it means that this transverse image does not have the pancreatic head, and set the X, Y coordinate values ​​of the center point of the pancreatic head area and the area size of the pancreatic head area corresponding to this image to 0.

[0109] The ultrasound image segmentation model of step 4 is called for the ultrasound image of the current EUS screen, and the same processing logic as the CT image is adopted. If the pancreatic head region exists, the X, Y coordinate values ​​of the center point of the pancreatic head region and the area of ​​the pancreatic head region are calculated. If it does not exist, the X, Y coordinate values ​​of the center point of the pancreatic head region and the area of ​​the pancreatic head region corresponding to this image are both 0.

[0110] The current ultrasound image and the 400 CT images are combined into 400 custom feature vectors in turn, and the CNN classification network of the step 5 module is called in turn to obtain the CT image that best matches the 400 CT images, that is, the unique identifier of the CT image section and the pancreatic contour area of ​​the current image are obtained.

[0111] The inspection screen, i.e., the ultrasound image, is acquired in real time, and the CT image that best matches the ultrasound image is found from the above 400 CT images.

[0112] MPR reconstruction of CT belongs to the existing technology and will not be elaborated here. The cross-line of MPR interaction is positioned at the coordinate position of the center point of the pancreatic contour area, which serves as the cross-line coordinate point of MPR, and displays the transverse, sagittal and coronal images after MPR reconstruction.

[0113] Embodiment 2

[0114] This embodiment discloses a precise navigation system for endoscopic debridement of encapsulated pancreatic necrosis.

[0115] like Figure 4 As shown in the figure, the precise navigation system for endoscopic debridement of pancreatic encapsulated necrosis includes:

[0116] The CT image recognition module is configured to: obtain multiple historical abdominal CT images of the patient before endoscopic drainage and debridement, and recognize the contour areas of the pancreatic head, pancreatic neck, gland, pancreatic tail, duodenum, gastric antrum and gastric body in the abdominal CT images;

[0117] The ultrasound image recognition module is configured to: obtain an endoscopic image and an ultrasound image during the endoscopic drainage and debridement process, recognize the gastroscope part in the endoscopic image, and recognize the pancreatic area in the ultrasound image;

[0118] The feature vector construction module is configured to: construct a feature vector based on the recognition result of the gastroscope part in the endoscopic image, combined with the pancreatic region recognition result in the ultrasound image and the abdominal CT image recognition result;

[0119] The matching module is configured to: input the feature vector into a pre-trained CNN network model, and obtain an abdominal CT image with the highest matching degree with the pancreatic region recognition result in the current ultrasound image from multiple historical abdominal CT images;

[0120] The multi-planar reconstruction module is configured to: perform multi-planar reconstruction and display of abdominal CT based on the pancreatic region recognition result in the current ultrasound image and the abdominal CT image with the highest matching degree, so as to realize the positioning navigation of endoscopic debridement of pancreatic encapsulated necrosis.

[0121] Those skilled in the art should understand that the modules or steps of the present invention described above can be implemented by a general-purpose computer device, or alternatively, they can be implemented by a program code executable by a computing device, so that they can be stored in a storage device and executed by the computing device, or they can be made into individual integrated circuit modules, or multiple modules or steps therein can be made into a single integrated circuit module for implementation. The present invention is not limited to any specific combination of hardware and software.

[0122] Although the above describes the specific implementation mode of the present invention in conjunction with the accompanying drawings, it is not intended to limit the scope of protection of the present invention. Those skilled in the art should understand that various modifications or variations that can be made by those skilled in the art on the basis of the technical solution of the present invention without creative work are still within the scope of protection of the present invention.

Claims

1. A precise navigation method for endoscopic debridement of pancreatic encapsulated necrosis, characterized in that: The following steps are involved: Obtain multiple historical abdominal CT images of the patient before endoscopic drainage and debridement, and identify the contour areas of the pancreatic head, pancreatic neck, gland, pancreatic tail, duodenum, gastric antrum, and gastric body in the abdominal CT images; Obtain endoscopic images and ultrasound images during endoscopic drainage and debridement, identify the gastroscopic part in the endoscopic image, and identify the pancreatic area in the ultrasound image; Based on the recognition results of the gastroscope part in the endoscopic image, the feature vector is constructed by combining the recognition results of the pancreatic region in the ultrasound image and the recognition results of the abdominal CT image; The feature vector is input into the pre-trained CNN network model to obtain the abdominal CT image with the highest matching degree with the pancreatic region recognition result in the current ultrasound image from multiple historical abdominal CT images; Based on the pancreatic region recognition results in the current ultrasound image and the abdominal CT image with the highest matching degree, abdominal CT multi-planar reconstruction and display are performed to achieve positioning navigation for endoscopic debridement of pancreatic encapsulated necrosis.

2. The precise navigation method for endoscopic debridement of pancreatic encapsulated necrosis according to claim 1, characterized in that: Build a CT thin-layer image segmentation model; Mask labels of the pancreatic head contour area, pancreatic neck contour area, gland contour area, pancreatic tail contour area, duodenum contour area, gastric antrum contour area and gastric body contour area are annotated on multiple historical abdominal CT images, and the CT thin-layer image segmentation model is trained using the annotated multiple historical abdominal CT images to obtain a trained CT thin-layer image segmentation model; The trained CT thin-layer image segmentation model is used to identify the contour areas of the pancreatic head, pancreatic neck, gland, pancreatic tail, duodenum, gastric antrum and gastric body in abdominal CT images.

3. The precise navigation method for endoscopic debridement of pancreatic encapsulated necrosis according to claim 1, characterized in that: It also includes using a pre-trained binary classification neural network model to classify endoscopic images and ultrasound images during endoscopic drainage and debridement, and obtaining classified endoscopic images and ultrasound images, respectively.

4. The precise navigation method for endoscopic debridement of pancreatic encapsulated necrosis according to claim 1, characterized in that: Identify the gastroscopic part in the endoscopic image, including: Build a gastroscope part classification network; Obtain historical endoscopic images and annotate gastric endoscopic images, including the cardia, gastric body, pylorus, gastric fundus, gastric angle, gastric antrum, and duodenum, and train the gastroscopic site classification network; The trained gastroscope part classification network is used to identify the endoscopic parts in the endoscopic images.

5. The precise navigation method for endoscopic debridement of pancreatic encapsulated necrosis according to claim 1, characterized in that: Identify the pancreatic region in ultrasound images, including: Build an EUS ultrasound image segmentation model; Ultrasound images of historical pancreatic encapsulated necrosis during endoscopic debridement were obtained, and mask labels of the pancreatic head contour area, pancreatic neck contour area, glandular contour area, pancreatic tail contour area, duodenum contour area, gastric antrum contour area, and gastric body contour area were marked to train the EUS ultrasound image segmentation model; The trained EUS ultrasound image segmentation model is used to identify the pancreatic area in the ultrasound image.

6. The precise navigation method for endoscopic debridement of pancreatic encapsulated necrosis according to claim 2, characterized in that: Construct feature vectors. The specific process is: Different endoscopic gastric sites were represented by different characteristic values, including the cardia, gastric body, pylorus, gastric fundus, gastric angle, gastric antrum, and duodenum; Different pancreatic sub-regions are represented by different eigenvalues, including the pancreatic head, pancreatic neck, gland, and pancreatic tail; obtaining a numerical representation of a characteristic value of a stomach region identified in an ultrasound image; Calculate the area size and X, Y coordinate values ​​of the center point of the pancreatic head, pancreatic neck, gland and / or pancreatic tail identified in the ultrasound image respectively; Obtaining numerical representation of characteristic values ​​of the stomach part identified by the abdominal CT image, and calculating the area size of the identified stomach part and the coordinates of the center point of the stomach area; Calculate the area size and center point X and Y coordinate values ​​of the pancreatic head, pancreatic neck, gland and / or pancreatic tail identified in each abdominal CT image; The numerical representation of the characteristic value of the stomach part identified in the ultrasound image, the area size and the X, Y coordinate values ​​of the center point of the pancreatic head, pancreatic neck, gland and / or pancreatic tail identified in the ultrasound image, the numerical representation of the characteristic value of the stomach part identified in the abdominal CT image, the area size of the stomach part and the coordinates of the center point of the stomach area identified in the abdominal CT image, and the area size and the X, Y coordinate values ​​of the center point of the pancreatic head, pancreatic neck, gland and / or pancreatic tail identified in the abdominal CT image, a total of 20 characteristic values, are collectively used as characteristic vectors.

7. The precise navigation method for endoscopic debridement of pancreatic encapsulated necrosis according to claim 6, characterized in that: If the pancreatic head, pancreatic neck, gland and / or pancreatic tail are not identified in the ultrasound image, the area size and the X and Y coordinate values ​​of the center point of the unidentified pancreatic region are set to 0; If the pancreatic head, pancreatic neck, gland and / or pancreatic tail are not identified in the abdominal CT image, the area size and the X and Y coordinate values ​​of the center point of the corresponding unidentified pancreatic sub-region are set to zero.

8. The precise navigation method for endoscopic debridement of pancreatic encapsulated necrosis according to claim 6, characterized in that: The input layer of the CNN network model is a 20X1 floating-point vector, including multiple fully connected layers. The output layer of the fully connected network uses a Sigmoid activation function for binary classification. The output layer returns two classifications and probabilities. The two classifications are respectively that the ultrasound image and the CT image match, and that the ultrasound image and the CT image do not match.

9. The precise navigation method for endoscopic debridement of pancreatic encapsulated necrosis according to claim 8, characterized in that: It also includes training the CNN network model, including: Based on the ultrasonic image recognition result and each historical abdominal CT image recognition result, a feature vector is constructed respectively, so as to obtain multiple feature vectors based on multiple historical abdominal CT image recognition results; The CNN network model is trained using multiple feature vectors.

10. Precision navigation system for endoscopic debridement of pancreatic encapsulated necrosis, characterized by: include: The CT image recognition module is configured to: obtain multiple historical abdominal CT images of the patient before endoscopic drainage and debridement, and recognize the contour areas of the pancreatic head, pancreatic neck, gland, pancreatic tail, duodenum, gastric antrum and gastric body in the abdominal CT images; The ultrasound image recognition module is configured to: obtain an endoscopic image and an ultrasound image during the endoscopic drainage and debridement process, recognize the gastroscope part in the endoscopic image, and recognize the pancreatic area in the ultrasound image; The feature vector construction module is configured to: construct a feature vector based on the recognition result of the gastroscope part in the endoscopic image, combined with the pancreatic region recognition result in the ultrasound image and the abdominal CT image recognition result; The matching module is configured to: input the feature vector into a pre-trained CNN network model, and obtain an abdominal CT image with the highest matching degree with the pancreatic region recognition result in the current ultrasound image from multiple historical abdominal CT images; The multi-planar reconstruction module is configured to: perform multi-planar reconstruction and display of abdominal CT based on the pancreatic region recognition result in the current ultrasound image and the abdominal CT image with the highest matching degree, so as to realize the positioning navigation of endoscopic debridement of pancreatic encapsulated necrosis.

Images