Image intubation difficulty assessment method and device, computer device and storage medium

Abstract

The application relates to an image intubation difficulty evaluation method and device, computer equipment and a storage medium. The method comprises the following steps: acquiring a duodenal papilla image of a target object through an endoscope; acquiring a morphology category and a structure feature of the duodenal papilla of the target object according to the duodenal papilla image; acquiring an X-ray image of the endoscope, and acquiring an angle feature of the endoscope according to at least one of the X-ray image and the duodenal papilla image; and judging the difficulty degree of image intubation of the target object according to the morphology category and the structure feature of the duodenal papilla and the angle feature of the endoscope, so as to obtain a difficulty evaluation result. The method not only considers the morphology structure features of the duodenal papilla of the target object, but also combines the intubation situation of the endoscope before intubation, can comprehensively evaluate the difficulty degree of image intubation of the target object from multiple aspects, and achieves the purpose of accurately evaluating the difficulty degree of image intubation.

Description

Technical Field

[0001] This application relates to the fields of medical technology and computer artificial intelligence technology, and in particular to a method, apparatus, computer device, storage medium and computer program product for assessing difficulties in image intubation. Background Technology

[0002] Endoscopic retrograde cholangiopancreatography (ERCP) is a technique that combines endoscopic and fluoroscopic examination to diagnose and treat certain problems in the biliary or pancreatic duct system. ERCP is primarily used to diagnose and treat conditions of the bile ducts and major pancreatic ducts, including gallstones, inflammatory strictures (scarring), leaks (due to trauma and surgery), and cancer. ERCP can be performed for both diagnostic and therapeutic purposes, although the development of safer and relatively non-invasive studies such as magnetic resonance cholangiopancreatography (MRCP) and endoscopic ultrasound means that ERCP is now rarely performed without therapeutic intent.

[0003] ERCP specifically refers to a technique where a duodenoscope is inserted into the descending duodenum to locate the duodenal papilla. A contrast catheter is then inserted through the biopsy tube to the opening of the papilla, and contrast agent is injected before X-ray imaging to visualize the pancreaticobiliary duct. However, traditional selective bile duct cannulation techniques have a high failure rate, and the skill level of each surgeon varies. For patients with difficult cannulation, repeated intubation over a long period increases the risk of postoperative complications, such as pancreatitis, bleeding after sphincterotomy, and gastrointestinal perforation. Therefore, accurately assessing the difficulty of cannulation beforehand is crucial.

[0004] Currently, there is an urgent need for a highly accurate method for assessing the difficulty of image-based cannulation. Summary of the Invention

[0005] Therefore, it is necessary to provide an image intubation difficulty assessment method, apparatus, computer equipment, computer-readable storage medium, and computer program product that can accurately assess the degree of image intubation difficulty in response to the above-mentioned technical problems.

[0006] Firstly, this application provides a method for assessing the difficulty of image cannulation. The method includes:

[0007] Images of the duodenal papillae of the target object were obtained using an endoscope;

[0008] Based on the images of the duodenal papilla, obtain the morphological category and structural features of the duodenal papilla of the target object;

[0009] Acquire X-ray images of the endoscope, and obtain the angular features of the endoscope based on at least one of the X-ray images and duodenal papilla images;

[0010] Based on the morphological type and structural characteristics of the duodenal papilla, as well as the angular characteristics of the endoscope, the difficulty of image cannulation of the target object is determined, and the difficulty assessment result is obtained.

[0011] In one embodiment, based on the duodenal papilla image, the morphological category and structural features of the duodenal papilla of the target object are obtained, including:

[0012] The first classification model is used to identify the morphological features in the images of duodenal papillae, and the morphological category of the duodenal papillae of the target object is determined based on the morphological features;

[0013] The first segmentation model is used to segment the duodenal papilla image to obtain the first segmented image of the duodenal papilla.

[0014] The second segmentation model was used to segment the duodenal papilla image to obtain a second segmented image that segmented the opening of the duodenal papilla.

[0015] Structural features of the duodenal papilla are obtained based on at least one of the first segmented image and the second segmented image.

[0016] In one embodiment, the structural features of the duodenal papilla are obtained based on at least one of the first segmented image and the second segmented image, including any one of the following:

[0017] The first type:

[0018] Based on the first segmented image, the structural features of the duodenal papilla are obtained; the structural features include the length of the duodenal papilla.

[0019] The second type:

[0020] Based on the second segmented image, the structural features of the duodenal papilla are obtained; the structural features include the opening type of the duodenal papilla opening;

[0021] The third type:

[0022] Based on the first and second segmented images, the structural features of the duodenal papilla are obtained; the structural features include papilla length and opening type.

[0023] In one embodiment, the structural features include the papillary length of the duodenal papilla; the structural features of the duodenal papilla are obtained based on the first segmented image, including:

[0024] Depth estimation is performed on images of the duodenal papilla to obtain depth images;

[0025] In the first segmented image, the maximum inner diameter of the segmented region of the duodenal papilla is determined, and two coordinate points are determined on the segmented region of the duodenal papilla based on the maximum inner diameter; the two coordinate points are used to determine the line segment of the maximum inner diameter of the segmented region of the duodenal papilla.

[0026] Determine the target rectangle based on two coordinate points, and determine the length of the first and second sides of the target rectangle;

[0027] Obtain the average value of the line segment with the largest inner diameter in the depth image;

[0028] Determine the pixel focal length of the endoscope, and calculate the papillary length of the duodenal papilla based on the pixel focal length, the first side length, and the second side length.

[0029] In one embodiment, the structural features include the opening type of the duodenal papilla; based on the second segmented image, the structural features of the duodenal papilla are obtained, including:

[0030] In the second segmented image, the papillary opening region corresponding to the duodenal papilla opening is determined;

[0031] A second classification model was used to identify the shape features of the duodenal papilla opening region, and the opening type of the duodenal papilla opening was determined based on the shape features.

[0032] In one embodiment, the angular features of the endoscope are obtained based on at least one of the X-ray image and the duodenal papilla image, including any one of the following:

[0033] The first type:

[0034] Based on the X-ray image, obtain the angular features of the endoscope; the angular features include the endoscope angle, which represents the degree of curvature of the endoscope;

[0035] The second type:

[0036] Based on the duodenal papilla image, the angular features of the endoscope are obtained; the angular features include the opening angle, which represents the angle between the duodenal papilla opening and the endoscope.

[0037] The third type:

[0038] Based on X-ray images and duodenal papilla images, the angular features of the endoscope are obtained; the angular features include the endoscope angle and the opening angle.

[0039] In one embodiment, the angular features include the endoscope angle; obtaining the angular features of the endoscope based on the X-ray image includes:

[0040] The X-ray image was segmented using a third segmentation model to obtain a third segmented image of the endoscope.

[0041] In the third segmented image, the endoscope region corresponding to the endoscope is determined, and the endoscope centerline of the endoscope region is obtained;

[0042] The endoscope angle is determined based on the curvature of the endoscope's centerline.

[0043] In one embodiment, the angular features include the opening angle; based on the duodenal papilla image, the angular features of the endoscope are obtained, including:

[0044] The second segmentation model was used to segment the duodenal papilla image to obtain a second segmented image that segmented the opening of the duodenal papilla.

[0045] In the second segmented image, the region corresponding to the duodenal papilla opening is determined, and the center point of the papilla opening in the region is obtained.

[0046] In the image of the duodenal papilla, determine the center point of the distal end of the duodenal papilla, and determine the first line segment based on the center point of the papilla opening and the center point of the distal end;

[0047] The fourth segmentation model was used to segment the duodenal papilla image, resulting in a fourth segmentation image that segments the duodenal papilla folds.

[0048] In the fourth segmented image, the region corresponding to the folded wall is determined, and the horizontal midline of the folded wall region is determined as the second line segment.

[0049] Determine the opening angle based on the included angle between the first and second line segments.

[0050] In one embodiment, the difficulty of image cannulation of the target object is determined based on the morphological type and structural characteristics of the duodenal papilla, as well as the angular characteristics of the endoscope, to obtain a difficulty assessment result, including:

[0051] The scores for each dimension are determined based on the morphological type and structural characteristics of the duodenal papilla and the angular characteristics of the endoscope.

[0052] The difficulty level score is obtained by weighting the scores of each dimension.

[0053] The difficulty assessment result is obtained based on the difficulty level score.

[0054] Secondly, this application also provides an image intubation difficulty assessment device. The device includes:

[0055] The image acquisition module is used to acquire images of the duodenal papilla of the target object through an endoscope;

[0056] The first processing module is used to obtain the morphological category and structural features of the duodenal papilla of the target object based on the duodenal papilla image;

[0057] The second processing module is used to acquire X-ray images of the endoscope and to acquire the angular features of the endoscope based on at least one of the X-ray images and the duodenal papilla images.

[0058] The difficulty assessment module is used to determine the difficulty of image cannulation of the target object based on the morphological type and structural characteristics of the duodenal papilla and the angular characteristics of the endoscope, and to obtain the difficulty assessment result.

[0059] Thirdly, this application also provides a computer device. The computer device includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program to perform the following steps:

[0060] Images of the duodenal papillae of the target object were obtained using an endoscope;

[0061] Based on the images of the duodenal papilla, obtain the morphological category and structural features of the duodenal papilla of the target object;

[0062] Acquire X-ray images of the endoscope, and obtain the angular features of the endoscope based on at least one of the X-ray images and duodenal papilla images;

[0063] Based on the morphological type and structural characteristics of the duodenal papilla, as well as the angular characteristics of the endoscope, the difficulty of image cannulation of the target object is determined, and the difficulty assessment result is obtained.

[0064] Fourthly, this application also provides a computer-readable storage medium. The computer-readable storage medium stores a computer program thereon, which, when executed by a processor, performs the following steps:

[0065] Images of the duodenal papillae of the target object were obtained using an endoscope;

[0066] Based on the images of the duodenal papilla, obtain the morphological category and structural features of the duodenal papilla of the target object;

[0067] Acquire X-ray images of the endoscope, and obtain the angular features of the endoscope based on at least one of the X-ray images and duodenal papilla images;

[0068] Based on the morphological type and structural characteristics of the duodenal papilla, as well as the angular characteristics of the endoscope, the difficulty of image cannulation of the target object is determined, and the difficulty assessment result is obtained.

[0069] Fifthly, this application also provides a computer program product. The computer program product includes a computer program that, when executed by a processor, performs the following steps:

[0070] Images of the duodenal papillae of the target object were obtained using an endoscope;

[0071] Based on the images of the duodenal papilla, obtain the morphological category and structural features of the duodenal papilla of the target object;

[0072] Acquire X-ray images of the endoscope, and obtain the angular features of the endoscope based on at least one of the X-ray images and duodenal papilla images;

[0073] Based on the morphological type and structural characteristics of the duodenal papilla, as well as the angular characteristics of the endoscope, the difficulty of image cannulation of the target object is determined, and the difficulty assessment result is obtained.

[0074] The aforementioned image-based cannulation difficulty assessment method, apparatus, computer equipment, storage medium, and computer program product acquire images of the duodenal papilla of the target object through an endoscope; based on the duodenal papilla images, obtain the morphological type and structural characteristics of the duodenal papilla of the target object; acquire X-ray images of the endoscope; based on at least one of the X-ray images and the duodenal papilla images, obtain the angular characteristics of the endoscope; and based on the morphological type, structural characteristics of the duodenal papilla, and the angular characteristics of the endoscope, determine the degree of difficulty of image-based cannulation of the target object, and obtain a difficulty assessment result. This method not only considers the morphological and structural characteristics of the duodenal papilla itself, but also combines the endoscope insertion status of the target object before cannulation, enabling a comprehensive assessment of the difficulty of image-based cannulation of the target object from multiple aspects, achieving the goal of accurately assessing the difficulty of image-based cannulation. Attached Figure Description

[0075] Figure 1 This is a schematic diagram of a scenario for evaluating the difficulty of image cannulation in one embodiment;

[0076] Figure 2 This is a flowchart illustrating the image cannulation difficulty assessment method in one embodiment;

[0077] Figure 3 This is a schematic diagram of a normal nipple in one embodiment;

[0078] Figure 4 This is a schematic diagram of a stone lodged in a nipple in one embodiment;

[0079] Figure 5 This is a schematic diagram of a nipple beside or inside a diverticulum in one embodiment;

[0080] Figure 6 This is a schematic diagram of the small nipple in one embodiment;

[0081] Figure 7 This is a schematic diagram of a long-nose nipple in one embodiment;

[0082] Figure 8 This is a schematic diagram of a papillary adenoma or tumor in one embodiment.

[0083] Figure 9 This is a schematic diagram of the duodenal papilla in one embodiment;

[0084] Figure 10 This is a schematic diagram of the first segmented image in one embodiment;

[0085] Figure 11 This is a schematic diagram of a depth image in one embodiment;

[0086] Figure 12 This is a schematic diagram illustrating the calculation of nipple length in one embodiment;

[0087] Figure 13 This is a schematic diagram of another duodenal papilla in one embodiment;

[0088] Figure 14 This is a schematic diagram of the second segmented image in one embodiment;

[0089] Figure 15 This is a schematic diagram of an X-ray image in one embodiment;

[0090] Figure 16 This is a schematic diagram of the third segmented image in one embodiment;

[0091] Figure 17 This is a schematic diagram illustrating the calculation of the endoscope angle in one embodiment;

[0092] Figure 18 This is a schematic diagram of the segmented outline of the duodenal papilla in one embodiment;

[0093] Figure 19 This is a schematic diagram of the fourth segmented image in one embodiment;

[0094] Figure 20 This is a schematic diagram illustrating the calculation of the opening angle in one embodiment;

[0095] Figure 21 This is a logic flowchart of an image cannulation difficulty assessment method in one embodiment;

[0096] Figure 22 This is a structural block diagram of an image cannulation difficulty assessment device in one embodiment;

[0097] Figure 23 This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation

[0098] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0099] The image intubation difficulty assessment method provided in this application embodiment can be applied to, for example... Figure 1 In the application environment shown, computer device 102 communicates with image intubation device 104 and fluoroscopy device 106 via wired or wireless communication. Image device 104, fluoroscopy device 106, and computer device work together in a specific medical environment (such as an operating room) to perform intubation on a target object. Image device 104 includes, but is not limited to, ERCP, MRCP, and CRCP devices. It is understood that the computer device can specifically be a terminal or a server. The terminal can be, but is not limited to, various personal computers, laptops, smartphones, tablets, smart medical devices, etc. The server can be implemented using a standalone server or a server cluster consisting of multiple servers.

[0100] In one embodiment, such as Figure 2 As shown, an image cannulation difficulty assessment method is provided, which is then applied to... Figure 1 Taking computer device 102 as an example, the following steps are included:

[0101] Step 202: Obtain an image of the duodenal papilla of the target object using an endoscope.

[0102] The target group refers to patients who need to undergo medical examinations. Imaging cannulation includes, but is not limited to, ERCP cannulation, MRCP cannulation, and CRCP cannulation.

[0103] Optionally, the endoscope is inserted into the descending duodenum of the target patient. The duodenal papilla is the common opening of the common bile duct and pancreatic duct. When the endoscope enters the descending duodenum, a longitudinal fold can be seen on the posteromedial wall, which is the impression left by the common bile duct obliquely passing through the duodenal wall. The protrusion at its lower end is called the duodenal (major / minor) papilla, located approximately 75 cm from the incisors. A normal duodenal papilla appears as a pinkish protrusion with several annular folds on its oral surface, called the fasciculus folds. On the anal side of the duodenal papilla, there are several slightly V-shaped folds called the fasciculus minor. These longitudinally oriented folds are collectively called the longitudinal duodenal folds, which are important landmarks for locating the duodenal papilla. After locating the duodenal papilla, an image of the duodenal papilla is taken using the endoscope. The image of the duodenal papilla includes at least the duodenal papilla, the opening of the duodenal papilla, and the folds. During ERCP cannulation, the endoscope refers to the duodenoscope used for ERCP cannulation.

[0104] Step 204: Based on the duodenal papilla image, obtain the morphological category and structural features of the duodenal papilla of the target object.

[0105] Among them, morphological type refers to the classification of duodenal papillae based on different shapes and sizes. For example, duodenal papillae can be divided into... Figure 3 The normal nipple shown, such as Figure 4 The stone is lodged in the nipple, as shown. Figure 5 The nipple next to or inside the diverticulum, as shown Figure 6 The small nipple shown, such as Figure 7 The long nose nipple shown, as Figure 8 The diagram illustrates six morphological types of papillary adenomas or tumors. Normal papillae are generally the easiest to cannulate. Cannulation becomes progressively more difficult with papillae impacted by stones, papillae adjacent to or within diverticula, small papillae, long-nosed papillae, and papillary adenomas or tumors. Structural features include, but are not limited to, the length of the duodenal papilla and the type of its opening. The opening type characterizes the shape of the duodenal papilla opening, which can be categorized as slit-like, villous, punctate, linear, single-orifice, and halo-shaped. The size of the duodenal papilla opening varies depending on the type. Generally, shorter papillae are easier to cannulate, while longer papillae become progressively more difficult. Larger duodenal papilla openings are easier to cannulate; for example, single-orifice openings are easier to cannulate. Smaller duodenal papilla openings are more difficult to cannulate; for example, slit-like, villous, or punctate openings are more challenging to cannulate.

[0106] Optionally, morphological features in the duodenal papilla image can be identified, and the morphological category of the duodenal papilla of the target object can be determined based on the morphological features.

[0107] Furthermore, a first segmented image containing only the duodenal papilla can be segmented from the duodenal papilla image. The papilla length in the first segmented image can be calculated, and the papilla length can be used as a structural feature of the duodenal papilla. Alternatively, a second segmented image containing only the duodenal papilla openings can be segmented from the duodenal papilla image. The opening type of the duodenal papilla openings in the second segmented image can be identified, and the opening type can be used as a structural feature of the duodenal papilla. Another approach is to first segment a first segmented image containing only the duodenal papilla, calculate the papilla length in the first segmented image, then segment a second segmented image containing only the duodenal papilla openings, identify the opening type of the duodenal papilla openings in the second segmented image, and finally use both the papilla length and opening type as structural features of the duodenal papilla.

[0108] Step 206: Obtain X-ray images of the endoscope, and obtain the angular features of the endoscope based on at least one of the X-ray images and the duodenal papilla images.

[0109] X-ray images refer to X-ray images of the endoscope taken from outside the body using fluoroscopic equipment when the endoscope is inserted into the target body. Angular characteristics include, but are not limited to, the endoscope angle and the opening angle. The endoscope angle characterizes the degree of curvature of the endoscope, while the opening angle characterizes the angle between the duodenal papilla opening and the endoscope lens. Generally, intubation is easiest when the endoscope angle is between 90° and 120°, and most difficult when the endoscope angle is less than 90° or greater than 135°. A 90° opening angle provides the optimal imaging position of the endoscope relative to the duodenal papilla opening, making intubation easiest. An opening angle other than 90° increases the difficulty of intubation.

[0110] Optionally, the endoscope's outline can be segmented from the X-ray image, and the endoscope's bending angle can be calculated to obtain the endoscope angle, which can then be used as the endoscope's angular feature. Alternatively, a second segmented image containing only the duodenal papilla opening can be segmented from the duodenal papilla image. The angle between the duodenal papilla opening and the endoscope can be determined based on the shooting angle of the second segmented image, and this opening angle can be used as the endoscope's angular feature. Another option is to first segment the endoscope's outline from the X-ray image, calculate the endoscope's bending angle to obtain the endoscope angle, then segment a second segmented image containing only the duodenal papilla opening from the duodenal papilla image, determine the angle between the duodenal papilla opening and the endoscope based on the shooting angle of the second segmented image, and finally use both the endoscope angle and the opening angle as the endoscope's angular feature.

[0111] Step 208: Based on the morphological type and structural characteristics of the duodenal papilla, as well as the angular characteristics of the endoscope, determine the difficulty of image cannulation of the target object and obtain the difficulty assessment result.

[0112] Optionally, based on four factors—morphological category, structural characteristics, endoscope angle, and opening angle—the difficulty of intubation is first determined under the influence of each factor. Finally, the overall difficulty of intubation is determined by combining all four factors, resulting in a difficulty assessment result.

[0113] In one feasible implementation, a weighted summation method is used to assign different weights to dimensions such as morphological category, structural features, and endoscopic angular features based on different usage scenarios or different physician skill levels. First, difficulty scores for morphological category, structural features, and endoscopic angular features are calculated separately. Then, the scores for each dimension are summed according to their weights to obtain the final difficulty assessment result. The endoscopic angular features may also include the endoscope angle and the opening angle. Therefore, different weights are assigned to morphological category, structural features, endoscope angle, and opening angle. First, difficulty scores for these four aspects are calculated separately. Then, the four difficulty scores are summed according to their weights to obtain the final difficulty assessment result.

[0114] In another feasible implementation, the morphological category, structural features, and angular features of the endoscope can be used as input to train a difficulty classification model. The output of the classification model is the difficulty level. Specifically, the difficulty level can include very difficult, moderately difficult, easy, and very easy. In other embodiments, more or fewer difficulty levels can be set as needed. In practical applications, the morphological category, structural features of the duodenal papilla, and angular features of the endoscope are processed by the classification model to output a classification result, which is the difficulty assessment result. Further, the angular features of the endoscope can also include the endoscope angle and the opening angle. A difficulty classification model can then be trained, using the morphological category, structural features, endoscope angle, and opening angle as input data to obtain the output result of the difficulty classification model, which is used as the difficulty assessment result. The output result of the difficulty assessment model can be divided into multiple difficulty levels such as difficult, moderately difficult, easy, and very easy. The difficulty assessment model can be obtained by training a neural network based on a pre-built training set. The training set includes multiple training instances, each training instance including a set of feature data and a difficulty level corresponding to the feature data. Each set of feature data includes at least one feature among morphological category, structural feature, endoscope angle and opening angle.

[0115] In the aforementioned image-based cannulation difficulty assessment method, an image of the duodenal papilla of the target object is obtained through endoscopy; based on the duodenal papilla image, the morphological type and structural characteristics of the duodenal papilla of the target object are obtained; an X-ray image of the endoscope is obtained; based on at least one of the X-ray image and the duodenal papilla image, the angular characteristics of the endoscope are obtained; based on the morphological type and structural characteristics of the duodenal papilla, as well as the angular characteristics of the endoscope, the difficulty of image-based cannulation of the target object is determined, and a difficulty assessment result is obtained. This method not only considers the morphological and structural characteristics of the duodenal papilla itself, but also combines the endoscope insertion status of the target object before cannulation, enabling a comprehensive assessment of the difficulty of image-based cannulation of the target object from multiple aspects, achieving the goal of accurately assessing the difficulty of image-based cannulation.

[0116] In one embodiment, obtaining the morphological category of the duodenal papilla of the target object based on the duodenal papilla image includes: using a first classification model to identify morphological features in the duodenal papilla image, and determining the morphological category of the duodenal papilla of the target object based on the morphological features.

[0117] The first classification model is obtained by training a neural network on a first classification training set. The first classification training set includes multiple training instances, each of which includes a sample duodenal papilla image and a corresponding morphological feature label. The first classification model can be trained using a convolutional neural network, and the type of convolutional neural network is not limited here.

[0118] Optionally, the duodenal papilla image of the target object is input into the first classification model to obtain the morphological features in the duodenal papilla image. Then, the correspondence between the morphological features and the morphological category is determined according to medical theory, thereby determining the morphological category of the duodenal papilla of the target object. The morphological category can be divided into normal papilla, stone-impacted papilla, papilla next to or inside the diverticulum, small papilla, long-nose papilla, and papillary adenoma or tumor.

[0119] In one feasible implementation, morphological quantification can be used to identify morphological features in duodenal papilla images. The duodenal papilla images can be preprocessed, including but not limited to: data format conversion, normalization, image binarization, data augmentation, flipping, translation, and rotation. Regions of Interest (ROIs) are then identified in the preprocessed duodenal papilla images. The portion of the duodenal papilla image containing only the duodenal papilla region is designated as the ROI, and its contour can be obtained. Based on the morphology of the duodenal papilla, feature recognition can be performed on the contour of the ROI to determine the morphological category of the target duodenal papilla.

[0120] In this embodiment, a first classification model is used to identify morphological features in duodenal papilla images, which can accurately determine the morphological category of the duodenal papilla of the target object based on the morphological features.

[0121] In one embodiment, the structural category of the duodenal papilla of the target object is obtained based on the duodenal papilla image, including any of the following:

[0122] The first method involves segmenting the duodenal papilla image using a first segmentation model to obtain a first segmented image of the duodenal papilla. Depth estimation is performed on the duodenal papilla image to obtain a depth image. In the first segmented image, the maximum inner diameter of the segmented region of the duodenal papilla is determined, and two coordinate points are determined on the segmented region based on the maximum inner diameter. These two coordinate points are used to determine the line segment representing the maximum inner diameter of the segmented region of the duodenal papilla. A target rectangle is determined based on the two coordinate points, and the first and second side lengths of the target rectangle are determined. The average value of the maximum inner diameter line segment in the depth image is obtained. The pixel focal length of the endoscope is determined, and the papilla length of the duodenal papilla is calculated based on the pixel focal length, the first side length, and the second side length. This papilla length is used as a structural feature of the duodenal papilla.

[0123] The second method involves using a second segmentation model to segment the duodenal papilla image, resulting in a second segmented image that identifies the duodenal papilla openings. Within this second segmented image, the papilla opening region corresponding to the duodenal papilla opening is determined. A second classification model is then used to identify the shape features of the papilla opening region. Based on these shape features, the opening type of the duodenal papilla is determined and used as a structural feature of the duodenal papilla.

[0124] The third method involves using the first segmentation model to segment the duodenal papilla image, resulting in a first segmented image of the duodenal papilla. Depth estimation is performed on the duodenal papilla image to obtain a depth image. In the first segmented image, the maximum inner diameter of the segmented region of the duodenal papilla is determined, and two coordinate points are determined on the segmented region based on the maximum inner diameter. These two coordinate points are used to determine the line segment of the maximum inner diameter of the segmented region of the duodenal papilla. A target rectangle is determined based on the two coordinate points, and the first and second side lengths of the target rectangle are determined. The average value of the maximum inner diameter line segment in the depth image is obtained. The pixel focal length of the endoscope is determined, and the papilla length of the duodenal papilla is calculated based on the pixel focal length, the first side length, and the second side length. The second segmentation model is then used to segment the duodenal papilla image, resulting in a second segmented image of the duodenal papilla opening. In the second segmented image, the papilla opening region corresponding to the duodenal papilla opening is determined. A second classification model is used to identify the shape features of the papilla opening region, and the opening type of the duodenal papilla opening is determined based on the shape features. The length of the duodenal papilla and the type of its opening are considered together as structural features of the duodenal papilla.

[0125] The first segmentation model is obtained by training a neural network on a first segmentation training set. The first segmentation training set includes multiple training instances, each of which includes a sample duodenal papilla image and a segmentation label for the duodenal papilla region corresponding to the sample duodenal papilla image. The first segmentation model can be trained using a convolutional neural network, and the type of convolutional neural network is not limited here.

[0126] The second segmentation model is obtained by training a neural network based on a second segmentation training set. The second segmentation training set includes multiple training instances, each of which includes a sample duodenal papilla image and a segmentation label for the duodenal papilla opening region corresponding to the sample duodenal papilla image. The second segmentation model can be trained using a convolutional neural network, and the type of convolutional neural network is not limited here.

[0127] The second classification model is obtained by training a neural network on a second classification training set. The second classification training set includes multiple training instances, each of which includes a sample image of the duodenal papilla opening and shape feature labels corresponding to the papilla opening region in the sample duodenal papilla opening image. The second classification model can be trained using a convolutional neural network, and the type of convolutional neural network is not limited here.

[0128] Specifically, the first approach: using the first segmentation model for... Figure 9 Image segmentation was performed on the duodenal papilla image shown, resulting in the following: Figure 10 The first segmented image is shown. Depth estimation is performed on the duodenal papilla image to obtain the following result: Figure 11The depth image is shown. In the first segmented image, two coordinate points for the maximum inner diameter are determined, namely Point1(x1,y1) and Point2(x2,y2), as shown. Figure 12 The line formed by these two points is shown. The average value of this line in the depth map is denoted as depth. avg Then, calculate the rectangle enclosed by these two points. Rectangle width: Wi = x2 - x1; rectangle height: Hi = y2 - y1; maximum inner diameter. Calculate the camera pixel focal length of the endoscope based on the camera's focal length in millimeters (f, in mm) and the sensor aperture (S). w S h (Sensor size, in mm), the width and height of the image taken by the camera, and the pixel focal lengths fx and fy: fx = max(width, height) * f / Sw, fy = max(width, height) * f / Sh. Calculate the actual width of the duodenal papilla: W = Wi * depthavg / fx, then calculate the actual height of the duodenal papilla: H = Hi * depthavg / fy, and finally calculate the length of the duodenal papilla. The length of the duodenal papilla is considered a structural feature of the duodenal papilla.

[0129] The second approach: Use the second segmentation model for... Figure 13 Image segmentation was performed on the duodenal papilla image shown, resulting in the following: Figure 14 The second segmented image is shown. In the second segmented image, the white area is identified as the papillary opening region. Then, the second segmented image is input into the second classification model to obtain the shape features of the papillary opening region. Based on medical theory, the correspondence between the shape features and the opening type is determined, thereby determining the opening type of the duodenal papilla of the target object. The opening type can be divided into slit-like, villous, punctate, strip-like, single-orifice, and halo-like. The opening type is used as the structural feature of the duodenal papilla.

[0130] The third option is to simultaneously implement the first and second options, and finally use the nipple length D and the opening type together as the structural features of the duodenal papilla.

[0131] In this embodiment, a first segmentation model is used to segment the duodenal papilla image to obtain a first segmented image of the duodenal papilla; a second segmentation model is used to segment the duodenal papilla image to obtain a second segmented image of the duodenal papilla opening; the accurate structural features of the duodenal papilla can be obtained based on at least one of the first and second segmented images.

[0132] In one embodiment, the angular features of the endoscope are obtained based on at least one of the X-ray image and the duodenal papilla image, including any one of the following:

[0133] The first method involves using a third segmentation model to segment the X-ray image, resulting in a third segmented image of the endoscope. In the third segmented image, the endoscope region corresponding to the endoscope is determined, and the endoscope centerline of the endoscope region is obtained. Based on the curvature of the endoscope centerline, the endoscope angle is determined, and the endoscope angle is used as the angular feature of the endoscope.

[0134] The second method involves segmenting the duodenal papilla image using a second segmentation model to obtain a second segmented image that identifies the duodenal papilla openings. In the second segmented image, the papilla opening region corresponding to the duodenal papilla opening is determined, and the center point of the papilla opening region is obtained. In the duodenal papilla image, the terminal center point of the duodenal papilla is determined, and a first line segment is determined based on the papilla opening center point and the terminal center point. A fourth segmentation model is then used to segment the duodenal papilla image to obtain a fourth segmented image that identifies the duodenal papilla's folds. In the fourth segmented image, the folds corresponding to the folds are determined, and the horizontal midline of the folds is defined as the second line segment. The opening angle is determined based on the angle between the first and second line segments, and this opening angle is used as the angular feature of the endoscope.

[0135] The third method involves segmenting the X-ray image using a third segmentation model to obtain a third segmented image of the endoscope. Within this third segmented image, the endoscope region corresponding to the endoscope is determined, and the endoscope centerline of that region is obtained. The endoscope angle is determined based on the curvature of the endoscope centerline. The second segmentation model is then used to segment the duodenal papilla image, resulting in a second segmented image of the duodenal papilla opening. Within this second segmented image, the papilla opening region corresponding to the duodenal papilla opening is determined, and the center point of the papilla opening is obtained. In the duodenal papilla image, the terminal center point of the duodenal papilla is determined, and a first line segment is defined based on the opening center point and the terminal center point. Finally, the fourth segmentation model is used to segment the duodenal papilla image, resulting in a fourth segmented image of the duodenal papilla's folded walls. Within this fourth segmented image, the folded wall region corresponding to the folded walls is determined, and the horizontal midline of this region is defined as the second line segment. The opening angle is determined based on the angle between the first and second line segments. The endoscope angle and the opening angle are used together as the angular features of the endoscope.

[0136] The third segmentation model is obtained by training a neural network based on a third segmentation training set. The third segmentation training set includes multiple training instances, each of which includes a sample X-ray image and a corresponding endoscopic region segmentation label. The third segmentation model can be trained using a convolutional neural network, and the type of convolutional neural network is not limited here.

[0137] The fourth segmentation model is obtained by training a neural network on a fourth segmentation training set. The fourth segmentation training set includes multiple training instances, each of which includes a sample duodenal papilla image and a segmentation label for the corresponding duodenal papilla fold region. The fourth segmentation model can be trained using a convolutional neural network, and the type of convolutional neural network is not limited here.

[0138] Optionally, the first approach: Use a third segmentation model for... Figure 15 The X-ray image shown is segmented to obtain the following: Figure 16 The third segmented image is shown. In the third segmented image, the white area is identified as the segmented endoscope region, as shown below. Figure 17 As shown, the midline of the endoscope region is taken, and then the slope of the midline is calculated to find two line segments formed by three inflection points. The angle between these two line segments is the angle of the endoscope under X-ray, denoted as the endoscope angle α. The endoscope angle is used as the angular feature of the endoscope.

[0139] The second approach: Use the second segmentation model for... Figure 13 Image segmentation was performed on the duodenal papilla image shown, resulting in the following: Figure 14 The second segmented image is shown. In the second segmented image, the white area is determined as the duodenal papilla opening region, and the center point of the duodenal papilla opening region, point1(x1,y1), is taken. The first segmentation model is used for... Figure 13 Image segmentation was performed on the duodenal papilla image shown, resulting in the following: Figure 18 The image shown is a segmented image of the duodenal papilla. The center point of the distal end of the duodenal papilla, point2(x2,y2), is determined within this segmented image. Points1 and 2 form a longitudinal fold line segment, denoted as the first line segment AB. The fourth segmentation model is then used to further segment the duodenal papilla. Figure 13 Image segmentation was performed on the duodenal papilla image shown, resulting in the following: Figure 19 The fourth segmented image is shown. In the fourth segmented image, the white area is identified as the folded wall region, and the horizontal midline of the folded wall region is identified as the folded wall line segment, denoted as the second line segment CD. Figure 20 As shown, according to the first line segment AB ( Figure 20 The oblique line segment in the middle) and the second line segment CD ( Figure 20The opening angle β is obtained by taking the angle between the horizontal line segments in the endoscope and using the opening angle β as the angular feature of the endoscope.

[0140] The third approach: Simultaneously implement the first and second approaches, and finally use the endoscope angle α and the opening angle β together as the angular features of the endoscope.

[0141] In this embodiment, the accurate angular features of the endoscope can be obtained based on at least one of the X-ray image and the duodenal papilla image.

[0142] In one embodiment, the difficulty of image cannulation of the target object is determined based on the morphological category and structural characteristics of the duodenal papilla and the angular characteristics of the endoscope, and a difficulty assessment result is obtained. This includes: determining the score of each dimension based on the morphological category and structural characteristics of the duodenal papilla and the angular characteristics of the endoscope; obtaining a difficulty score based on the weighted sum of the scores of each dimension; and obtaining a difficulty assessment result based on the difficulty score.

[0143] The angular features of the endoscope can include the endoscope angle and the opening angle. Based on the morphological category and structural features of the duodenal papilla, and the angular features of the endoscope, scores are determined for each dimension. Specifically, this includes: determining a first score based on morphological category (the first score is lowest if the morphological category is a normal papilla); determining a second score based on structural features (the second score is lowest if the structural features conform to a preset standard structure); determining a third score based on the endoscope angle (the third score is lowest if the endoscope angle is within a preset angle range); and determining a fourth score based on the opening angle (the fourth score is lowest if the opening angle is 90°). Then, a difficulty score is obtained by weighted summing the scores for each dimension. Specifically, the first, second, third, and fourth scores are weighted and summed according to preset weighting coefficients to calculate the difficulty score. The magnitude of the difficulty score is used to characterize the difficulty of image cannulation of the target object.

[0144] Optionally, a first score, Score1, is determined based on the morphological category. The first score is the lowest when the morphological category is normal papilla. For example, Score1 = 0 when the duodenal papilla of the target object is a normal papilla, Score1 = 1 when it is a papilla with impacted stones, Score1 = 2 when it is a papilla adjacent to or within a diverticulum, Score1 = 3 when it is a small papilla, Score1 = 4 when it is a long-nose papilla, and Score1 = 5 when it is a papillary adenoma or tumor.

[0145] Based on the structural features, a second score, Score2, is determined. The second score is lowest when the structural features conform to a preset standard structure. Taking the papilla length D as a structural feature as an example, the preset standard structure can be that the papilla length D does not exceed 10 mm. Then, Score2 = 0 when 0 < D < 10 mm, Score2 = 1 when 10 < D < 20 mm, and Score2 = 2 when D > 20 mm. Taking the opening type as a structural feature as an example, the preset standard structure can be that the opening type is single-orifice. In this case, the duodenal papilla opening is the largest and most regular. Then, Score2 = 0 when the opening type is single-orifice, Score2 = 1 when the opening type is slit-shaped, dot-shaped, or strip-shaped, Score2 = 3 when the opening type is halo-shaped, and Score2 = 4 when the opening type is villous. Taking nipple length and opening type as structural features as an example, sub-scores for nipple length D and opening type can be calculated separately. The second score Score2 is obtained by averaging the two sub-scores. Alternatively, different weights can be assigned to nipple length and opening type, and the second score Score2 can be obtained by weighting the weights and the two sub-scores.

[0146] The third score, Score3, is determined based on the endoscopic angle. The score is lowest when the endoscopic angle α is within a preset angle range. The preset angle range can be between 90° and 120°. Therefore, Score3 = 0 when 90°≤α≤120°, Score3 = 1 when 120°≤α≤135°, and Score3 = 2 when α<90° or α>135°.

[0147] Based on the opening angle, the fourth score, Score4, is determined. When the opening angle β is 90°, the endoscope's imaging position of the duodenal papilla opening is most accurate, and the fourth score value is the smallest. That is, when β = 90°, Score3 = 0, and when β ≠ 90°, Score3 = 1.

[0148] Finally, calculate the overall difficulty score:

[0149] total_score=a1*Score1+a2*Score2+a3*Score3+a4*Score4,

[0150] Here, a1, a2, a3, and a4 are preset weighting coefficients for each score item. Each preset weighting coefficient can be customized according to different usage scenarios or different doctor skill levels. For example, when the doctor's skill level is high and the endoscopic angle has little impact on the doctor's intubation operation, a3 and a4 can be set to smaller values, while a1 and a2 can be set to larger values, focusing on calculating the difficulty score based on the characteristics of the duodenal papilla. When the doctor's skill level is greatly affected by the endoscopic shooting angle, and the opening angle has a greater impact on the doctor's intubation operation, a4 can be set to a larger value, while a1, a2, and a3 can be set to smaller values, focusing on calculating the difficulty score based on the shooting angle of the endoscope relative to the duodenal papilla opening. The smaller the difficulty score, the lower the difficulty of image-based intubation.

[0151] In one feasible implementation, a doctor mapping table corresponding to different scores is pre-constructed. The table records the association information between each doctor and different difficulty levels. For example, doctors with older experience and higher operational skills correspond to higher difficulty levels, while doctors with younger experience and lower operational skills correspond to lower difficulty levels. In this way, after each patient assessment to obtain a difficulty level score, the most suitable doctor to perform the intubation procedure can be directly matched from the doctor mapping table based on the difficulty level score.

[0152] In this embodiment, a difficulty score is obtained based on morphological category, structural features, endoscope angle, and opening angle. The numerical value of the difficulty score is used to characterize the difficulty of image cannulation of the target object, and the difficulty score is used as the difficulty assessment result. This approach not only considers the morphological and structural characteristics of the duodenal papilla itself but also incorporates the endoscope insertion status before cannulation, enabling a comprehensive assessment of the difficulty of image cannulation of the target object from multiple perspectives, thus achieving the goal of accurately assessing the difficulty of image cannulation.

[0153] In one embodiment, such as Figure 21 As shown, a method for assessing the difficulty of ERCP cannulation includes:

[0154] Images of the duodenal papillae of the target object are obtained using an endoscope.

[0155] A first classification model is used to identify morphological features in images of duodenal papillae, and the morphological category of the duodenal papillae of the target object is determined based on these features. A first score is determined based on the morphological category; if the morphological category is normal papillae, the first score is the lowest.

[0156] The duodenal papilla image is segmented using a first segmentation model to obtain a first segmented image of the duodenal papilla. Depth estimation is performed on the duodenal papilla image to obtain a depth image. In the first segmented image, the maximum inner diameter of the segmented region of the duodenal papilla is determined, and two coordinate points are determined on the segmented region based on the maximum inner diameter. These two coordinate points are used to determine the line segment representing the maximum inner diameter of the segmented region. A target rectangle is determined based on the two coordinate points, and the first and second side lengths of the target rectangle are determined. The average value of the maximum inner diameter line segment in the depth image is obtained. The pixel focal length of the endoscope is determined, and the papilla length of the duodenal papilla is calculated based on the pixel focal length, the first side length, and the second side length. A second score is determined based on the papilla length; if the papilla length conforms to a preset standard structure, the second score is minimized.

[0157] A third segmentation model is used to segment the X-ray image, resulting in a third segmented image of the endoscope. Within this third segmented image, the endoscope region corresponding to the endoscope is determined, and the endoscope centerline of that region is acquired. The endoscope angle is determined based on the curvature of the endoscope centerline. A third score is then determined based on the endoscope angle; the third score is minimized if the endoscope angle falls within a preset angle range.

[0158] A second segmentation model is used to segment the duodenal papilla image, resulting in a second segmented image showing the duodenal papilla openings. In this second segmented image, the papilla opening region corresponding to the opening is determined, and the center point of the opening is obtained. In the duodenal papilla image, the center point of the distal end is determined, and a first line segment is defined based on the opening and distal center points. A fourth segmentation model is then used to segment the duodenal papilla image, resulting in a fourth segmented image showing the tufted folds of the duodenal papilla. In this fourth segmented image, the tufted fold region corresponding to the tufted folds is determined, and the horizontal midline of this region is defined as the second line segment. The opening angle is determined based on the angle between the first and second line segments. A fourth score is determined based on the opening angle; if the opening angle is 90°, the fourth score is the lowest.

[0159] The first, second, third, and fourth scores are weighted and summed according to a preset weighting coefficient to calculate the difficulty score, which is used as the difficulty assessment result. The magnitude of the difficulty score is used to characterize the difficulty of ERCP cannulation for the target object.

[0160] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.

[0161] Based on the same inventive concept, this application also provides an image intubation difficulty assessment device for implementing the image intubation difficulty assessment method described above. The solution provided by this device is similar to the solution described in the above method. Therefore, the specific limitations of one or more image intubation difficulty assessment device embodiments provided below can be found in the limitations of the image intubation difficulty assessment method above, and will not be repeated here.

[0162] In one embodiment, such as Figure 22 As shown, an image intubation difficulty assessment device 2200 is provided, comprising: an image acquisition module 2201, a first processing module 2202, a second processing module 2203, and a difficulty assessment module 2204, wherein:

[0163] Image acquisition module 2201 is used to acquire images of the duodenal papilla of the target object through an endoscope;

[0164] The first processing module 2202 is used to obtain the morphological category and structural features of the duodenal papilla of the target object based on the duodenal papilla image;

[0165] The second processing module 2203 is used to acquire X-ray images of the endoscope and to acquire the angular features of the endoscope based on at least one of the X-ray images and the duodenal papilla images.

[0166] The difficulty assessment module 2204 is used to determine the difficulty of image cannulation of the target object based on the morphological type and structural characteristics of the duodenal papilla and the angular characteristics of the endoscope, and to obtain the difficulty assessment result.

[0167] In one embodiment, the first processing module 2202 is further configured to: identify morphological features in the duodenal papilla image using a first classification model; determine the morphological category of the duodenal papilla of the target object based on the morphological features; perform image segmentation on the duodenal papilla image using a first segmentation model to obtain a first segmented image of the duodenal papilla; perform image segmentation on the duodenal papilla image using a second segmentation model to obtain a second segmented image of the duodenal papilla opening; and obtain the structural features of the duodenal papilla based on at least one of the first segmented image and the second segmented image.

[0168] In one embodiment, the first processing module 2202 is further configured to obtain structural features of the duodenal papilla based on the first segmented image; the structural features include the papilla length of the duodenal papilla;

[0169] The first processing module 2202 is further configured to obtain the structural features of the duodenal papilla based on the second segmented image; the structural features include the opening type of the duodenal papilla opening;

[0170] The first processing module 2202 is also used to obtain the structural features of the duodenal papilla based on the first segmented image and the second segmented image; the structural features include the papilla length and the type of opening.

[0171] In one embodiment, the first processing module 2202 is further configured to perform depth estimation on the duodenal papilla image to obtain a depth image; determine the maximum inner diameter of the segmented region of the duodenal papilla in the first segmented image, and determine two coordinate points on the segmented region of the duodenal papilla based on the maximum inner diameter; the two coordinate points are used to determine the line segment of the maximum inner diameter of the segmented region of the duodenal papilla; determine the target rectangle based on the two coordinate points, and determine the first side length and the second side length of the target rectangle; obtain the average value of the line segment of the maximum inner diameter in the depth image; determine the pixel focal length of the endoscope, and calculate the papilla length of the duodenal papilla based on the pixel focal length, the first side length and the second side length, and use the papilla length as a structural feature of the duodenal papilla.

[0172] In one embodiment, the first processing module 2202 is further configured to determine the papillary opening region corresponding to the duodenal papilla opening in the second segmented image; identify the shape features of the papillary opening region using a second classification model; determine the opening type of the duodenal papilla opening based on the shape features; and use the opening type as the structural feature of the duodenal papilla.

[0173] In one embodiment, the second processing module 2203 is further configured to obtain the angular features of the endoscope based on the X-ray image; the angular features include the endoscope angle, which characterizes the degree of curvature of the endoscope;

[0174] The second processing module 2203 is also used to obtain the angular features of the endoscope based on the duodenal papilla image; the angular features include the opening angle, which represents the angle between the opening of the duodenal papilla and the endoscope.

[0175] The second processing module 2203 is also used to obtain the angular features of the endoscope based on the X-ray image and the duodenal papilla image; the angular features include the endoscope angle and the opening angle.

[0176] In one embodiment, the second processing module 2203 is further configured to perform image segmentation on the X-ray image using a third segmentation model to obtain a third segmented image of the endoscope; in the third segmented image, determine the endoscope region corresponding to the endoscope and obtain the endoscope centerline of the endoscope region; determine the endoscope angle according to the curvature of the endoscope centerline and use the endoscope angle as the angular feature of the endoscope.

[0177] In one embodiment, the second processing module 2203 is further configured to perform image segmentation on the duodenal papilla image using a second segmentation model to obtain a second segmented image with the duodenal papilla opening segmented out; in the second segmented image, determine the papilla opening region corresponding to the duodenal papilla opening and obtain the papilla opening center point of the papilla opening region; in the duodenal papilla image, determine the terminal center point of the duodenal papilla and determine a first line segment based on the papilla opening center point and the terminal center point; perform image segmentation on the duodenal papilla image using a fourth segmentation model to obtain a fourth segmented image with the duodenal papilla folds segmented out; in the fourth segmented image, determine the folds region corresponding to the folds and determine the horizontal midline of the folds region as a second line segment; determine the opening angle based on the angle between the first line segment and the second line segment, and use the opening angle as the angular feature of the endoscope.

[0178] In one embodiment, the difficulty assessment module 2204 is further configured to determine the scores for each dimension based on the morphological category and structural characteristics of the duodenal papilla and the angular characteristics of the endoscope; obtain a difficulty score based on the weighted sum of the scores for each dimension; and obtain a difficulty assessment result based on the difficulty score.

[0179] Specifically, a first score is determined based on the morphological category; if the morphological category is a normal nipple, the first score has the lowest value. A second score is determined based on the structural features; if the structural features conform to a preset standard structure, the second score has the lowest value. A third score is determined based on the endoscopic angle; if the endoscopic angle is within a preset angle range, the third score has the lowest value. A fourth score is determined based on the opening angle; if the opening angle is 90°, the fourth score has the lowest value. The first, second, third, and fourth scores are weighted and summed according to preset weighting coefficients to calculate the difficulty score, which is used as the difficulty assessment result. The magnitude of the difficulty score is used to characterize the difficulty of image cannulation of the target object.

[0180] Each module in the aforementioned image intubation difficulty assessment device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device, or stored in the memory of a computer device as software, so that the processor can call and execute the corresponding operations of each module.

[0181] In one embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be as follows: Figure 23 As shown, this computer device includes a processor, memory, input / output (I / O) interfaces, and a communication interface. The processor, memory, and I / O interfaces are connected via a system bus, and the communication interface is also connected to the system bus via the I / O interfaces. The processor provides computational and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and a database. The internal memory provides the environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The database stores image data. The I / O interfaces are used for exchanging information between the processor and external devices. The communication interface is used for communicating with external terminals via a network connection. When executed by the processor, the computer program implements a method for evaluating the difficulty of image insertion.

[0182] Those skilled in the art will understand that Figure 23 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0183] In one embodiment, a computer device is also provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps in the above method embodiments.

[0184] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon that, when executed by a processor, implements the steps in the above method embodiments.

[0185] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps in the above method embodiments.

[0186] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of the relevant data shall comply with the relevant laws, regulations and standards of the relevant countries and regions.

[0187] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited to these.

[0188] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0189] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A method for assessing the difficulty of image cannulation, characterized in that, The method includes: Images of the duodenal papillae of the target object were obtained using an endoscope; Based on the duodenal papilla image, the morphological category and structural features of the duodenal papilla of the target object are obtained; the structural features include at least one of the papilla length or the opening type of the duodenal papilla. Obtain an X-ray image of the endoscope, and based on at least one of the X-ray image and the duodenal papilla image, obtain the angular features of the endoscope; including any one of the following: The first method involves obtaining the angular features of the endoscope based on the X-ray image; the angular features include the endoscope angle, which characterizes the degree of curvature of the endoscope. The second method involves obtaining the angular features of the endoscope based on the duodenal papilla image; the angular features include the opening angle, which represents the angle between the duodenal papilla opening and the endoscope. The third method involves obtaining the angular features of the endoscope based on the X-ray image and the duodenal papilla image; the angular features include the endoscope angle and the opening angle. Based on the morphological type and structural characteristics of the duodenal papilla, and the angular characteristics of the endoscope, the difficulty of image cannulation of the target object is determined, and a difficulty assessment result is obtained; including: The scores for each dimension are determined based on the morphological type and structural characteristics of the duodenal papilla and the angular characteristics of the endoscope. The difficulty level score is obtained by weighting the scores of each dimension. The difficulty assessment result is obtained based on the difficulty level score. A doctor mapping table is constructed corresponding to different difficulty level scores. The doctor mapping table records the association information between each doctor and different difficulty level scores, and is used to match the doctor to perform the intubation operation from the doctor mapping table according to the difficulty level score.

2. The method according to claim 1, characterized in that, The step of obtaining the morphological category and structural features of the duodenal papilla of the target object based on the duodenal papilla image includes: A first classification model is used to identify the morphological features in the duodenal papilla image, and the morphological category of the duodenal papilla of the target object is determined based on the morphological features. The first segmentation model is used to segment the duodenal papilla image to obtain a first segmented image of the duodenal papilla. The second segmentation model is used to segment the duodenal papilla image to obtain a second segmented image that segments out the opening of the duodenal papilla. The structural features of the duodenal papilla are obtained based on at least one of the first segmented image and the second segmented image.

3. The method according to claim 2, characterized in that, The step of using a first classification model to identify morphological features in the duodenal papilla image and determining the morphological category of the duodenal papilla of the target object based on the morphological features includes: The duodenal papilla image is preprocessed; the preprocessing includes at least one of data format conversion, normalization, image binarization, data augmentation, flipping, translation, or rotation. Region of interest identification was performed on the preprocessed duodenal papilla image to extract the contour containing the duodenal papilla region; Based on the morphology of the duodenal papilla, feature recognition is performed on the contour of the duodenal papilla region to obtain the morphological features of the duodenal papilla, and the morphological category of the duodenal papilla of the target object is determined according to the morphological features.

4. The method according to claim 2, characterized in that, The step of obtaining the structural features of the duodenal papilla based on at least one of the first segmented image and the second segmented image includes any one of the following: The first type: Based on the first segmented image, the structural features of the duodenal papilla are obtained; the structural features include the papilla length of the duodenal papilla; The second type: Based on the second segmented image, the structural features of the duodenal papilla are obtained; the structural features include the opening type of the duodenal papilla opening; The third type: Based on the first segmented image and the second segmented image, the structural features of the duodenal papilla are obtained; the structural features include the papilla length and the opening type.

5. The method according to claim 4, characterized in that, The structural features of the duodenal papilla include papilla length; obtaining the structural features of the duodenal papilla based on the first segmented image includes: Depth estimation is performed on the duodenal papilla image to obtain a depth image; In the first segmented image, the maximum inner diameter of the duodenal papilla segmented region is determined, and two coordinate points are determined on the duodenal papilla segmented region based on the maximum inner diameter; the two coordinate points are used to determine the maximum inner diameter line segment of the duodenal papilla segmented region. The target rectangle is determined based on the two coordinate points, and the first and second side lengths of the target rectangle are determined. Obtain the average value of the maximum inner diameter line segment in the depth image; The pixel focal length of the endoscope is determined, and the papillary length of the duodenal papilla is calculated based on the pixel focal length, the first side length, and the second side length.

6. The method according to claim 4, characterized in that, The structural features include the type of opening of the duodenal papilla; The step of obtaining the structural features of the duodenal papilla based on the second segmented image includes: In the second segmented image, the papillary opening region corresponding to the duodenal papillary opening is determined; A second classification model is used to identify the shape features of the duodenal papilla opening region, and the opening type of the duodenal papilla opening is determined based on the shape features.

7. The method according to claim 1, characterized in that, The angular features include the endoscope angle; obtaining the angular features of the endoscope based on the X-ray image includes: The X-ray image is segmented using a third segmentation model to obtain a third segmented image of the endoscope. In the third segmented image, the endoscope region corresponding to the endoscope is determined, and the endoscope centerline of the endoscope region is obtained; The included angle of the endoscope is determined based on the curvature of the endoscope's centerline.

8. The method according to claim 1, characterized in that, The angular features include the opening angle; obtaining the angular features of the endoscope based on the duodenal papilla image includes: The second segmentation model is used to segment the duodenal papilla image to obtain a second segmented image that segments out the opening of the duodenal papilla. In the second segmented image, the papillary opening region corresponding to the duodenal papilla opening is determined, and the center point of the papillary opening in the papillary opening region is obtained; In the duodenal papilla image, the center point of the distal end of the duodenal papilla is determined, and a first line segment is determined based on the center point of the papilla opening and the center point of the distal end; The duodenal papilla image is segmented using the fourth segmentation model to obtain a fourth segmentation image that segments the duodenal papilla folds. In the fourth segmented image, the region corresponding to the wrapped wrinkle wall is determined, and the horizontal midline of the wrapped wrinkle wall region is determined as the second line segment; The opening angle is determined based on the included angle between the first line segment and the second line segment.

9. The method according to claim 1, characterized in that, The angular characteristics of the endoscope include the endoscope angle and the opening angle; the scoring of each dimension is determined based on the morphological category and structural characteristics of the duodenal papilla, as well as the angular characteristics of the endoscope, including: A first score is determined based on the morphological category of the duodenal papilla; if the morphological category is normal papilla, the value of the first score is equal to 0. A second score is determined based on the structural characteristics of the duodenal papilla; if the structural characteristics conform to a preset standard structure, the value of the second score is equal to 0. A third score is determined based on the endoscopic angle of the duodenal papilla; if the endoscopic angle is within a preset angle range, the value of the third score is equal to 0. A fourth score is determined based on the opening angle of the duodenal papilla; if the opening angle is equal to 90°, the value of the fourth score is equal to 0.

10. An image cannulation difficulty assessment device, characterized in that, The device includes: The image acquisition module is used to acquire images of the duodenal papilla of the target object through an endoscope; The first processing module is used to obtain the morphological category and structural features of the duodenal papilla of the target object based on the duodenal papilla image; the structural features include at least one of the papilla length or the opening type of the duodenal papilla. The second processing module is used to acquire the X-ray image of the endoscope, and to acquire the angular features of the endoscope based on at least one of the X-ray image and the duodenal papilla image; including any one of the following: The first method involves obtaining the angular features of the endoscope based on the X-ray image; the angular features include the endoscope angle, which characterizes the degree of curvature of the endoscope. The second method involves obtaining the angular features of the endoscope based on the duodenal papilla image; the angular features include the opening angle, which represents the angle between the duodenal papilla opening and the endoscope. The third method involves obtaining the angular features of the endoscope based on the X-ray image and the duodenal papilla image; the angular features include the endoscope angle and the opening angle. The difficulty assessment module is used to determine the difficulty of image cannulation of the target object based on the morphological type and structural characteristics of the duodenal papilla and the angular characteristics of the endoscope, and to obtain a difficulty assessment result; including: The scores for each dimension are determined based on the morphological type and structural characteristics of the duodenal papilla and the angular characteristics of the endoscope. The difficulty level score is obtained by weighting the scores of each dimension. The difficulty assessment result is obtained based on the difficulty level score. A doctor mapping table is constructed corresponding to different difficulty level scores. The doctor mapping table records the association information between each doctor and different difficulty level scores, and is used to match the doctor to perform the intubation operation from the doctor mapping table according to the difficulty level score.

Images