Surgical ligation auxiliary method, device and related equipment

Abstract

The present application provides a surgical ligation assistance method, apparatus, and related equipment. The method comprises identifying a target abnormal object in a pre-acquired endoscopic image, obtaining a detection frame for marking the location of the target abnormal object and the position information of the detection frame; obtaining the position characteristics, epithelial attribute characteristics, venous attribute characteristics, and lymph node attribute characteristics of each target pixel point in the endoscopic image, where the target pixel point is a pixel point within a target area, which includes an area outside the detection frame; determining a dataset of target type objects included in the endoscopic image based on the position characteristics, epithelial attribute characteristics, venous attribute characteristics, and lymph node attribute characteristics of each target pixel point in the endoscopic image; and determining an auxiliary strategy for surgical ligation of the target abnormal object based on the position information of the detection frame and the dataset of target type objects. Thus, the intelligent level of surgical ligation assistance is improved, and the misdiagnosis rate and subsequent complications of patients are reduced.

Description

Technical Field

[0001] The present application relates to the field of auxiliary medical technology, and specifically to a surgical ligation auxiliary method, device and related equipment. Background Art

[0002] Anorectoscopy is an endoscope used to examine the rectum, also known as an anoscope or proctoscopy. Anorectal endoscopy is a routine examination method for anorectal diseases. It is suitable for lesions in the anal canal, the terminal rectum, and near the dentate line. It can also perform biopsy.

[0003] The dentate line of the anus is an important anatomical structure around the anus. The area above the anus develops from the endoderm, while the area below the anus develops from the ectoderm. It is an important dividing line. Clinically, target abnormalities located above the dentate line are usually referred to as internal abnormalities, those below the dentate line as external abnormalities, and those with both abnormalities as mixed abnormalities. For example, hemorrhoids located above the dentate line are called internal hemorrhoids, those below the dentate line as external hemorrhoids, and those with both abnormalities as mixed hemorrhoids. The inventors of the present application have found that it is difficult for doctors to accurately determine the specific location of the target abnormality at the dentate line during surgical ligation, which can easily cause serious pain and harm to patients.

[0004] Therefore, in order to effectively avoid the occurrence of the above problems, how to improve the intelligence of surgical ligation assistance is a technical problem that urgently needs to be solved in the current field of auxiliary medical technology. Summary of the Invention

[0005] The present application provides a surgical ligation assistance method, device and related equipment, aiming to solve the problem of how to effectively improve the intelligence of surgical ligation assistance.

[0006] In one aspect, the present application provides a surgical ligation assist method, comprising:

[0007] Identifying a target abnormal object in a pre-acquired endoscopic image, obtaining a detection frame for marking a location of the target abnormal object and position information of the detection frame, wherein the endoscopic image is an endoscopic image taken during an anal endoscopic examination of a patient;

[0008] Acquiring positional features, epithelial attribute features, venous attribute features, and lymph node attribute features of each target pixel in the endoscopic image, wherein the target pixel is a pixel within a target area, and the target area includes an area excluding a detection frame;

[0009] Determining a data set of target type objects included in the endoscopic image based on position features, epithelial attribute features, vein attribute features, and lymph node attribute features of each target pixel point in the endoscopic image;

[0010] An auxiliary strategy for performing surgical ligation on the target abnormal object is determined based on the position information of the detection frame and the data set of the target type object.

[0011] In a possible implementation of the present application, determining an auxiliary strategy for surgical ligation of the target abnormal object based on the position information of the detection frame and the dataset of the target type object includes:

[0012] If the data set of the target type object only includes the dentate line anal canal, determining that the auxiliary strategy for performing surgical ligation on the target abnormal object is to prompt the physician not to recommend surgical ligation;

[0013] If the dataset of the target type object only includes the dentate line rectum, determining that the auxiliary strategy for performing surgical ligation on the target abnormal object is to prompt the physician to perform the ligation normally;

[0014] If the data set of the target type object includes both the dentate line anal canal and the dentate line rectum, obtaining a boundary line between the dentate line anal canal and the dentate line rectum;

[0015] An auxiliary strategy for surgical ligation of the target abnormal object is determined based on the position information of the detection frame and the boundary line.

[0016] In a possible implementation of the present application, determining an auxiliary strategy for surgical ligation of the target abnormal object based on the position information of the detection frame and the boundary line includes:

[0017] Performing image transformation processing on the boundary line to obtain a surgical warning area of a target shape;

[0018] Determining a positional relationship between the target abnormal object and the surgery warning area based on the position information of the detection frame and the surgery warning area;

[0019] If the positional relationship is that the target abnormal object and the surgical warning area have an intersection relationship, determining that the auxiliary strategy for performing surgical ligation on the target abnormal object is to prompt the physician not to recommend surgical ligation;

[0020] If the positional relationship is that the target abnormal object and the surgical warning area are separated, the auxiliary strategy for performing surgical ligation on the target abnormal object is to prompt the physician to perform the ligation normally.

[0021] In one possible implementation of the present application, obtaining the position features of each target pixel in the endoscopic image includes:

[0022] Obtaining the endoscope body outline in the endoscope image;

[0023] Obtaining a center line of the mirror body contour and two intersection points of the center line and the mirror body contour;

[0024] determining the position of the anus based on the distance from each of the intersection points to the center point of the endoscopic image;

[0025] Based on the Euclidean distance from each target pixel point in the endoscopic image to the position of the anus, the position feature of each target pixel point is determined.

[0026] In a possible implementation of the present application, determining the position of the anus based on the distance from each of the intersection points to the center point of the endoscopic image includes:

[0027] respectively obtaining the distance from each intersection point to the center point of the endoscopic image;

[0028] The distances between each intersection point and the center point of the endoscopic image are compared, and the intersection point with the shorter distance to the center point of the endoscopic image is selected as the anus position.

[0029] In one possible implementation of the present application, determining a dataset of target type objects included in the endoscopic image based on positional features, epithelial attribute features, venous attribute features, and lymph node attribute features of each target pixel point in the endoscopic image includes:

[0030] Performing weighted fitting on the position features, epithelial attribute features, venous attribute features, and lymph node attribute features of each target pixel in the endoscopic image to obtain target type object parameters;

[0031] A data set of target type objects included in the endoscopic image is determined based on the target type object parameter and a preset target type object parameter threshold.

[0032] In a possible implementation of the present application, identifying a target abnormal object in a pre-acquired endoscopic image and obtaining a detection frame for marking a location of the target abnormal object and position information of the detection frame include:

[0033] Based on a pre-trained target abnormal object recognition model, a target abnormal object in a pre-acquired endoscopic image is identified, and a detection frame for marking a position of the target abnormal object and position information of the detection frame are obtained.

[0034] On the other hand, the present application provides a surgical ligation assist device, comprising:

[0035] a first recognition unit, configured to recognize a target abnormal object in a pre-acquired endoscopic image, and obtain a detection frame for marking a location of the target abnormal object and position information of the detection frame, wherein the endoscopic image is an endoscopic image taken during an anal endoscopic examination of a patient;

[0036] a first acquisition unit, configured to acquire position features, epithelial attribute features, venous attribute features, and lymph node attribute features of each target pixel point in the endoscopic image, wherein the target pixel point is a pixel point within a target area, and the target area includes an area excluding a detection frame;

[0037] a first determining unit, configured to determine a data set of target type objects included in the endoscopic image based on position features, epithelial attribute features, venous attribute features, and lymph node attribute features of each target pixel point in the endoscopic image;

[0038] The second determining unit is configured to determine an auxiliary strategy for performing surgical ligation on the target abnormal object based on the position information of the detection frame and the data set of the target type object.

[0039] In a possible implementation of the present application, the second determining unit specifically includes:

[0040] a third determining unit, configured to determine, if the data set of the target type object only includes the dentate line anal canal, that the auxiliary strategy for performing surgical ligation on the target abnormal object is to prompt the physician not to recommend surgical ligation;

[0041] a fourth determining unit, configured to determine, if the data set of the target type object only includes the dentate line rectum, that the auxiliary strategy for performing surgical ligation on the target abnormal object is to prompt the physician to perform normal surgical ligation;

[0042] a second acquiring unit, configured to acquire a boundary line between the dentate line anal canal and the dentate line rectum if the dataset of the target type object includes both the dentate line anal canal and the dentate line rectum;

[0043] A fifth determining unit is configured to determine an auxiliary strategy for performing surgical ligation on the target abnormal object based on the position information of the detection frame and the boundary line.

[0044] In a possible implementation of the present application, the fifth determining unit is specifically configured to:

[0045] Performing image transformation processing on the boundary line to obtain a surgical warning area of a target shape;

[0046] Determining a positional relationship between the target abnormal object and the surgery warning area based on the position information of the detection frame and the surgery warning area;

[0047] If the positional relationship is that the target abnormal object and the surgical warning area have an intersection relationship, determining that the auxiliary strategy for performing surgical ligation on the target abnormal object is to prompt the physician not to recommend surgical ligation;

[0048] If the positional relationship is that the target abnormal object and the surgical warning area are separated, the auxiliary strategy for performing surgical ligation on the target abnormal object is to prompt the physician to perform the ligation normally.

[0049] In a possible implementation of the present application, the first acquiring unit specifically includes:

[0050] a third acquiring unit, configured to acquire a body contour of the endoscope in the endoscope image;

[0051] a fourth acquiring unit, configured to acquire a center line of the lens body contour and two intersection points between the center line and the lens body contour;

[0052] a sixth determining unit, configured to determine the position of the anus based on the distances from each of the intersection points to the center point of the endoscopic image;

[0053] The seventh determining unit is configured to determine a position feature of each target pixel point based on a Euclidean distance from each target pixel point in the endoscopic image to the position of the anus.

[0054] In a possible implementation of the present application, the sixth determining unit is specifically configured to:

[0055] respectively obtaining the distance from each intersection point to the center point of the endoscopic image;

[0056] The distances between each intersection point and the center point of the endoscopic image are compared, and the intersection point with the shorter distance to the center point of the endoscopic image is selected as the anus position.

[0057] In a possible implementation of the present application, the first determining unit is specifically configured to:

[0058] Performing weighted fitting on the position features, epithelial attribute features, venous attribute features, and lymph node attribute features of each target pixel in the endoscopic image to obtain target type object parameters;

[0059] A data set of target type objects included in the endoscopic image is determined based on the target type object parameter and a preset target type object parameter threshold.

[0060] In a possible implementation of the present application, the first identification unit is specifically configured to:

[0061] Based on a pre-trained target abnormal object recognition model, a target abnormal object in a pre-acquired endoscopic image is identified, and a detection frame for marking a position of the target abnormal object and position information of the detection frame are obtained.

[0062] On the other hand, the present application further provides a computer device, comprising:

[0063] one or more processors;

[0064] Memory; and

[0065] One or more application programs, wherein the one or more application programs are stored in the memory and configured to be executed by the processor to implement the surgical ligation assistance method.

[0066] On the other hand, the present application also provides a computer-readable storage medium having a computer program stored thereon, and the computer program is loaded by a processor to execute the steps in the surgical ligation assisting method.

[0067] The surgical ligation assistance method provided in an embodiment of the present application obtains a detection frame for marking the position of the target abnormal object and the position information of the detection frame by identifying a target abnormal object in a pre-acquired endoscopic image, wherein the endoscopic image is an endoscopic image taken during an anal endoscopic examination of a patient; obtains the position characteristics, epithelial attribute characteristics, venous attribute characteristics and lymph node attribute characteristics of each target pixel point in the endoscopic image, wherein the target pixel point is a pixel point within a target area, and the target area includes an area other than the detection frame; based on the position characteristics, epithelial attribute characteristics, venous attribute characteristics and lymph node attribute characteristics of each target pixel point in the endoscopic image, determines a data set of target type objects included in the endoscopic image; based on the position information of the detection frame and the data set of the target type objects, determines an assistance strategy for surgical ligation of the target abnormal object. Compared with traditional methods, when it is impossible to effectively provide intelligent assistance for surgical ligation of the target type, the present application automatically identifies the target abnormal object and uses the relationship between the position information of its corresponding detection frame and the data set of the target type object included in the identified endoscopic image to intelligently provide an auxiliary strategy for surgical ligation of the target abnormal object, thereby improving the intelligence level of surgical ligation assistance and reducing the patient's misdiagnosis rate and subsequent complications. BRIEF DESCRIPTION OF THE DRAWINGS

[0068] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For those skilled in the art, other drawings can be obtained based on these drawings without creative work.

[0069] Figure 1 This is a schematic diagram of a scenario of a surgical ligation assisting system provided in an embodiment of the present application;

[0070] Figure 2 This is a schematic flow chart of an embodiment of the surgical ligation assistance method provided in the embodiments of the present application;

[0071] Figure 3 is a schematic diagram of an endoscopic image provided in an embodiment of the present application;

[0072] Figure 4 Schematic diagram of the center line and intersection of the mirror body outline provided in the embodiment of the present application;

[0073] Figure 5 is a schematic diagram of a surgical warning area provided in an embodiment of the present application;

[0074] Figure 6 This is a schematic structural diagram of an embodiment of a surgical ligation assisting device provided in an embodiment of the present application;

[0075] Figure 7 It is a schematic diagram of the structure of an embodiment of the computer device provided in the embodiments of the present application. DETAILED DESCRIPTION

[0076] The following will be combined with the drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are only part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without making creative efforts are within the scope of protection of this application.

[0077] In the description of the present application, it should be understood that the terms "center", "longitudinal", "lateral", "length", "width", "thickness", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", "outside" and the like indicate orientations or positional relationships based on the orientations or positional relationships shown in the accompanying drawings, and are only for the convenience of describing the present application and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as limiting the present application. In addition, the terms "first" and "second" are used for descriptive purposes only and cannot be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "first" and "second" may explicitly or implicitly include one or more of the said features. In the description of the present application, "multiple" means two or more, unless otherwise clearly and specifically defined.

[0078] In this application, the word "exemplary" is used to mean "serving as an example, illustration, or illustration." Any embodiment described in this application as "exemplary" is not necessarily to be construed as preferred or advantageous over other embodiments. The following description is given to enable any person skilled in the art to implement and use the present application. In the following description, details are listed for the purpose of explanation. It should be understood that one of ordinary skill in the art can recognize that the present application can be implemented without using these specific details. In other instances, well-known structures and processes are not elaborated in detail to avoid obscuring the description of the present application with unnecessary details. Therefore, the present application is not intended to be limited to the embodiments shown, but is consistent with the widest scope consistent with the principles and features disclosed in this application.

[0079] The embodiments of the present application provide a surgical ligation auxiliary method, apparatus, and related equipment, which are described in detail below.

[0080] like Figure 1 As shown, Figure 1 Schematic diagram of a surgical ligation assisting system according to an embodiment of the present invention. The surgical ligation assisting system may include a computer device 100, in which a surgical ligation assisting device is integrated. Figure 1 The computer device 100 in FIG.

[0081] In the embodiment of the present application, the computer device 100 is mainly used to identify a target abnormal object in a pre-acquired endoscopic image, obtain a detection frame for marking the position of the target abnormal object and the position information of the detection frame, wherein the endoscopic image is an endoscopic image taken when a patient undergoes an anal endoscopic examination; obtain the position characteristics, epithelial attribute characteristics, venous attribute characteristics and lymph node attribute characteristics of each target pixel point in the endoscopic image, wherein the target pixel point is a pixel point within a target area, and the target area includes an area other than the detection frame; based on the position characteristics, epithelial attribute characteristics, venous attribute characteristics and lymph node attribute characteristics of each target pixel point in the endoscopic image, determine a data set of target type objects included in the endoscopic image; based on the position information of the detection frame and the data set of the target type object, determine an auxiliary strategy for surgical ligation of the target abnormal object.

[0082] In the embodiments of the present application, the computer device 100 may be a terminal or a server. When the computer device 100 is a server, it may be an independent server or a server network or server cluster composed of servers. For example, the computer device 100 described in the embodiments of the present application includes but is not limited to a computer, a network host, a single network server, a set of multiple network servers, or a cloud server constructed by multiple servers. The cloud server is constructed by a large number of computers or network servers based on cloud computing.

[0083] It is understood that in the embodiments of the present application, when the computer device 100 is a terminal, the terminal used can be a device that includes both receiving and transmitting hardware, that is, a device that has receiving and transmitting hardware capable of performing two-way communication over a two-way communication link. Such devices may include: cellular or other communication devices that have a single-line display, a multi-line display, or a cellular or other communication device without a multi-line display. The specific computer device 100 can be a desktop terminal or a mobile terminal. The computer device 100 can also be a mobile phone, a tablet computer, a laptop computer, a medical auxiliary instrument, etc.

[0084] Those skilled in the art will understand that Figure 1 The application environment shown in the figure is only one application scenario of the present application solution and is not intended to limit the application scenario of the present application solution. Other application environments may also include Figure 1 More or fewer computer devices as shown in Figure 1 Only one computer device is shown in the figure. It can be understood that the surgical ligation auxiliary system can also include one or more other computer devices, which are not limited here.

[0085] In addition, if Figure 1As shown, the surgical ligation assistance system may further include a memory 200 for storing data, such as endoscopic images taken during anal endoscopic examination of the patient and surgical ligation assistance data, such as surgical ligation assistance data when the surgical ligation assistance system is running.

[0086] It should be noted that Figure 1 The scenario diagram of the surgical ligation assist system shown is merely an example. The surgical ligation assist system and scenario described in the embodiment of the present application are intended to more clearly illustrate the technical solution of the embodiment of the present application, and do not constitute a limitation on the technical solution provided in the embodiment of the present application. A person skilled in the art will appreciate that, with the evolution of the surgical ligation assist system and the emergence of new business scenarios, the technical solution provided in the embodiment of the present application is equally applicable to similar technical problems.

[0087] Next, the surgical ligation assisting method provided in the embodiment of the present application is introduced.

[0088] In the embodiment of the surgical ligation assistance method of the present invention, a surgical ligation assistance device is used as the execution subject. For the sake of simplicity and ease of description, the execution subject will be omitted in the subsequent method embodiments. The surgical ligation assistance device is applied to a computer device. The method includes: identifying a target abnormal object in a pre-acquired endoscopic image, obtaining a detection frame for marking the position of the target abnormal object and the position information of the detection frame, wherein the endoscopic image is an endoscopic image taken when the patient undergoes an anal endoscopic examination; obtaining the position characteristics, epithelial attribute characteristics, venous attribute characteristics and lymph node attribute characteristics of each target pixel point in the endoscopic image, wherein the target pixel point is a pixel point within a target area, and the target area includes an area other than the detection frame; based on the position characteristics, epithelial attribute characteristics, venous attribute characteristics and lymph node attribute characteristics of each target pixel point in the endoscopic image, determining a data set of target type objects included in the endoscopic image; and determining an auxiliary strategy for surgical ligation of the target abnormal object based on the position information of the detection frame and the data set of the target type object.

[0089] See also Figures 2 to 7 , Figure 2 This is a flow chart of an embodiment of a surgical ligation assistance method provided in an embodiment of the present application. The surgical ligation assistance method includes steps 201 to 204:

[0090] 201. Identify a target abnormal object in a pre-acquired endoscopic image, and obtain a detection frame for marking a location of the target abnormal object and position information of the detection frame.

[0091] Among them, the endoscopic image is an endoscopic image taken when the patient undergoes an anal endoscopic examination. The target abnormal object is an abnormal existence in the area to be inspected, such as polyps, hemorrhoids or other foreign bodies. This application will use hemorrhoids as an example for the target abnormal object.

[0092] It should be noted that before performing step 201, the video captured by the anal endoscopy equipment needs to be preprocessed to obtain its endoscopic image. Specifically, the video can be first decoded into a first target image in RGB format, and then the obtained first target image can be resized to a preset target size. The target size in this application can be set according to actual needs, preferably 640*640. In a specific embodiment, if the size of the first target image is too large, the first target image can be cropped. If the size of the first target image is too small, the first target image can be padded.

[0093] In an embodiment of the present application, specifically based on a pre-trained target abnormal object recognition model, the target abnormal object in the pre-acquired endoscopic image can be identified, and a detection frame for marking the location of the target abnormal object and the position information of the detection frame can be obtained, wherein the target abnormal object recognition model preferably adopts a yolov7 network structure.

[0094] The embodiment of the present application adopts a pre-trained target abnormal object recognition model based on the YOLOv7 network structure, which can efficiently and accurately identify target abnormal objects in pre-acquired endoscopic images, thereby improving the overall recognition efficiency and accuracy.

[0095] 202. Obtain position features, epithelial attribute features, venous attribute features, and lymph node attribute features of each target pixel in the endoscopic image.

[0096] The target pixel point is a pixel point within a target area, and the target area includes an area excluding the detection frame.

[0097] The present application can obtain the position characteristics, epithelial attribute characteristics, venous attribute characteristics and lymph node attribute characteristics of each target pixel point in the endoscopic image in sequence or simultaneously through a variety of methods, wherein the position characteristics of each target pixel point in the endoscopic image refer to the specific position characteristics of the human body part actually corresponding to each target pixel point in the endoscopic image, for example, it can be the dentate line rectum, the dentate line anal canal, or it can include the dentate line rectum, the dentate line anal canal and the dentate line at the same time; and the epithelial attribute characteristics refer to the epithelial attribute characteristics corresponding to the human body part actually corresponding to each target pixel point in the endoscopic image, for example, the epithelial attribute characteristics can be simple columnar epithelium or stratified squamous epithelium; similarly, the venous attribute characteristics can include the portal vein and the inferior vena cava; the lymph node attribute characteristics include the inferior mesenteric A lymph nodes and the superficial inguinal lymph nodes.

[0098] Exemplarily, in an embodiment of the present application, obtaining the position features of each target pixel in the endoscopic image may specifically include steps A1 to A4:

[0099] A1. Obtaining the endoscope body outline in the endoscope image;

[0100] Among them, the mirror body outline is the outline of the mirror body of the anal endoscope examination device, as follows Figure 3 As shown, the shooting angle corresponding to the image is taken in reverse by the anal endoscope inspection device located in the body. The black stick in the upper right area of the figure is the body of the anal endoscope inspection device. Specifically, the embodiment of the present application can segment the endoscopic image through a pre-trained body segmentation model to obtain a segmented image, which is a binary image of the body. Then, by scanning all pixels of the binary image of the body, the outline of the body is extracted, thereby obtaining the outline of the body in the endoscopic image. The body segmentation model is preferably a Unet++ network. The specific method for extracting the outline of the body is as follows:

[0101]

[0102] where U(i,8) is the eight-neighborhood centered at point i.

[0103] A2. Obtaining a center line of the lens body contour and two intersection points between the center line and the lens body contour;

[0104] Among them, the following Figure 4 As shown, Figure 4 There is a center line that divides the mirror body outline, and two intersection points between the center line and the mirror body outline.

[0105] In some embodiments of the present application, a straight line closest to all points on the contour of the lens body can be fitted by the least squares method, which is the center line of the lens body contour. Then, extend this center line to intersect with the lens body contour to obtain two intersection points.

[0106] A3. Determine the position of the anus based on the distances from each of the intersection points to the center point of the endoscopic image;

[0107] Among them, the shooting perspective corresponding to the following figure mentioned above indicates that the anorectal endoscopy device is already located inside the human body. And because it is a reverse shooting, the lens body can be seen. In addition, the anorectal endoscopy device is inserted into the human anus. Therefore, Figure 4 the position of the anus may exist in the following.

[0108] In the embodiments of the present application, the determining the position of the anus based on the distances from each of the intersection points to the center point of the endoscopic image may specifically include B1 and B2:

[0109] B1. Obtain the distances from each of the intersection points to the center point of the endoscopic image respectively;

[0110] Among them, the center point of the endoscopic image is the center point of the entire endoscopic image. For example, assuming that the endoscopic image is a regular circular image, then the center point of the endoscopic image is its center of the circle. If the endoscopic image is a rectangular image, then the center point of the endoscopic image is the intersection point of its diagonals. The center point can be pre-marked in the image and its coordinates can be marked.

[0111] In a specific embodiment, assuming that the coordinates of the center point of the endoscopic image are (x0, y0), and the coordinates of its two intersection points are (x1, y1) and (x2, y2) respectively, then the Euclidean distances d1 and d2 from each intersection point to the center point can be calculated through their coordinates.

[0112] B2. Compare the lengths of the distances from each of the intersection points to the center point of the endoscopic image, and select the intersection point with a shorter distance to the center point of the endoscopic image as the position of the anus.

[0113] Based on the example in step B1 above, if d1 < d2, then set the coordinates of the position of the anus (x g , y g ) = (x1, y1). If d1 > d2, then set the coordinates of the position of the anus (x g , y g ) = (x2, y2).

[0114] A4. Determine the position characteristics of each target pixel point based on the Euclidean distance from each target pixel point in the endoscopic image to the position of the anus.

[0115] Among them, we can first count the pixels (x n ,y n ) to the coordinates of the anus (x g ,y g ), and then determine the position feature ρ of each pixel point by the following formula: s :

[0116]

[0117] For example, in the embodiment of the present application, the epithelial attribute feature ρ of each target pixel in the endoscopic image can be identified by a pre-trained epithelial feature classification model. e , where the model labels are 0-simple columnar epithelium and 1-stratified squamous epithelium.

[0118] For example, in the embodiment of the present application, the vein attribute feature ρ of each target pixel in the endoscopic image can be identified by a pre-trained vein feature classification model. v , where the model labels are 0-portal vein and 1-inferior vena cava.

[0119] For example, in the embodiment of the present application, the lymph node attribute feature ρ of each target pixel in the endoscopic image can be identified by a pre-trained lymph node feature classification model. l , where the model labels are 0-inferior mesenteric A lymph nodes and 1-superficial inguinal lymph nodes.

[0120] 203. Determine a data set of target type objects included in the endoscopic image based on the position features, epithelial attribute features, vein attribute features, and lymph node attribute features of each target pixel point in the endoscopic image.

[0121] In an embodiment of the present application, determining a dataset of target type objects included in the endoscopic image based on the positional features, epithelial attribute features, venous attribute features, and lymph node attribute features of each target pixel point in the endoscopic image includes steps C1 and C2:

[0122] C1. performing weighted fitting on the position features, epithelial attribute features, venous attribute features, and lymph node attribute features of each target pixel in the endoscopic image to obtain target type object parameters;

[0123] In a specific embodiment, the target type object parameter P is calculated as shown in the following formula:

[0124] P=ω1ρ s +ω2ρ e +ω3ρ v +ω4ρ l ;

[0125] Among them, ω1, ω2, ω3, and ω4 are weights trained by the preset machine learning algorithm, and ρ s , ρ e , ρ v , ρ l These are the location features, epithelial attribute features, venous attribute features, and lymph node attribute features of each target pixel point.

[0126] C2. Determine a data set of target type objects included in the endoscopic image based on the target type object parameter and a preset target type object parameter threshold.

[0127] Among them, the target type object parameter threshold can be set according to actual needs and is not specifically limited.

[0128] In the embodiment of the present application, assuming that the target type object parameter threshold may be P, and the target type object parameter threshold is P0, when P is greater than P0, it is determined that the dentate line anal canal exists, otherwise, it is determined that the dentate line rectum exists.

[0129] Specifically, if the target type object parameter P corresponding to each target pixel point is greater than the target type object parameter threshold P0, it is determined that the data set of target type objects included in the endoscopic image only includes the dentate line anal canal; if the target type object parameter P corresponding to each target pixel point is less than the target type object parameter threshold P0, it is determined that the data set of target type objects included in the endoscopic image only includes the dentate line rectum; if the target type object parameter P corresponding to a part of the target pixel points is greater than the target type object parameter threshold P0, and the target type object parameter P corresponding to another part of the target pixel points is less than the target type object parameter threshold P0, it is determined that the data set of target type objects included in the endoscopic image simultaneously includes the dentate line anal canal, the dentate line, and the dentate line rectum.

[0130] 204. Based on the position information of the detection frame and the dataset of the target type object, determine an auxiliary strategy for surgical ligation of the target abnormal object.

[0131] In some embodiments of the present application, determining an auxiliary strategy for surgical ligation of the target abnormal object based on the position information of the detection frame and the dataset of the target type object includes steps D1 to D4:

[0132] D1. If the dataset of the target type object only includes the dentate line anal canal, determining that the auxiliary strategy for performing surgical ligation on the target abnormal object is to prompt the physician not to recommend surgical ligation;

[0133] Specifically, after surgical ligation is not recommended, the physician may also be advised to adopt other appropriate methods for treating the target abnormality in the dentate line anal canal region.

[0134] D2. If the dataset of the target type object only includes the dentate line rectum, determining that the auxiliary strategy for performing surgical ligation on the target abnormal object is to prompt the physician to perform the ligation normally;

[0135] D3. If the dataset of the target type object includes both the dentate line anal canal and the dentate line rectum, obtaining a boundary line between the dentate line anal canal and the dentate line rectum;

[0136] D4. Determine an auxiliary strategy for surgical ligation of the target abnormal object based on the position information of the detection frame and the boundary line.

[0137] The embodiment of the present application, through the above-mentioned disclosed solution, provides different auxiliary strategies for different parts of the target abnormal object, thereby improving the humanization and intelligence of the solution.

[0138] In the embodiment of the present application, the auxiliary strategy for performing surgical ligation on the target abnormal object based on the position information of the detection frame and the boundary line includes steps E1 to E4:

[0139] E1. Performing image transformation processing on the boundary line to obtain a surgical warning area of a target shape;

[0140] Specifically, the image transformation process specifically adopts Hough circle transformation to obtain a circular surgical warning area.

[0141] E2. Determine a positional relationship between the target abnormal object and the surgical warning area based on the position information of the detection frame and the surgical warning area;

[0142] Specifically, the positional relationship between the target abnormal object and the surgical warning area can be determined by calculating the intersection-over-union ratio of the detection frame and the surgical warning area. The positional relationship between the two generally includes separation and intersection.

[0143] E3. If the positional relationship is that the target abnormal object and the surgical warning area have an intersection relationship, determining that the auxiliary strategy for performing surgical ligation on the target abnormal object is to prompt the physician not to recommend surgical ligation;

[0144] E4. If the positional relationship is that the target abnormal object and the surgical warning area are separated, determining that the auxiliary strategy for performing surgical ligation on the target abnormal object is to prompt the physician to perform the ligation normally.

[0145] Compared with traditional methods, the surgical ligation assistance method disclosed in the embodiments of the present application automatically identifies the target abnormal object and uses the relationship between the position information of its corresponding detection frame and the data set of the target type object included in the identified endoscopic image to intelligently provide an auxiliary strategy for surgical ligation of the target abnormal object, thereby improving the intelligence level of surgical ligation assistance and reducing the patient's misdiagnosis rate and subsequent complications.

[0146] In order to better implement the surgical ligation assisting method in the embodiment of the present application, on the basis of the surgical ligation assisting method, the embodiment of the present application also provides a surgical ligation assisting device, such as Figure 6 As shown, the surgical ligation auxiliary device 600 includes:

[0147] a first recognition unit 601 for recognizing a target abnormal object in a pre-acquired endoscopic image, and obtaining a detection frame for marking a location of the target abnormal object and position information of the detection frame, wherein the endoscopic image is an endoscopic image taken during an anal endoscopic examination of a patient;

[0148] A first acquisition unit 602 is configured to acquire position features, epithelial attribute features, venous attribute features, and lymph node attribute features of each target pixel in the endoscopic image, wherein the target pixel is a pixel within a target area, and the target area includes an area outside a detection frame;

[0149] A first determining unit 603 is configured to determine a dataset of target type objects included in the endoscopic image based on position features, epithelial attribute features, vein attribute features, and lymph node attribute features of each target pixel point in the endoscopic image;

[0150] The second determining unit 604 is configured to determine an auxiliary strategy for performing surgical ligation on the target abnormal object based on the position information of the detection frame and the dataset of the target type object.

[0151] In a possible implementation of the present application, the second determining unit 604 specifically includes:

[0152] a third determining unit, configured to determine, if the data set of the target type object only includes the dentate line anal canal, that the auxiliary strategy for performing surgical ligation on the target abnormal object is to prompt the physician not to recommend surgical ligation;

[0153] a fourth determining unit, configured to determine, if the data set of the target type object only includes the dentate line rectum, that the auxiliary strategy for performing surgical ligation on the target abnormal object is to prompt the physician to perform normal surgical ligation;

[0154] a second acquiring unit, configured to acquire a boundary line between the dentate line anal canal and the dentate line rectum if the dataset of the target type object includes both the dentate line anal canal and the dentate line rectum;

[0155] A fifth determining unit is configured to determine an auxiliary strategy for performing surgical ligation on the target abnormal object based on the position information of the detection frame and the boundary line.

[0156] In a possible implementation of the present application, the fifth determining unit is specifically configured to:

[0157] Performing image transformation processing on the boundary line to obtain a surgical warning area of a target shape;

[0158] Determining a positional relationship between the target abnormal object and the surgery warning area based on the position information of the detection frame and the surgery warning area;

[0159] If the positional relationship is that the target abnormal object and the surgical warning area have an intersection relationship, determining that the auxiliary strategy for performing surgical ligation on the target abnormal object is to prompt the physician not to recommend surgical ligation;

[0160] If the positional relationship is that the target abnormal object and the surgical warning area are separated, the auxiliary strategy for performing surgical ligation on the target abnormal object is to prompt the physician to perform the ligation normally.

[0161] In a possible implementation of the present application, the first acquiring unit 602 specifically includes:

[0162] a third acquiring unit, configured to acquire a body contour of the endoscope in the endoscope image;

[0163] a fourth acquiring unit, configured to acquire a center line of the lens body contour and two intersection points between the center line and the lens body contour;

[0164] a sixth determining unit, configured to determine the position of the anus based on the distances from each of the intersection points to the center point of the endoscopic image;

[0165] The seventh determining unit is configured to determine a position feature of each target pixel point based on a Euclidean distance from each target pixel point in the endoscopic image to the position of the anus.

[0166] In a possible implementation of the present application, the sixth determining unit is specifically configured to:

[0167] respectively obtaining the distance from each intersection point to the center point of the endoscopic image;

[0168] The distances between each intersection point and the center point of the endoscopic image are compared, and the intersection point with the shorter distance to the center point of the endoscopic image is selected as the anus position.

[0169] In a possible implementation of the present application, the first determining unit 603 is specifically configured to:

[0170] Performing weighted fitting on the position features, epithelial attribute features, venous attribute features, and lymph node attribute features of each target pixel in the endoscopic image to obtain target type object parameters;

[0171] A data set of target type objects included in the endoscopic image is determined based on the target type object parameter and a preset target type object parameter threshold.

[0172] In a possible implementation of the present application, the first identification unit 601 is specifically configured to:

[0173] Based on a pre-trained target abnormal object recognition model, a target abnormal object in a pre-acquired endoscopic image is identified, and a detection frame for marking a position of the target abnormal object and position information of the detection frame are obtained.

[0174] The surgical ligation assisting device provided in an embodiment of the present application uses a first recognition unit 601 to identify a target abnormal object in a pre-acquired endoscopic image, and obtain a detection frame for marking the position of the target abnormal object and the position information of the detection frame. The endoscopic image is an endoscopic image taken during an anal endoscopic examination of a patient; a first acquisition unit 602 is used to obtain the position characteristics, epithelial attribute characteristics, venous attribute characteristics, and lymph node attribute characteristics of each target pixel point in the endoscopic image, wherein the target pixel point is a pixel point within a target area, and the target area includes an area other than the detection frame; a first determination unit 603 is used to determine a data set of target type objects included in the endoscopic image based on the position characteristics, epithelial attribute characteristics, venous attribute characteristics, and lymph node attribute characteristics of each target pixel point in the endoscopic image; and a second determination unit 604 is used to determine an auxiliary strategy for surgical ligation of the target abnormal object based on the position information of the detection frame and the data set of the target type object. Compared with traditional devices, when it is impossible to effectively provide intelligent assistance for surgical ligation of the target type, the present application automatically identifies the target abnormal object and uses the relationship between the position information of its corresponding detection frame and the data set of the target type object included in the identified endoscopic image to intelligently provide an auxiliary strategy for surgical ligation of the target abnormal object, thereby improving the intelligence level of surgical ligation assistance and reducing the patient's misdiagnosis rate and subsequent complications.

[0175] In addition to the above-described surgical ligation assisting method and device, the present embodiment further provides a computer device that integrates any of the surgical ligation assisting devices provided in the present embodiment, the computer device comprising:

[0176] one or more processors;

[0177] Memory; and

[0178] One or more applications, wherein the one or more applications are stored in the memory and configured to cause the processor to execute the operations of any method described in any of the above-mentioned surgical ligation assistance method embodiments.

[0179] The present application also provides a computer device that integrates any of the surgical ligation assisting devices provided in the present application. Figure 7 , which shows a schematic diagram of the structure of the computer device involved in the embodiment of the present application, specifically:

[0180] The computer device may include one or more processing core processors 701, one or more computer readable storage medium storage units 702, a power supply 703 and an input unit 704. Those skilled in the art will understand that Figure 7 The computer device structure shown in the figure does not constitute a limitation on the computer device, and may include more or fewer components than shown in the figure, or combine certain components, or arrange components differently.

[0181] Processor 701 is the control center of the computer device. It connects the various components of the entire computer device using various interfaces and lines. By running or executing software programs and / or modules stored in storage unit 702 and accessing data stored in storage unit 702, it performs various functions of the computer device and processes data, thereby providing overall monitoring of the computer device. Optionally, processor 701 may include one or more processing cores; preferably, processor 701 may integrate an application processor and a modem processor, wherein the application processor primarily processes the operating system, user interface, and application programs, while the modem processor primarily handles wireless communications. It is understood that the modem processor may not be integrated into processor 701.

[0182] The storage unit 702 can be used to store software programs and modules. The processor 701 executes various functional applications and data processing by running the software programs and modules stored in the storage unit 702. The storage unit 702 may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc.; the data storage area may store data created according to the use of the computer device, etc. In addition, the storage unit 702 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other volatile solid-state storage device. Accordingly, the storage unit 702 may also include a memory controller to provide the processor 701 with access to the storage unit 702.

[0183] The computer device also includes a power supply 703 for supplying power to various components. Preferably, the power supply 703 can be logically connected to the processor 701 via a power management system, thereby enabling the power management system to manage charging, discharging, and power consumption. The power supply 703 can also include one or more DC or AC power supplies, a recharging system, a power failure detection circuit, a power converter or inverter, a power status indicator, and other arbitrary components.

[0184] The computer device may further include an input unit 704, which may be configured to receive input digital or character information and generate keyboard, mouse, joystick, optical or trackball signal input related to user settings and function control.

[0185] Although not shown, the computer device may further include a display unit, etc., which will not be described in detail here. Specifically, in the embodiment of the present application, the processor 701 in the computer device will load the executable files corresponding to the processes of one or more application programs into the storage unit 702 according to the following instructions, and the processor 701 will run the application programs stored in the storage unit 702, thereby realizing various functions as follows:

[0186] Identify a target abnormal object in a pre-acquired endoscopic image, obtain a detection frame for marking the position of the target abnormal object and position information of the detection frame, wherein the endoscopic image is an endoscopic image taken during an anal endoscopic examination of a patient; obtain position features, epithelial attribute features, venous attribute features, and lymph node attribute features of each target pixel point in the endoscopic image, wherein the target pixel point is a pixel point within a target area, and the target area includes an area other than the detection frame; determine a data set of target type objects included in the endoscopic image based on the position features, epithelial attribute features, venous attribute features, and lymph node attribute features of each target pixel point in the endoscopic image; and determine an auxiliary strategy for surgical ligation of the target abnormal object based on the position information of the detection frame and the data set of the target type object.

[0187] The present application provides a surgical ligation assistance method. Compared with traditional methods, when it is impossible to effectively provide intelligent assistance for surgical ligation of the target type, the present application automatically identifies the target abnormal object and uses the relationship between the position information of its corresponding detection frame and the data set of the target type object included in the identified endoscopic image to intelligently provide an auxiliary strategy for surgical ligation of the target abnormal object, thereby improving the intelligence level of surgical ligation assistance and reducing the patient's misdiagnosis rate and subsequent complications.

[0188] To this end, an embodiment of the present application provides a computer-readable storage medium, which may include: a read-only memory (ROM), a random access memory (RAM), a disk, or an optical disk. The computer-readable storage medium stores multiple instructions that can be loaded by a processor to execute the steps of any of the surgical ligation assistance methods provided in the embodiments of the present application. For example, the instructions may execute the following steps:

[0189] Identify a target abnormal object in a pre-acquired endoscopic image, obtain a detection frame for marking the position of the target abnormal object and position information of the detection frame, wherein the endoscopic image is an endoscopic image taken during an anal endoscopic examination of a patient; obtain position features, epithelial attribute features, venous attribute features, and lymph node attribute features of each target pixel point in the endoscopic image, wherein the target pixel point is a pixel point within a target area, and the target area includes an area other than the detection frame; determine a data set of target type objects included in the endoscopic image based on the position features, epithelial attribute features, venous attribute features, and lymph node attribute features of each target pixel point in the endoscopic image; and determine an auxiliary strategy for surgical ligation of the target abnormal object based on the position information of the detection frame and the data set of the target type object.

[0190] In the above embodiments, the description of each embodiment has its own focus. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0191] The above is a detailed introduction to a surgical ligation auxiliary method, device and related equipment provided in the embodiments of the present application. Specific examples are used herein to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method and core ideas of the present application. At the same time, for those skilled in the art, according to the ideas of the present application, there may be changes in the specific implementation methods and application scopes. In summary, the content of this specification should not be understood as limiting the present application.

Claims

1. A surgical ligation auxiliary method, characterized in that: The method comprises: Identifying a target abnormal object in a pre-acquired endoscopic image, obtaining a detection frame for marking a location of the target abnormal object and position information of the detection frame, wherein the endoscopic image is an endoscopic image taken during an anal endoscopic examination of a patient; Acquiring positional features, epithelial attribute features, venous attribute features, and lymph node attribute features of each target pixel in the endoscopic image, wherein the target pixel is a pixel within a target area, and the target area includes an area excluding a detection frame; Determining a data set of target type objects included in the endoscopic image based on position features, epithelial attribute features, vein attribute features, and lymph node attribute features of each target pixel point in the endoscopic image; An auxiliary strategy for performing surgical ligation on the target abnormal object is determined based on the position information of the detection frame and the data set of the target type object.

2. The surgical ligation assisting method according to claim 1, characterized in that: The determining of an auxiliary strategy for surgical ligation of the target abnormal object based on the position information of the detection frame and the data set of the target type object includes: If the data set of the target type object only includes the dentate line anal canal, determining that the auxiliary strategy for performing surgical ligation on the target abnormal object is to prompt the physician not to recommend surgical ligation; If the dataset of the target type object only includes the dentate line rectum, determining that the auxiliary strategy for performing surgical ligation on the target abnormal object is to prompt the physician to perform the ligation normally; If the data set of the target type object includes both the dentate line anal canal and the dentate line rectum, obtaining a boundary line between the dentate line anal canal and the dentate line rectum; An auxiliary strategy for performing surgical ligation on the target abnormal object is determined based on the position information of the detection frame and the boundary line.

3. The surgical ligation assisting method according to claim 2, characterized in that: The determining of an auxiliary strategy for surgical ligation of the target abnormal object based on the position information of the detection frame and the boundary line includes: Performing image transformation processing on the boundary line to obtain a surgical warning area of a target shape; Determining a positional relationship between the target abnormal object and the surgery warning area based on the position information of the detection frame and the surgery warning area; If the positional relationship is that the target abnormal object and the surgical warning area have an intersection relationship, determining that the auxiliary strategy for performing surgical ligation on the target abnormal object is to prompt the physician not to recommend surgical ligation; If the positional relationship is that the target abnormal object and the surgical warning area are separated, the auxiliary strategy for performing surgical ligation on the target abnormal object is to prompt the physician to perform the ligation normally.

4. The surgical ligation assisting method according to claim 1, characterized in that: Obtaining the positional features of each target pixel in the endoscopic image includes: Obtaining the endoscope body outline in the endoscope image; Obtaining a center line of the mirror body contour and two intersection points of the center line and the mirror body contour; determining the position of the anus based on the distance from each of the intersection points to the center point of the endoscopic image; Based on the Euclidean distance from each target pixel point in the endoscopic image to the position of the anus, the position feature of each target pixel point is determined.

5. The surgical ligation assisting method according to claim 4, characterized in that: The determining the position of the anus based on the distance from each intersection point to the center point of the endoscopic image includes: respectively obtaining the distance from each intersection point to the center point of the endoscopic image; The distances between each intersection point and the center point of the endoscopic image are compared, and the intersection point with the shorter distance to the center point of the endoscopic image is selected as the anus position.

6. The surgical ligation assisting method according to claim 1, characterized in that: The step of determining a dataset of target type objects included in the endoscopic image based on the positional features, epithelial attribute features, venous attribute features, and lymph node attribute features of each target pixel point in the endoscopic image comprises: Performing weighted fitting on the position features, epithelial attribute features, venous attribute features, and lymph node attribute features of each target pixel in the endoscopic image to obtain target type object parameters; A data set of target type objects included in the endoscopic image is determined based on the target type object parameter and a preset target type object parameter threshold.

7. The surgical ligation assisting method according to claim 1, characterized in that: The identifying of the target abnormal object in the pre-acquired endoscopic image to obtain a detection frame for marking the location of the target abnormal object and position information of the detection frame includes: Based on a pre-trained target abnormal object recognition model, a target abnormal object in a pre-acquired endoscopic image is identified, and a detection frame for marking a position of the target abnormal object and position information of the detection frame are obtained.

8. A surgical ligation assisting device, characterized in that: The device comprises: a first recognition unit, configured to recognize a target abnormal object in a pre-acquired endoscopic image, and obtain a detection frame for marking a location of the target abnormal object and position information of the detection frame, wherein the endoscopic image is an endoscopic image taken during an anal endoscopic examination of a patient; a first acquisition unit, configured to acquire position features, epithelial attribute features, venous attribute features, and lymph node attribute features of each target pixel point in the endoscopic image, wherein the target pixel point is a pixel point within a target area, and the target area includes an area excluding a detection frame; a first determining unit, configured to determine a data set of target type objects included in the endoscopic image based on position features, epithelial attribute features, venous attribute features, and lymph node attribute features of each target pixel point in the endoscopic image; The second determining unit is configured to determine an auxiliary strategy for performing surgical ligation on the target abnormal object based on the position information of the detection frame and the data set of the target type object.

9. A computer device, characterized in that: The computer device comprises: one or more processors; Memory; and One or more application programs, wherein the one or more application programs are stored in the memory and configured to be executed by the processor to implement the surgical ligation assisting method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that A computer program is stored thereon, and the computer program is loaded by a processor to execute the steps of the surgical ligation assisting method according to any one of claims 1 to 7.

Images