Auxiliary detection method, device, equipment and endoscope system for digestive tract

Abstract

The application belongs to the technical field of medical treatment and provides an auxiliary detection method, device and equipment for a digestive tract and an endoscope system. The auxiliary detection method comprises the following steps: obtaining a historical three-dimensional model of the digestive tract obtained by three-dimensional reconstruction based on historical digestive tract images, the historical three-dimensional model being marked with historical position information and historical cleanliness information at a plurality of position points of the digestive tract; obtaining a current digestive tract image obtained by shooting a current position point; determining corresponding current position information of the current digestive tract image in the digestive tract; determining target position information associated with the current position information from the historical position information and determining associated historical cleanliness information; marking a position point corresponding to the target position information in a target three-dimensional model of the digestive tract, and the target three-dimensional model being marked with auxiliary information obtained according to the historical cleanliness information associated with the target position information. The embodiments of the application can assist in improving the detection efficiency of endoscopic detection.

Description

TECHNICAL FIELD

[0001] The present application belongs to the technical field of medical treatment, and particularly relates to an auxiliary detection method, device and equipment for a digestive tract and an endoscope system. BACKGROUND

[0002] Intestinal health is related to the quality of people's life. At present, the mainstream method for doctors to examine and treat intestinal health problems is to use an endoscope to perform a colonoscopy (hereinafter referred to as an endoscopic examination). Through the endoscopic examination, adenomas can be found in the early stage and precancerous lesions can be removed, thereby reducing the risk of death from colorectal cancer. The quality of bowel preparation has a great influence on the detection rate of adenomas, and high-quality bowel preparation can significantly improve the detection rate compared with low-quality bowel preparation. Therefore, bowel preparation has become an important indicator for evaluating the quality of endoscopic examination.

[0003] In the related art, before the endoscopic examination, the intestinal tract needs to be cleaned, including removing feces, digestive residues and foreign matters in the colon. Generally, before the endoscopic examination, the method of fasting in advance and orally taking an intestinal cleaning agent can be used for bowel preparation. However, in actual application, there are some patients with insufficient bowel preparation, which causes doctors to be unable to normally perform colonoscopy. At this time, the common practice is to stop the endoscopic examination and withdraw the endoscope.

[0004] In the mode of stopping the endoscopic examination and withdrawing the endoscope, the patient needs to complete the bowel preparation again before re-entering the endoscope, and the entire intestinal tract needs to be re-examined by the endoscope, which is low in detection efficiency. SUMMARY

[0005] The embodiments of the present application provide an auxiliary detection method, device and equipment for a digestive tract and an endoscope system, which can improve the detection efficiency during the endoscopic examination.

[0006] The first aspect of the embodiments of the present application provides an auxiliary detection method for a digestive tract, applied to an image processing device, and the auxiliary detection method for the digestive tract comprises the following steps: acquiring a historical three-dimensional model of the digestive tract, the historical three-dimensional model being obtained by three-dimensional reconstruction based on a historical digestive tract image, and the historical three-dimensional model being marked with historical position information and historical cleanliness information at a plurality of position points of the digestive tract; acquiring a current digestive tract image photographed at a current position point; determining corresponding current position information of the current digestive tract image in the digestive tract; determining target position information associated with the current position information from the historical position information, and determining the historical cleanliness information associated with the target position information; and marking a position point corresponding to the target position information in a target three-dimensional model of the digestive tract, and the target three-dimensional model being marked with auxiliary information obtained according to the historical cleanliness information associated with the target position information.

[0007] In some embodiments of the first aspect, before the marking of the position point corresponding to the target position information in the target three-dimensional model of the digestive tract, the auxiliary detection method of the digestive tract further comprises: registering a first three-dimensional model according to the current position information and the historical position information to obtain the target three-dimensional model; the first three-dimensional model is the historical three-dimensional model or a three-dimensional model obtained by three-dimensional reconstruction according to the current digestive tract image.

[0008] In some embodiments of the first aspect, the current digestive tract image and the historical digestive tract image are both images obtained by endoscopy; the registration of the first three-dimensional model according to the current position information and the historical position information to obtain the target three-dimensional model comprises: determining a first moving speed according to the historical position information, the first moving speed being a moving speed of the endoscope when the historical digestive tract image is taken; determining a second moving speed according to the current position information, the second moving speed being a moving speed of the endoscope when the current digestive tract image is taken; and adjusting the first three-dimensional model according to the first moving speed and the second moving speed to obtain the target three-dimensional model.

[0009] In some embodiments of the first aspect, the registration of the first three-dimensional model according to the current position information and the historical position information to obtain the target three-dimensional model comprises: determining a first length of a digestive tract segment in the digestive tract when the historical digestive tract image is taken according to the historical position information; determining a second length of the digestive tract segment when the current digestive tract image is taken according to the current position information; and adjusting the first three-dimensional model according to the first length and the second length to obtain the target three-dimensional model.

[0010] In some embodiments of the first aspect, the current digestive tract image is an image obtained by endoscopy; after the obtaining of the current digestive tract image taken at the current position point, the auxiliary detection method of the digestive tract further comprises: detecting whether the endoscope has passed through the digestive tract segment.

[0011] In some embodiments of the first aspect, the marking of the position point corresponding to the target position information in the target three-dimensional model of the digestive tract comprises: aligning the target three-dimensional model and the historical three-dimensional model; and marking the position point corresponding to the target position information in the target three-dimensional model according to the alignment result.

[0012] In some embodiments of the first aspect, the marking, in the target three-dimensional model of the digestive tract, of the position point corresponding to the target position information comprises: taking the historical three-dimensional model as the target three-dimensional model; and marking, in the target three-dimensional model, the position point corresponding to the target position information.

[0013] In some embodiments of the first aspect, after the current digestive tract image taken at the current position point is obtained, the auxiliary detection method for the digestive tract further comprises: performing a cleanliness evaluation according to the current digestive tract image to obtain current cleanliness information corresponding to the current digestive tract image in the digestive tract.

[0014] In some embodiments of the first aspect, after the marking, in the target three-dimensional model of the digestive tract, of the position point corresponding to the target position information, the auxiliary detection method for the digestive tract further comprises: sending the target three-dimensional model to a display.

[0015] In some embodiments of the first aspect, the target three-dimensional model is a model obtained based on images collected during an endoscope insertion process, and the current digestive tract image is an image taken during an endoscope withdrawal process; after the current digestive tract image taken at the current position point is obtained, the auxiliary detection method for the digestive tract further comprises: optimizing the target three-dimensional model according to the current digestive tract image.

[0016] In some embodiments of the first aspect, the historical three-dimensional model is further marked with a proportion of a target digestive tract part in the digestive tract, and the target digestive tract part is a digestive tract part corresponding to the historical cleanliness information satisfying a preset cleanliness condition.

[0017] The second aspect of the embodiments of the present application provides an auxiliary detection device for a digestive tract, which is configured in an image processing device. The auxiliary detection device for the digestive tract comprises: a model acquisition unit configured to acquire a historical three-dimensional model of a digestive tract, the historical three-dimensional model being obtained by three-dimensional reconstruction based on historical digestive tract images, the historical three-dimensional model being marked with historical position information and historical cleanliness information at a plurality of position points of the digestive tract; an image acquisition unit configured to acquire a current digestive tract image taken at a current position point; a first determination unit configured to determine current position information corresponding to the current digestive tract image in the digestive tract; a second determination unit configured to determine target position information associated with the current position information from the historical position information, and to determine historical cleanliness information associated with the target position information; and a model marking unit configured to mark, in a target three-dimensional model of the digestive tract, a position point corresponding to the target position information, the target three-dimensional model being marked with auxiliary information obtained according to the historical cleanliness information associated with the target position information.

[0018] The third aspect of the embodiments of the present application provides an image processing device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, and the processor implements the steps of the auxiliary detection method of the digestive tract when executing the computer program.

[0019] The fourth aspect of the embodiments of the present application provides an endoscope system, including a light source device, an endoscope, an image processing device, and a display; wherein the image processing device is configured to execute the steps of the auxiliary detection method of the digestive tract as described in the first aspect.

[0020] The fifth aspect of the embodiments of the present application provides a computer readable storage medium, the computer readable storage medium stores a computer program, and the computer program implements the steps of the auxiliary detection method of the digestive tract when executed by a processor.

[0021] The sixth aspect of the embodiments of the present application provides a computer program product, when the computer program product is executed on an image processing device, the image processing device executes the steps of the auxiliary detection method of the digestive tract.

[0022] In the embodiments of the present application, at the first endoscopy, a three-dimensional model is obtained by three-dimensional reconstruction based on historical digestive tract images, and historical position information and historical cleanliness information of the digestive tract part are marked to detect the digestive tract part that can be detected first. At the second endoscopy, the current position information corresponding to the current digestive tract image in the digestive tract is determined by acquiring the current digestive tract image, the target position information associated with the current position information is determined from the historical position information, and the historical cleanliness information associated with the target position information is determined. Then, in the target three-dimensional model of the digestive tract, the position point corresponding to the target position information is marked. Since the target three-dimensional model is marked with auxiliary information obtained according to the historical cleanliness information associated with the target position information, the doctor can obtain the historical cleanliness information of the position point corresponding to the first endoscopy according to the auxiliary information in the target three-dimensional model, and then skip the digestive tract part that has been detected at the first endoscopy, without the need to detect the entire digestive tract again, thereby improving the detection efficiency of endoscopic detection. BRIEF DESCRIPTION OF DRAWINGS

[0023] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following will briefly introduce the drawings needed to be used in the embodiments or prior art description. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can also be obtained by those skilled in the art without creative labor.

[0024] Figure 1is a structural schematic diagram of an endoscope system provided by an embodiment of the present application;

[0025] Figure 2 is an implementation flowchart of an auxiliary detection method for a digestive tract provided by an embodiment of the present application;

[0026] Figure 3 is a schematic diagram of intestines corresponding to different cleanliness scores provided by an embodiment of the present application;

[0027] Figure 4 is a structural schematic diagram of intestines provided by an embodiment of the present application;

[0028] Figure 5 is a first schematic diagram of marking a position point and auxiliary information provided by an embodiment of the present application;

[0029] Figure 6 is a second schematic diagram of marking a position point and auxiliary information provided by an embodiment of the present application;

[0030] Figure 7 is a first implementation flowchart of registration of a first three-dimensional model provided by an embodiment of the present application;

[0031] Figure 8 is a second implementation flowchart of registration of a first three-dimensional model provided by an embodiment of the present application;

[0032] Figure 9 is a structural schematic diagram of an auxiliary detection device for a digestive tract provided by an embodiment of the present application;

[0033] Figure 10 is a structural schematic diagram of an image processing device provided by an embodiment of the present application. DETAILED DESCRIPTION

[0034] In order to make the objectives, technical solutions and advantages of the present application clearer, the present application will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and do not limit the present application. Based on the embodiments of the present application, all other embodiments obtained by those skilled in the art without creative work belong to the present application.

[0035] It should be understood that when used in the specification and the appended claims of the present application, the term "comprising" indicates the presence of the described features, integers, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or sets thereof.

[0036] In the description and drawings of the present application, the terms "first", "second", "third", etc. are used only to distinguish descriptions, and cannot be understood as indicating or implying relative importance.

[0037] In the present application, the reference "one embodiment" or "some embodiments" means that the specific features, structures or characteristics described in connection with the embodiment are included in one or more embodiments of the present application. Therefore, the statements "in one embodiment", "in some embodiments", "in other some embodiments", "in further some embodiments" and the like appearing in different places in the specification are not necessarily all referring to the same embodiment, but mean "one or more but not all embodiments", unless otherwise specifically emphasized. The terms "include", "contain", "have" and their variants mean "include but not limited to", unless otherwise specifically emphasized.

[0038] In the related art, before endoscopy, the intestinal tract needs to be cleaned, including removing feces, digestive residues and foreign matter in the colon. Generally, before endoscopy, intestinal preparation can be performed by fasting in advance and orally taking an intestinal cleaning agent. However, in actual application, there are some patients with insufficient intestinal preparation, which causes doctors to be unable to normally perform colonoscopy. At this time, the common practice is to stop endoscopy and withdraw the endoscope.

[0039] In the mode of stopping endoscopy and withdrawing the endoscope, the patient needs to complete intestinal preparation again before re-entering the endoscope, and the entire intestinal tract needs to be re-endoscoped, which is low in detection efficiency.

[0040] In view of this, the present application provides an auxiliary detection method for the digestive tract, which can refer to a historical three-dimensional model, mark auxiliary information in a target three-dimensional model, and the auxiliary information is related to historical cleanliness information at a corresponding position. Doctors can know the historical cleanliness information at the corresponding position point in the first endoscopy according to the auxiliary information, and then select whether to skip the digestive tract part that has been detected in the first endoscopy according to needs, without the need to detect the entire digestive tract again, thereby improving the detection efficiency of endoscopy detection.

[0041] In order to illustrate the technical solutions of the present application, specific embodiments are described below.

[0042] Please refer to Figure 1 , Figure 1 is a schematic diagram of an endoscope system provided by an embodiment of the present application. The endoscope system can include a light source device 101, an endoscope 102 (also referred to as a scope), an image processing device 103 and a display 104.

[0043] The light source device 101 can include a light source and an optical element. The light source can generate a light beam of a specific wavelength. The optical element can include, but is not limited to, a collimating lens, a condensing lens, and the like. The light source device 101 can be configured to converge the light beam generated by the light source and make the light beam incident on the end surface of the light guide of the endoscope 102, so that the light beam is emitted from the front end of the endoscope 102 to provide illumination light for the endoscope 102.

[0044] The endoscope 102 can include an insertion portion in an elongated shape. The insertion portion can be internally provided with an image pickup element. The endoscope 102 can capture an image of the front end of the endoscope 102 by the image pickup element, and feed the captured image to the image processing device 103 through a connection line. More specifically, a doctor can insert the insertion portion into a patient's body, and capture an image of the patient's body by the image pickup element. It should be understood that the endoscope 102 can also be connected with an operation portion at one end, and the operation portion can generate a control signal for controlling the insertion portion under the operation of the doctor.

[0045] The image processing device 103 can be a computer or a dedicated processing device specially used for the endoscope system. The image processing device 103 can perform image processing based on the image captured by the endoscope 102, and send the image processing result to the display 104. It should be understood that the image processing includes, but is not limited to, image recognition, image cropping, and image enhancement.

[0046] The display 104 can be a Liquid Crystal Display (LCD) display, a Light-emitting Diode (LED) display, or other display devices with display capability. The display 104 can display according to the image processing result provided by the image processing device 103.

[0047] In some examples, the display 104 can display the image captured by the image pickup element of the endoscope 102 in real time, and the recognition result of the image recognition (e.g., lesion recognition) performed by the image processing device 103 on the image. In other examples, the display 104 can also display a three-dimensional model obtained by the image processing device 103 based on the three-dimensional reconstruction of the image.

[0048] It should be noted that, Figure 1 The endoscope system is not limited in this regard, and in actual applications, the endoscope system can include more or fewer components than those shown, or can combine certain components, or different components. For example, the endoscope system can also include a power supply device, an Artificial Intelligence (AI) processing device specially used for executing an AI algorithm, and the like, which are not limited by the present application.

[0049] To solve the problem of low detection efficiency in endoscopy in the related art, please refer to Figure 2 , Figure 2 An implementation flow diagram of an auxiliary detection method of a digestive tract provided by an embodiment of the present application is shown. The method can be applied to the image processing device 103 described above and can be applied to a situation where the detection efficiency of endoscopy needs to be improved.

[0050] Specifically, the auxiliary detection method of the digestive tract described above can include the following steps S201 to S205.

[0051] Step S201, obtaining a historical three-dimensional model of the digestive tract.

[0052] In an embodiment of the present application, the digestive tract refers to any part of the digestive tract system, including but not limited to the intestinal tract, esophagus, and throat.

[0053] In an embodiment of the present application, the historical three-dimensional model can be obtained by three-dimensional reconstruction based on historical digestive tract images. The historical digestive tract images can refer to images obtained by the camera element of the endoscope 102 when capturing the digestive tract during a past endoscopy (e.g., a previous endoscopy). Based on the historical digestive tract images, a historical three-dimensional model of the digestive tract can be obtained by three-dimensional reconstruction. The historical three-dimensional model can show the condition of the digestive tract during the past endoscopy.

[0054] It should be understood that the historical three-dimensional model can be stored in the memory of the image processing device 103 or input by the user to the image processing device 103, and the present application does not limit this.

[0055] In an embodiment of the present application, the historical three-dimensional model can be marked with historical position information and historical cleanliness information at multiple position points of the digestive tract. The position point refers to a point inside the digestive tract and can refer to a marked point on a specific digestive tract part. For example, the position point can be a corresponding marked point on each intestinal segment.

[0056] The historical position information is used to represent the position of each position point on the digestive tract during the process of collecting the historical digestive tract images. The historical position information can be represented in the form of absolute coordinates, relative coordinates, etc. In an embodiment of the present application, the historical position information can be obtained by identifying the historical digestive tract images, or can be recorded in real time by the positioning module built in the endoscope 102, and the present application does not limit this.

[0057] The historical cleanliness information is used to represent the cleanliness of each position point in the process of collecting the historical digestive tract image. The historical cleanliness information can be expressed in the form of a score or a rating. Taking the intestine as an example, the cleanliness of the position point can be scored according to the Boston Bowel Preparation Scale (BBPS). The Boston Bowel Preparation Scale rates the cleanliness from bad to good as 0, 1, 2, and 3 points. Please refer to Figure 3 , Figure 3 In the Boston Bowel Preparation Scale, a score of 0 corresponds to A, indicating that solid or liquid feces cannot be removed, and the mucosa of the entire intestinal segment cannot be observed; a score of 1 corresponds to B, indicating that stains, cloudy liquid, and residual feces cause partial mucosa to be unobservable; a score of 2 corresponds to C, and a score of 3 corresponds to D, indicating that the intestine is relatively clean, and the doctor can perform an endoscopic examination. The historical cleanliness information can be obtained by identifying the historical digestive tract image. The identification method can use a convolutional neural network model, a fully connected neural network model, etc., which is not limited in this application.

[0058] In step S202, a current digestive tract image captured at a current position point is obtained.

[0059] The current digestive tract image is an image captured by the camera element of the endoscope 102 at the current position point. The current position point can represent the point in the digestive tract currently captured by the camera element of the endoscope 102. It can be understood that the endoscope 102 can move in the patient's body, and the current digestive tract image can specifically include images continuously captured during the movement of the endoscope 102.

[0060] In step S203, the current position information corresponding to the current digestive tract image in the digestive tract is determined.

[0061] The current position information can represent the position of the current position point. Similarly, the current position information can be obtained based on the current digestive tract image or based on the positioning module built in the endoscope 102.

[0062] In step S204, the target position information associated with the current position information is determined from the historical position information, and the historical cleanliness information associated with the target position information is determined.

[0063] Specifically, the image processing device 103 can determine the position point corresponding to the current position point during the past endoscopic examination according to the current position information, and then determine the historical position information of the position point in the historical position information as the target position information associated with the current position information.

[0064] After obtaining the target position information, the image processing device 103 can obtain the historical cleanliness information at the same position point as the historical cleanliness information associated with the target position information.

[0065] Step S205: Mark the location points corresponding to the target location information in the target three-dimensional model of the digestive tract.

[0066] The target 3D model can be either the aforementioned historical 3D model or a new 3D model; this application does not impose any restrictions on this. When the target 3D model is a new 3D model, the new 3D model differs from the historical 3D model and can be used to demonstrate the state of the digestive tract during this endoscopic examination.

[0067] In embodiments of this application, the image processing device 103 can mark position points corresponding to target position information in the target 3D model using markers of a specific style. Markers of a specific style may include, but are not limited to, markers of a specific color or markers of a specific pattern.

[0068] In embodiments of this application, the target 3D model is further marked with auxiliary information obtained from historical cleanliness information associated with the target location information. This auxiliary information can be used to identify the cleanliness of the location point corresponding to the target location information, or whether the location point corresponding to the target location information has been examined in previous endoscopic examinations.

[0069] Please refer to Figure 4 , Figure 4 This is a schematic diagram of the intestinal structure for applying the auxiliary detection method of this embodiment. Figure 4 As shown, between the ascending colon 401 and the transverse colon 403, there is a right colonic flexure 402; between the transverse colon 403 and the descending colon-sigmoid colon 405, there is a left colonic flexure 404; in addition, the intestine also includes the appendix, cecum, ileum, rectum and anal canal (not shown in the figure).

[0070] As an example, such as Figure 5 As shown, point markers 51 can be used to mark the location points corresponding to the target location information in the target 3D model of the intestine. Then, uninspected areas can be determined based on historical cleanliness information. For example, if the cleanliness score is lower than the score threshold based on the historical cleanliness information corresponding to the location point, it is considered an uninspected area, and a bounding box 52 is generated in the target 3D model as auxiliary information. The area within the bounding box 52 is the uninspected area.

[0071] As another example, such as Figure 6 As shown, in the target 3D model of the intestine, the location points corresponding to the target location information can be marked by dot markers 61. Then, an information box 62 is generated in the target 3D model as auxiliary information. The information box 62 can display the cleanliness score corresponding to the historical cleanliness information. It can be understood that when the cleanliness score is lower than a certain threshold, such as the BBPS cleanliness score of the intestinal segment being lower than 2 points, that location is an area that has not been examined.

[0072] In actual applications, the marking manner of the position points and the auxiliary information can be adjusted according to the needs of doctors or patients, and the application does not limit this.

[0073] In the embodiments of the application, in the first endoscopy, a historical three-dimensional model is obtained by three-dimensional reconstruction based on historical digestive tract images, and historical position information and historical cleanliness information of the digestive tract part are marked to detect the digestive tract part that can be detected first. In the second endoscopy, the current position information corresponding to the current digestive tract image in the digestive tract is determined by acquiring the current digestive tract image, the target position information associated with the current position information is determined from the historical position information, and the historical cleanliness information associated with the target position information is determined. Then, in the target three-dimensional model, the position point corresponding to the target position information is marked. Since the auxiliary information obtained according to the historical cleanliness information associated with the target position information is marked in the target three-dimensional model, the doctor can know the historical cleanliness information at the corresponding position point in the first endoscopy according to the auxiliary information in the target three-dimensional model, and thus the digestive tract part detected in the first endoscopy can be skipped without the need to detect the entire digestive tract again, thereby improving the detection efficiency of endoscopic detection.

[0074] In some embodiments, the target three-dimensional model can be obtained based on the historical three-dimensional model, or can be obtained based on a three-dimensional model reconstructed from the digestive tract images acquired in the current examination process.

[0075] Specifically, before the position point corresponding to the target position information is marked in the target three-dimensional model of the digestive tract, the image processing device 103 can register the first three-dimensional model to obtain the target three-dimensional model according to the current position information and the historical position information. The first three-dimensional model can be the historical three-dimensional model, or can be a three-dimensional model obtained by three-dimensional reconstruction based on the digestive tract images acquired in the current examination process (including the current digestive tract images collected in real time and the digestive tract images collected before this in the current examination process).

[0076] The purpose of registration is to solve the mapping relationship between the two models. In short, since the digestive tract such as the intestine can wriggle, resulting in a certain change in the length and spatial position of the digestive tract, in the embodiments of the application, by registration, the change relationship between the situation of the digestive tract presented by the historical three-dimensional model in the past endoscopy and the current situation of the digestive tract can be determined, and the target three-dimensional model is obtained.

[0077] With the first three-dimensional model as a historical three-dimensional model as an example, by registration, a position point on the historical three-dimensional model can be aligned to a new position point to obtain a target three-dimensional model. That is, by adjusting an old three-dimensional model established during a past endoscopy, a new three-dimensional model can be obtained. Correspondingly, if the historical cleanliness information at a position point on the historical three-dimensional model does not meet the preset cleanliness condition, for example, the corresponding cleanliness score is lower than the score threshold, the position point corresponding to the position point on the new three-dimensional model is a position point that has not been examined.

[0078] With the first three-dimensional model as a three-dimensional model obtained by three-dimensional reconstruction according to the current digestive tract image as an example, by registration, a position point on the first three-dimensional model can be aligned to a position point on the historical three-dimensional model to determine whether each position point on the first three-dimensional model needs to be examined. Specifically, according to the position point on the first three-dimensional model corresponding to the position point on the historical three-dimensional model, if the historical cleanliness information does not meet the preset cleanliness condition, for example, the corresponding cleanliness score is lower than the score threshold, the position point on the first three-dimensional model is a position point that has not been examined.

[0079] Specifically, in some embodiments of the present application, the current digestive tract image and the historical digestive tract image are both images obtained by the endoscope 102. Please refer to Figure 7 The above registration of the first three-dimensional model to obtain the target three-dimensional model can include the following steps S701 to S703.

[0080] Step S701, determining a first moving speed according to the historical position information.

[0081] The first moving speed is the moving speed of the endoscope 102 when the historical digestive tract image is captured.

[0082] Step S702, determining a second moving speed according to the current position information.

[0083] The second moving speed is the moving speed of the endoscope 102 when the current digestive tract image is captured.

[0084] Step S703, adjusting the first three-dimensional model according to the first moving speed and the second moving speed to obtain the target three-dimensional model.

[0085] Specifically, the endoscope 102 can be slowly moved in the digestive tract under the operation of a doctor, and the digestive tract images are collected in the moving process. In the past endoscopy, the endoscope 102 moves in the digestive tract at a first moving speed, and based on the historical position information and the collection time corresponding to the historical position information, the first moving speed of the endoscope 102 when the historical digestive tract image is shot can be calculated. Similarly, in the current endoscopy, the endoscope 102 moves in the digestive tract at a second moving speed, and based on the current position information and the collection time corresponding to the current position information, the second moving speed of the endoscope 102 when the current digestive tract image is shot can be calculated.

[0086] Taking the first three-dimensional model as a historical three-dimensional model as an example, the image processing device 103 can proportionally convert and align the position points on the historical three-dimensional model to new position points based on the first moving speed and the second moving speed, to obtain a target three-dimensional model. Taking the intestinal segment A as an example, if the first moving speed of the endoscope 102 through the intestinal segment A is v1, and the second moving speed of the endoscope 102 through the intestinal segment A is v2, then the length s1 of the intestinal segment A in the historical three-dimensional model can be proportionally adjusted to the current length s2 according to the scaling ratio v2 / v1, to obtain the target three-dimensional model, so that the length of the intestinal segment A on the target three-dimensional model matches the actual length of the intestinal segment A.

[0087] Taking the first three-dimensional model as a three-dimensional model obtained by three-dimensional reconstruction according to the current digestive tract image as an example, the image processing device 103 can proportionally map based on the first moving speed and the second moving speed. Taking the intestinal segment A as an example, if the first moving speed of the endoscope 102 through the intestinal segment A is v1, and the second moving speed of the endoscope 102 through the intestinal segment A is v2, then the part of the intestinal segment A on the first three-dimensional model and the part of the intestinal segment A in the historical three-dimensional model can be mapped to each other according to the scaling ratio v2 / v1.

[0088] In some embodiments of the present application, referring to Figure 8 The above-mentioned registration of the first three-dimensional model to obtain the target three-dimensional model can also include the following steps S801 to S803.

[0089] Step S801, determining a first length of a digestive tract segment in the digestive tract when a historical digestive tract image is shot according to historical position information.

[0090] Step S802, determining a second length of the digestive tract segment when a current digestive tract image is shot according to current position information.

[0091] Step S803, adjusting the first three-dimensional model according to the first length and the second length to obtain a target three-dimensional model.

[0092] The digestive tract segment is a partial region on the digestive tract. In embodiments of the present application, the digestive tract segment can specifically refer to a digestive tract site. For example, taking the digestive tract as the intestinal tract, the digestive tract segment can refer to Figure 4 a segment of the ascending colon 401, the right colon 402, the transverse colon 403, and the like. Of course, the digestive tract segment can also be a segment formed by any two points, and the present application does not limit this.

[0093] In embodiments of the present application, according to the historical position information, the coordinates of the starting point and the coordinates of the end point on the digestive tract segment during the past endoscopy can be determined, and then the first length of the digestive tract segment when the historical digestive tract image is captured can be calculated. Similarly, according to the current position information, the coordinates of the starting point and the coordinates of the end point on the digestive tract segment during the current endoscopy can be determined, and then the second length of the digestive tract segment when the current digestive tract image is captured can be calculated.

[0094] Taking the first three-dimensional model as a historical three-dimensional model, based on the first length and the second length, the digestive tract segment on the historical three-dimensional model can be aligned to a new position in a proportional manner to obtain a target three-dimensional model. Taking the intestinal segment A as an example, if the first length of the intestinal segment A when the historical digestive tract image is captured is s1, and the second length when the current digestive tract image is captured is s2, then the part of the intestinal segment A in the historical three-dimensional model can be correspondingly adjusted to the current second length s2 in a scaling ratio of s2 / s1, so that the length of the part of the intestinal segment A on the target three-dimensional model matches the actual current length of the intestinal segment A.

[0095] Taking the first three-dimensional model as a three-dimensional model obtained by three-dimensional reconstruction based on the current digestive tract image as an example, the image processing device 103 can perform mapping in a proportional manner based on the first length and the second length. Taking the intestinal segment A as an example, if the first length of the intestinal segment A passed by the endoscope 102 is s1, and the second length is s2, then the part of the intestinal segment A on the first three-dimensional model and the part of the intestinal segment A in the historical three-dimensional model can be mapped to each other in a scaling ratio of s2 / s1.

[0096] In this way, the obtained target three-dimensional model can provide more accurate auxiliary detection guidance for doctors and patients.

[0097] It should be noted that the image processing device 103 can detect whether the endoscope 102 has passed the digestive tract segment after obtaining the current digestive tract image captured at the current position point.

[0098] For example, whether the endoscope 102 has passed the digestive tract segment can be detected based on whether the end point of the digestive tract segment is recognized. For another example, whether the endoscope 102 has passed the digestive tract segment can be determined according to the relative positional relationship between the coordinates of the end point of the digestive tract segment and the coordinates of the current endoscope 102.

[0099] If the endoscope 102 has passed through the segment of the digestive tract, it is convenient to calculate the first length and the second length, at this time, the registration can be performed in the manner shown in Figure 8 If the endoscope 102 has not passed through the segment of the digestive tract, the registration can be performed in the manner shown in Figure 7 Thus, the effectiveness of the registration can be guaranteed, and the situation of invalid registration due to the failure to identify the complete segment of the digestive tract can be avoided.

[0100] It should be further noted that the endoscopy generally includes two stages of insertion and withdrawal, and in actual application, the above registration process can be realized based on the current position information collected during the insertion stage of the current endoscopy, or can be performed in real time based on the current position information collected during the withdrawal stage of the current endoscopy, and the present application does not limit this.

[0101] In addition, for the case that the first three-dimensional model is a three-dimensional model obtained by three-dimensional reconstruction based on the current digestive tract image, before marking the position point corresponding to the target position information, the image processing device 103 can perform three-dimensional reconstruction based on the current digestive tract image to obtain the first three-dimensional model.

[0102] The three-dimensional reconstruction manner can adopt a reconstruction manner based on image disparity, a deep learning manner, or other existing three-dimensional reconstruction algorithms, and the present application does not limit this.

[0103] In order to make the target three-dimensional model more accurate, the first three-dimensional model can be a model obtained based on images collected during the insertion process of the endoscope 102, and the current digestive tract image can further include images captured during the withdrawal process of the endoscope 102. After obtaining the current digestive tract image captured at the current position point, the auxiliary detection method of the digestive tract further includes: optimizing the aforementioned target three-dimensional model based on the current digestive tract image captured during the withdrawal process of the endoscope 102.

[0104] That is, three-dimensional reconstruction can be performed based on images collected during the insertion process of the endoscope 102, and registration can be performed to obtain the target three-dimensional model, and then the target three-dimensional model can be further optimized based on images collected during the withdrawal process of the endoscope 102.

[0105] Since the examination process of the endoscopy generally occurs during the withdrawal process, the movement speed of the endoscope 102 during the withdrawal is relatively slower than during the insertion, and therefore clearer current digestive tract images can be collected during the withdrawal process. Therefore, based on the clearer current digestive tract images collected during the withdrawal, the target three-dimensional model can be optimized, so that the target three-dimensional model is more fine and accurate.

[0106] Similarly, the historical three-dimensional model can also be obtained by optimizing an initial three-dimensional model based on the historical digestive tract images collected in the withdrawal process, and the initial three-dimensional model is obtained by three-dimensional reconstruction based on the historical digestive tract images collected in the insertion process.

[0107] It can be understood that, whether the historical three-dimensional model is registered or the three-dimensional model is directly reconstructed based on the current digestive tract images, a target three-dimensional model different from the historical three-dimensional model will be obtained. At this time, in order to mark the position point corresponding to the target position information, the image processing device 103 can align the target three-dimensional model and the historical three-dimensional model, and then mark the position point corresponding to the target position information in the target three-dimensional model according to the alignment result.

[0108] Specifically, by aligning the target three-dimensional model and the historical three-dimensional model, the position of the position point corresponding to the target position information in the historical three-dimensional model in the target three-dimensional model can be obtained, or in other words, the mapping relationship between the positions of the historical three-dimensional model and the target three-dimensional model can be obtained. According to the mapping relationship, the position point corresponding to the target position information can be marked in the target three-dimensional model, and the auxiliary information can be further marked, so as to facilitate the doctor to confirm whether the current position point has been detected in the past endoscopy based on the marking of the auxiliary information and the position point.

[0109] In some embodiments of the present application, the image processing device 103 can also mark the position point corresponding to the target position information in the historical three-dimensional model as the target three-dimensional model.

[0110] At this time, the image processing device 103 does not need to generate a new target three-dimensional model, but uses the existing historical three-dimensional model to mark the position point corresponding to the target position information on the historical three-dimensional model, so as to reduce the high computational problem of the image processing device 103 caused by three-dimensional reconstruction.

[0111] It should be noted that the historical three-dimensional model can also be marked with the proportion of the target digestive tract part in the digestive tract, and the target digestive tract part is the digestive tract part corresponding to the historical cleanliness information satisfying the preset cleanliness condition.

[0112] The digestive tract part satisfying the preset cleanliness condition is also the digestive tract part that can be subjected to endoscopy. Taking the intestinal tract as an example, if the historical cleanliness information of the intestinal segment corresponds to a BBPS cleanliness score of 2 or 3, the intestinal segment can be confirmed as the target digestive tract part.

[0113] The proportion of the target digestive tract site in the digestive tract can refer to a length proportion or a quantity proportion. The length proportion refers to a ratio between the length of the target digestive tract site and the total length of the digestive tract. The quantity proportion refers to a ratio between the number of the target digestive tract site and the total number of all digestive tract sites in the digestive tract.

[0114] The higher the proportion, the higher the degree of cleaning of the digestive tract, and the more sufficient the preoperative preparation of the patient; the lower the proportion, the lower the degree of cleaning of the digestive tract, and the less sufficient the preoperative preparation of the patient. Therefore, by marking the proportion in the historical three-dimensional model, the preoperative preparation of the patient can be prompted to the doctor, and the doctor can provide the patient with a suggestion for preoperative preparation, so that the preoperative preparation of the patient for this endoscopy is more sufficient.

[0115] In some embodiments of the present application, the image processing device 103 can also perform a cleaning degree evaluation according to the current digestive tract image to obtain current cleaning degree information corresponding to the current digestive tract image in the digestive tract.

[0116] The manner of cleaning degree evaluation can refer to the foregoing description of historical cleaning degree information, and will not be repeated here. Through the cleaning degree evaluation, the image processing device 103 can provide the current cleaning degree information to assist the doctor in judging whether the current cleaning degree of the digestive tract meets the needs of the endoscopy.

[0117] After obtaining the target three-dimensional model, the image processing device 103 can also send the target three-dimensional model to the display 104. The display 104 can display the target three-dimensional model to visually provide the current examination condition for the doctor or the patient.

[0118] As a specific example but not limited, the image processing device 103 can obtain the historical digestive tract image captured by the camera element of the endoscope 102 and the historical position information recorded by the positioning module when the patient performs the first endoscopy. Then, by performing a cleaning degree score on the historical digestive tract image, the corresponding historical cleaning degree information is obtained. Based on the historical digestive tract image, the image processing device 103 can obtain the historical three-dimensional model, and mark the historical position information and the historical cleaning degree information at each position in the historical three-dimensional model. In this process, the three-dimensional reconstruction can be performed when the endoscope is inserted, and the three-dimensional model can be updated and optimized when the endoscope is withdrawn. During the withdrawal process, the doctor can check the positions whose historical cleaning degree information meets the preset cleaning degree condition.

[0119] When the patient is undergoing the second endoscopy, the image processing device 103 can read the historical three-dimensional model obtained during the first endoscopy from the storage in response to the operation instruction of the doctor. Meanwhile, the image processing device 103 can acquire the current digestive tract image captured by the camera element of the endoscope 102 and the current position information recorded by the positioning module when the patient is undergoing the second endoscopy, identify the position point, determine the target position information associated with the current position information, and determine the historical cleanliness information associated with the target position information, and mark in the target three-dimensional model. The doctor can select whether to skip the region (i.e., the position point corresponding to the obtained historical cleanliness information satisfying the preset cleanliness condition) that has been detected during the first endoscopy according to the marking of the target position information and the auxiliary information in the target three-dimensional model, find the undetected region (i.e., the position point corresponding to the obtained historical cleanliness information not satisfying the preset cleanliness condition) for detection, and thus improve the efficiency of the endoscopy.

[0120] It should be noted that, for the foregoing method embodiments, in order to simply describe, they are all expressed as a series of action combinations, but those skilled in the art should know that the present application is not limited to the action sequence described, because according to the present application, certain steps can be performed in other sequences.

[0121] As Figure 9 Fig. 9 shows a structural schematic diagram of an auxiliary detection device for a digestive tract 900 provided by an embodiment of the present application, which is configured on the image processing device 103.

[0122] Specifically, the auxiliary detection device for a digestive tract 900 can include:

[0123] a model acquisition unit 901, configured to acquire a historical three-dimensional model of a digestive tract, the historical three-dimensional model being obtained by three-dimensional reconstruction based on historical digestive tract images, and the historical three-dimensional model being marked with historical position information and historical cleanliness information of a plurality of position points of the digestive tract;

[0124] an image acquisition unit 902, configured to acquire a current digestive tract image captured for a current position point;

[0125] a first determination unit 903, configured to determine current position information corresponding to the current digestive tract image in the digestive tract;

[0126] a second determination unit 904, configured to determine target position information associated with the current position information from the historical position information, and determine the historical cleanliness information associated with the target position information;

[0127] The model marking unit 905 is configured to mark a position point corresponding to the target position information in a target three-dimensional model of the digestive tract, and the target three-dimensional model is marked with auxiliary information obtained according to the historical cleanliness information associated with the target position information.

[0128] In some embodiments of the present application, the auxiliary detection device 900 of the digestive tract can include a model processing unit configured to: register the first three-dimensional model according to the current position information and the historical position information to obtain the target three-dimensional model; and the first three-dimensional model is the historical three-dimensional model or a three-dimensional model obtained by three-dimensional reconstruction according to the current digestive tract image.

[0129] In some embodiments of the present application, the current digestive tract image and the historical digestive tract image are images obtained by the endoscope 102; and the model processing unit can be specifically configured to: determine a first moving speed according to the historical position information, the first moving speed being a moving speed of the endoscope 102 when the historical digestive tract image is captured; determine a second moving speed according to the current position information, the second moving speed being a moving speed of the endoscope 102 when the current digestive tract image is captured; and adjust the first three-dimensional model according to the first moving speed and the second moving speed to obtain the target three-dimensional model.

[0130] In some embodiments of the present application, the model processing unit can also be specifically configured to: determine a first length of a digestive tract segment in the digestive tract when the historical digestive tract image is captured according to the historical position information; determine a second length of the digestive tract segment when the current digestive tract image is captured according to the current position information; and adjust the first three-dimensional model according to the first length and the second length to obtain the target three-dimensional model.

[0131] In some embodiments of the present application, the current digestive tract image is an image obtained by the endoscope 102; and the model processing unit can also be specifically configured to: detect whether the endoscope 102 has passed through the digestive tract segment.

[0132] In some embodiments of the present application, the model marking unit 905 can be specifically configured to: align the target three-dimensional model and the historical three-dimensional model; and mark the position point corresponding to the target position information in the target three-dimensional model according to an alignment result.

[0133] In some embodiments of the present application, the model marking unit 905 can be specifically configured to: take the historical three-dimensional model as the target three-dimensional model; and mark the position point corresponding to the target position information in the target three-dimensional model.

[0134] In some embodiments of the present application, the auxiliary detection device 900 for the digestive tract can comprise a cleanliness evaluation unit configured to perform a cleanliness evaluation based on the current digestive tract image to obtain corresponding current cleanliness information of the current digestive tract image in the digestive tract.

[0135] In some embodiments of the present application, the auxiliary detection device 900 for the digestive tract can comprise a communication unit configured to send the target three-dimensional model to the display 104.

[0136] In some embodiments of the present application, the target three-dimensional model is a model obtained based on images collected during the insertion process of the endoscope 102, and the current digestive tract image is an image taken during the withdrawal process of the endoscope 102; the model processing unit can be further configured to optimize the target three-dimensional model based on the current digestive tract image.

[0137] In some embodiments of the present application, the historical three-dimensional model is further marked with a proportion of the target digestive tract part in the digestive tract, and the target digestive tract part is a digestive tract part corresponding to the historical cleanliness information satisfying a preset cleanliness condition.

[0138] It should be noted that, for the convenience and brevity of description, the specific working process of the auxiliary detection device 900 for the digestive tract can refer to Figures 1 to 8 the corresponding process of the method, which will not be repeated here.

[0139] As Figure 10 shown, it is a schematic diagram of an image processing device 103 provided by an embodiment of the present application. Specifically, the image processing device 103 can comprise a processor 1030, a memory 1031, and a computer program 1032 stored in the memory 1031 and executable on the processor 1030, such as an auxiliary detection program for the digestive tract. The processor 1030 implements the steps in each of the auxiliary detection methods for the digestive tract embodiments when executing the computer program 1032, such as Figure 2 steps S201 to S205 shown. Alternatively, the processor 1030 implements the functions of each module / unit in each of the device embodiments when executing the computer program 1032, such as Figure 9 the functions of the model obtaining unit 901, the image obtaining unit 902, the first determining unit 903, the second determining unit 904, and the model marking unit 905 shown.

[0140] The computer program can be divided into one or more modules / units, which are stored in the memory 1031 and executed by the processor 1030 to complete the present application. The one or more modules / units can be a series of computer program instruction segments capable of completing a specific function, which are used to describe the execution process of the computer program in the image processing device.

[0141] For example, the computer program can be divided into a model obtaining unit, an image obtaining unit, a first determining unit, a second determining unit and a model marking unit. The specific functions of each unit are as follows: the model obtaining unit is configured to obtain a historical three-dimensional model of a digestive tract, the historical three-dimensional model being obtained by three-dimensional reconstruction based on historical digestive tract images, and the historical three-dimensional model being marked with historical position information and historical cleanliness information at a plurality of position points of the digestive tract; the image obtaining unit is configured to obtain a current digestive tract image taken at a current position point; the first determining unit is configured to determine current position information corresponding to the current digestive tract image in the digestive tract; the second determining unit is configured to determine target position information associated with the current position information from the historical position information, and determine the historical cleanliness information associated with the target position information; and the model marking unit is configured to mark a position point corresponding to the target position information in a target three-dimensional model of the digestive tract, and the target three-dimensional model is marked with auxiliary information obtained according to the historical cleanliness information associated with the target position information.

[0142] The image processing device 103 can include, but is not limited to, a processor 1030 and a memory 1031. Those skilled in the art can understand that Figure 10 The image processing device is only an example and does not constitute a limitation on the image processing device, and can include more or fewer components than those shown, or combine certain components, or different components, for example, the image processing device can also include an input / output device, a network access device, a bus, etc.

[0143] The processor 1030 can be a central processing unit (CPU), and can also be other general-purpose processors, digital signal processors (DSPs), application specific integrated circuits (ASICs), ready programmable gate arrays or other programmable logic devices, discrete gates or transistor logic components, discrete hardware components, etc. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor.

[0144] The memory 1031 can be an internal storage unit of the image processing device, for example, a hard disk or a memory of the image processing device. The memory 1031 can also be an external storage device of the image processing device, for example, a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. equipped on the image processing device. Further, the memory 1031 can include both the internal storage unit and the external storage device of the image processing device. The memory 1031 is used to store the computer program and other programs and data required by the image processing device. The memory 1031 can also be used to temporarily store data that has been output or will be output.

[0145] It should be noted that, for the convenience and brevity of description, the structure of the image processing device 103 can also refer to the specific description of the structure in the method embodiments, which will not be repeated here.

[0146] Those skilled in the art can clearly understand that, for the convenience and brevity of description, only the above-mentioned division of each functional unit and module is exemplified, and in actual application, the above-mentioned functions can be completed by different functional units and modules according to needs, that is, the internal structure of the device is divided into different functional units or modules to complete all or part of the functions described above. Each functional unit and module in the embodiment can be integrated in one processing unit, or each unit can exist physically, or two or more units can be integrated in one unit. The integrated unit can be realized in the form of hardware or software. In addition, the specific name of each functional unit and module is only for easy distinction, and does not limit the protection scope of the present application. The specific working process of the unit and module in the above system can refer to the corresponding process in the foregoing method embodiments, which will not be repeated here.

[0147] In the above embodiments, the description of each embodiment has its own emphasis, and the parts not described or recorded in detail in a certain embodiment can refer to the relevant description of other embodiments.

[0148] Those of ordinary skill in the art can realize that the units and algorithm steps of each example described in combination with the embodiments disclosed herein can be realized by electronic hardware or a combination of computer software and electronic hardware. Whether the functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. A person skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the present application.

[0149] In the embodiments of the present application, it should be understood that the disclosed apparatuses / devices and methods can be implemented in other manners. For example, the embodiments of the apparatuses / devices described above are merely schematic, and the division of the modules or units is merely logical function division, and there can be another division manner in actual implementation. For example, a plurality of units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the displayed or discussed mutual couplings or direct couplings or communication connections between the units can be indirect couplings or communication connections through some interfaces, devices or units, and can be electrical, mechanical or in other forms.

[0150] The units described as separate components can or can not be physically separate, and the components shown as units can or can not be physical units, i.e., can be located in one place, or can be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the embodiments.

[0151] In addition, each functional unit in the various embodiments of the present application can be integrated in one processing unit, or each unit can be a physically independent unit, or two or more units can be integrated in one unit. The integrated unit can be implemented in the form of hardware or in the form of a software functional unit.

[0152] The integrated module / unit, if implemented in the form of a software functional unit and sold or used as an independent product, can be stored in a computer readable storage medium. Based on such understanding, all or part of the flow of the above-mentioned embodiment methods can be completed by a computer program instructing related hardware, and the computer program can be stored in a computer readable storage medium. When the processor executes the computer program, the steps of each method embodiment described above can be implemented. The computer program includes computer program code, which can be in the form of source code, object code, executable file or some intermediate form. The computer readable medium can include any entity or device capable of carrying the computer program code, recording medium, U disk, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signal, telecommunication signal and software distribution medium, etc. It should be noted that the computer readable medium can include appropriate contents according to the requirements of legislation and patent practice in the jurisdiction, for example, in some jurisdictions, according to legislation and patent practice, the computer readable medium does not include electrical carrier signals and telecommunication signals.

[0153] The above-described embodiments are only used to illustrate the technical solutions of the present application, but not limit them; although the present application is described in detail with reference to the foregoing embodiments, those skilled in the art should understand that: it can still modify the technical solutions recorded in the foregoing embodiments, or make equivalent replacement to part of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application, and should be included in the protection scope of the present application.

Claims

1. A method of assisted detection of the digestive tract, characterized in that, The method comprises: obtaining a historical three-dimensional model of the digestive tract, the historical three-dimensional model being obtained by three-dimensional reconstruction based on historical digestive tract images, the historical three-dimensional model being marked with historical position information and historical cleanliness information at a plurality of position points of the digestive tract; obtaining a current digestive tract image taken at a current position point; determining corresponding current position information of the current digestive tract image in the digestive tract; determining target position information associated with the current position information from the historical position information, and determining the historical cleanliness information associated with the target position information; in a target three-dimensional model of the digestive tract, marking a position point corresponding to the target position information, the target three-dimensional model being marked with auxiliary information obtained according to the historical cleanliness information associated with the target position information.

2. The method of assisted detection of the digestive tract according to claim 1, characterized in that, Before the marking of the position point corresponding to the target position information in the target three-dimensional model of the digestive tract, the auxiliary detection method of the digestive tract further comprises: registering a first three-dimensional model according to the current position information and the historical position information to obtain the target three-dimensional model; the first three-dimensional model is the historical three-dimensional model or a three-dimensional model obtained by three-dimensional reconstruction according to the current digestive tract image.

3. The method of assisted detection of the digestive tract according to claim 2, characterized in that, The current digestive tract image and the historical digestive tract image are both images taken by an endoscope; The registering of the first three-dimensional model according to the current position information and the historical position information to obtain the target three-dimensional model comprises: determining a first moving speed according to the historical position information, the first moving speed being a moving speed of the endoscope when taking the historical digestive tract image; determining a second moving speed according to the current position information, the second moving speed being a moving speed of the endoscope when taking the current digestive tract image; adjusting the first three-dimensional model according to the first moving speed and the second moving speed to obtain the target three-dimensional model.

4. The method of assisted detection of the digestive tract according to claim 2, characterized in that, The registering of the first three-dimensional model according to the current position information and the historical position information to obtain the target three-dimensional model comprises: determining a first length of a digestive tract segment in the digestive tract when taking the historical digestive tract image according to the historical position information; determining a second length of the digestive tract segment when taking the current digestive tract image according to the current position information; adjusting the first three-dimensional model according to the first length and the second length to obtain the target three-dimensional model.

5. The method of assisted detection of the digestive tract according to claim 4, characterized in that, The current digestive tract image is an image taken by an endoscope; After the obtaining of the current digestive tract image taken at a current position point, the auxiliary detection method of the digestive tract further comprises: detecting whether the endoscope has passed through the digestive tract segment.

6. The method of assisted detection of the digestive tract according to any one of claims 1 to 5, characterized in that, The marking of the position point corresponding to the target position information in the target three-dimensional model of the digestive tract comprises: taking the historical three-dimensional model as the target three-dimensional model; marking the position point corresponding to the target position information in the target three-dimensional model.

7. The method according to any one of claims 1 to 5, wherein the method is a method for assisting detection of the digestive tract. The target three-dimensional model is a model obtained based on images collected in an endoscope insertion process, and the current digestive tract image is an image obtained by shooting in an endoscope withdrawal process; After the current digestive tract image obtained by shooting at the current position point is acquired, the auxiliary detection method of the digestive tract further includes: optimizing the target three-dimensional model according to the current digestive tract image.

8. An apparatus for assisting detection of the digestive tract, characterized by comprising: The auxiliary detection device of the digestive tract is configured in an image processing device, and includes: a model acquisition unit configured to acquire a historical three-dimensional model of the digestive tract, the historical three-dimensional model being obtained by three-dimensional reconstruction based on historical digestive tract images, and the historical three-dimensional model being marked with historical position information and historical cleanliness information at a plurality of position points of the digestive tract; an image acquisition unit configured to acquire a current digestive tract image obtained by shooting at a current position point; a first determination unit configured to determine current position information corresponding to the current digestive tract image in the digestive tract; a second determination unit configured to determine target position information associated with the current position information from the historical position information, and determine historical cleanliness information associated with the target position information; a model marking unit configured to mark a position point corresponding to the target position information in a target three-dimensional model of the digestive tract, and mark auxiliary information obtained according to the historical cleanliness information associated with the target position information in the target three-dimensional model.

9. An image processing apparatus comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, The processor executes the computer program to implement the steps of the auxiliary detection method of the digestive tract according to any one of claims 1 to 7.

10. An endoscope system characterized by comprising: It includes: a light source device, an endoscope, an image processing device, and a display; The image processing device is configured to execute the steps of the auxiliary detection method of the digestive tract according to any one of claims 1 to 7.

Images