Auxiliary detection method, device and equipment for digestive tract and endoscope system

Abstract

The invention is suitable for the technical field of medical treatment, and provides an auxiliary detection method, device and equipment for a digestive tract and an endoscope system. The auxiliary detection method for the alimentary canal comprises the steps that a historical three-dimensional model, obtained through three-dimensional reconstruction based on a historical alimentary canal image, of the alimentary canal is obtained, and historical position information and historical cleanliness information of multiple position points of the alimentary canal are marked on the historical three-dimensional model; obtaining a current alimentary canal image obtained by shooting the current position point; determining corresponding current position information of the current alimentary canal image in the alimentary canal; target position information associated with the current position information is determined from the historical position information, and associated historical cleanliness information is determined; in a target three-dimensional model of the digestive tract, marking a position point corresponding to the target position information, and marking auxiliary information obtained according to historical cleanliness information associated with the target position information in the target three-dimensional model. The embodiment of the invention can assist in improving the detection efficiency of endoscope detection.

Description

Technical Field

[0001] The present application belongs to the field of medical technology, and in particular relates to an auxiliary detection method, device, equipment and endoscope system for the digestive tract. Background Art

[0002] Intestinal health is related to people's quality of life. At present, the mainstream method for doctors to examine and treat intestinal health problems is to use an endoscope for colonoscopy (hereinafter referred to as endoscopy). Through endoscopy, adenomas can be detected and precancerous lesions can be removed at an early stage, reducing the risk of death from rectal cancer. The quality of intestinal preparation has a great impact on the detection rate of adenomas. Compared with low-quality intestinal preparation, high-quality intestinal preparation can significantly improve this detection rate. Therefore, intestinal preparation has become an important indicator for evaluating the quality of endoscopic examinations.

[0003] In the related art, before endoscopic examination, the intestine needs to be cleaned, including removing feces, digestive residual fluids, and foreign objects in the intestine. Usually, the intestine can be prepared by fasting in advance and taking intestinal cleansers orally before endoscopic examination. However, in actual applications, some patients have insufficient intestinal preparation, which makes it impossible for doctors to perform colonoscopy normally. At this time, the usual practice is to stop the endoscopic examination and withdraw the scope.

[0004] When the endoscopic examination is stopped and the endoscope is withdrawn, the patient needs to wait for the endoscope to be reinserted after completing the intestinal preparation, and the entire intestine needs to be re-endoscopically examined, which results in low detection efficiency. Summary of the invention

[0005] The embodiments of the present application provide a method, device, equipment and endoscope system for auxiliary detection of the digestive tract, which can improve the detection efficiency during endoscopic examination.

[0006] A first aspect of an embodiment of the present application provides an auxiliary detection method for the digestive tract, which is applied to an image processing device. The auxiliary detection method for the digestive tract includes: obtaining a historical three-dimensional model of the digestive tract, the historical three-dimensional model is obtained by three-dimensional reconstruction based on historical digestive tract images, and the historical three-dimensional model is marked with historical position information and historical cleanliness information at multiple position points of the digestive tract; obtaining a current digestive tract image obtained by photographing a current position point; determining current position information corresponding to the current digestive tract image in the digestive tract; determining target position information associated with the current position information from the historical position information, and determining the historical cleanliness information associated with the target position information; marking the position point corresponding to the target position information in the target three-dimensional model of the digestive tract, and the target three-dimensional model is marked with auxiliary information obtained according to the historical cleanliness information associated with the target position information.

[0007] In some embodiments of the first aspect, in the target three-dimensional model of the digestive tract, before marking the position point corresponding to the target position information, the auxiliary detection method of the digestive tract also includes: aligning a first three-dimensional model according to the current position information and the historical position information to obtain the target three-dimensional model; the first three-dimensional model is the historical three-dimensional model or the three-dimensional model obtained by three-dimensional reconstruction based on the current digestive tract image.

[0008] In some embodiments of the first aspect, the current digestive tract image and the historical digestive tract image are both images taken by an endoscope; and the first three-dimensional model is registered according to the current position information and the historical position information to obtain the target three-dimensional model, including: determining a first moving speed according to the historical position information, the first moving speed being the moving speed of the endoscope when taking the historical digestive tract image; determining a second moving speed according to the current position information, the second moving speed being the moving speed of the endoscope when taking the current digestive tract image; and adjusting the first three-dimensional model according to the first moving speed and the second moving speed to obtain the target three-dimensional model.

[0009] In some embodiments of the first aspect, the registering the first three-dimensional model according to the current position information and the historical position information to obtain the target three-dimensional model includes: determining a first length of a digestive tract segment in the digestive tract when the historical digestive tract image is taken according to the historical position information; determining a second length of the digestive tract segment when the current digestive tract image is taken according to the current position information; and adjusting the first three-dimensional model according to the first length and the second length to obtain the target three-dimensional model.

[0010] In some embodiments of the first aspect, the current digestive tract image is an image taken by an endoscope; after obtaining the current digestive tract image taken at the current position point, the auxiliary detection method of the digestive tract also includes: detecting whether the endoscope has passed through the digestive tract segment.

[0011] In some embodiments of the first aspect, marking the position points corresponding to the target position information in the target three-dimensional model of the digestive tract includes: aligning the target three-dimensional model with the historical three-dimensional model; and marking the position points corresponding to the target position information in the target three-dimensional model according to the alignment result.

[0012] In some embodiments of the first aspect, the position points corresponding to the target position information are marked in the target three-dimensional model of the digestive tract, including: using the historical three-dimensional model as the target three-dimensional model; and marking the position points corresponding to the target position information in the target three-dimensional model.

[0013] In some embodiments of the first aspect, after acquiring the current digestive tract image captured at the current position point, the auxiliary detection method of the digestive tract further includes: performing a cleanliness assessment based on the current digestive tract image to obtain current cleanliness information corresponding to the current digestive tract image in the digestive tract.

[0014] In some embodiments of the first aspect, after marking the position points corresponding to the target position information in the target three-dimensional model of the digestive tract, the auxiliary detection method of the digestive tract further includes: sending the target three-dimensional model to a display.

[0015] In some embodiments of the first aspect, the target three-dimensional model is a model obtained based on images collected during the advancement of the endoscope, and the current digestive tract image is an image taken during the withdrawal of the endoscope; after obtaining the current digestive tract image taken at the current position point, the auxiliary detection method of the digestive tract also includes: optimizing the target three-dimensional model based on the current digestive tract image.

[0016] In some embodiments of the first aspect, the historical three-dimensional model is also marked with the proportion of the target digestive tract site in the digestive tract, and the target digestive tract site is the digestive tract site whose corresponding historical cleanliness information meets a preset cleanliness condition.

[0017] A second aspect of an embodiment of the present application provides an auxiliary detection device for the digestive tract, which is configured in an image processing device, and the auxiliary detection device for the digestive tract includes: a model acquisition unit, used to acquire a historical three-dimensional model of the digestive tract, the historical three-dimensional model is obtained by three-dimensional reconstruction based on historical digestive tract images, and the historical three-dimensional model is marked with historical position information and historical cleanliness information at multiple position points of the digestive tract; an image acquisition unit, used to acquire a current digestive tract image obtained by photographing a current position point; a first determination unit, used to determine the current position information corresponding to the current digestive tract image in the digestive tract; a second determination unit, used to determine target position information associated with the current position information from the historical position information, and determine the historical cleanliness information associated with the target position information; a model marking unit, used to mark the position point corresponding to the target position information in the target three-dimensional model of the digestive tract, and the target three-dimensional model is marked with auxiliary information obtained according to the historical cleanliness information associated with the target position information.

[0018] A third aspect of an embodiment of the present application provides an image processing device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the above-mentioned auxiliary detection method of the digestive tract when executing the computer program.

[0019] A fourth aspect of an embodiment of the present application provides an endoscope system, comprising a light source device, an endoscope, an image processing device and a display; wherein the image processing device is used to execute the steps of the auxiliary detection method of the digestive tract as described in the first aspect.

[0020] A fifth aspect of an embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the above-mentioned auxiliary detection method of the digestive tract are implemented.

[0021] A sixth aspect of an embodiment of the present application provides a computer program product, which, when executed on an image processing device, enables the image processing device to execute the steps of the above-mentioned auxiliary detection method of the digestive tract.

[0022] In the implementation mode of the present application, during the first endoscopic examination, a historical three-dimensional model is obtained by three-dimensional reconstruction based on the historical digestive tract image, and the historical position information and historical cleanliness information of the digestive tract part are marked, so as to first detect the digestive tract part that can be detected. During the second endoscopic examination, the current digestive tract image is obtained to determine the current position information corresponding to the current digestive tract image in the digestive tract, and the target position information associated with the current position information and the historical cleanliness information associated with the target position information are determined from the historical position information, and then the position point corresponding to the target position information is marked in the target three-dimensional model of the digestive tract. Since the target three-dimensional model is marked with auxiliary information obtained according to the historical cleanliness information associated with the target position information, the doctor can obtain the historical cleanliness information of the corresponding position point during the first endoscopic examination based on the auxiliary information in the target three-dimensional model, and then skip the digestive tract part that has been detected during the first endoscopic examination, without the need to re-detect the entire digestive tract, thereby improving the detection efficiency of endoscopic detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0024] Figure 1is a schematic structural diagram of an endoscope system provided in an embodiment of the present application;

[0025] Figure 2 It is a schematic diagram of the implementation process of a digestive tract auxiliary detection method provided in an embodiment of the present application;

[0026] Figure 3 Schematic diagram of the intestine corresponding to different cleanliness scores provided in the embodiments of the present application;

[0027] Figure 4 is a schematic diagram of the structure of the intestine provided in an embodiment of the present application;

[0028] Figure 5 This is a first schematic diagram for marking position points and auxiliary information provided by an embodiment of the present application;

[0029] Figure 6 This is a second schematic diagram of marking position points and auxiliary information provided in an embodiment of the present application;

[0030] Figure 7 is a schematic diagram of a first implementation process of registering a first three-dimensional model provided in an embodiment of the present application;

[0031] Figure 8 is a schematic diagram of a second implementation process of registering a first three-dimensional model provided in an embodiment of the present application;

[0032] Fig. 9 It is a schematic diagram of the structure of an auxiliary detection device for the digestive tract provided in an embodiment of the present application;

[0033] Fig.10 It is a structural schematic diagram of the image processing device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0034] In order to make the purpose, technical solutions and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not intended to limit the present application. Based on the embodiments of the present application, all other embodiments obtained by those skilled in the art without making creative work are protected by the present application.

[0035] It should be understood that when used in the present specification and the appended claims, the term "comprising" indicates the presence of described features, integers, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or combinations thereof.

[0036] In the description of the present application specification and the appended claims, the terms "first", "second", "third", etc. are only used to distinguish the descriptions and cannot be understood as indicating or implying relative importance.

[0037] References to "one embodiment" or "some embodiments" etc. described in the specification of this application mean that one or more embodiments of the present application include specific features, structures or characteristics described in conjunction with the embodiment. Therefore, the statements "in one embodiment", "in some embodiments", "in some other embodiments", "in some other embodiments", etc. that appear in different places in this specification do not necessarily refer to the same embodiment, but mean "one or more but not all embodiments", unless otherwise specifically emphasized in other ways. The terms "including", "comprising", "having" and their variations all mean "including but not limited to", unless otherwise specifically emphasized in other ways.

[0038] In the related art, before endoscopic examination, the intestine needs to be cleaned, including removing feces, digestive residual fluids, and foreign objects in the intestine. Usually, the intestine can be prepared by fasting in advance and taking intestinal cleansers orally before endoscopic examination. However, in actual applications, some patients have insufficient intestinal preparation, which makes it impossible for doctors to perform colonoscopy normally. At this time, the usual practice is to stop the endoscopic examination and withdraw the scope.

[0039] When the endoscopic examination is stopped and the endoscope is withdrawn, the patient needs to wait for the endoscope to be reinserted after completing the intestinal preparation, and the entire intestine needs to be re-endoscopically examined, which results in low detection efficiency.

[0040] In view of this, the present application proposes an auxiliary detection method for the digestive tract, which can refer to the historical three-dimensional model and mark auxiliary information in the target three-dimensional model. The auxiliary information is related to the historical cleanliness information of the corresponding position. The doctor can obtain the historical cleanliness information of the corresponding position point during the first endoscopic examination based on the auxiliary information, and then choose whether to skip the digestive tract part that has been inspected during the first endoscopic examination according to needs, without the need to re-inspect the entire digestive tract, thereby improving the detection efficiency of endoscopic examination.

[0041] In order to illustrate the technical solution of the present application, a specific embodiment is provided below for illustration.

[0042] Please refer to Figure 1 , Figure 1 1 is a schematic diagram of an endoscope system provided in an embodiment of the present application. The endoscope system may include: a light source device 101 , an endoscope 102 (also referred to as a mirror body), an image processing device 103 and a display 104 .

[0043] The light source device 101 may include a light source and an optical element, wherein the light source may generate a light beam of a specific wavelength, and the optical element includes but is not limited to a collimating lens, a focusing lens, etc. The light source device 101 may be used to converge the light beam generated by the light source and inject it into the end face of the light guide of the endoscope 102, so that the light beam is emitted from the front end of the endoscope 102, and provides illumination light for the endoscope 102.

[0044] The endoscope 102 may include an insertion portion in an elongated shape. The insertion portion may be equipped with a built-in imaging element. The endoscope 102 may collect images of the front end of the endoscope 102 through the imaging element, and feed the collected images back to the image processing device 103 through a connecting line. More specifically, the doctor may insert the insertion portion into the patient's body and collect images of the patient's body through the imaging element. It should be understood that an operating portion may also be connected to one end of the endoscope 102, and the operating portion may generate a control signal for controlling the insertion portion under the operation of the doctor.

[0045] The image processing device 103 may be a computer or a dedicated processing device for an endoscope system. The image processing device 103 can perform image processing based on the image collected by the endoscope 102 and send the image processing result to the display 104. It is understood that image processing includes but is not limited to: image recognition, image cropping, and image enhancement.

[0046] The display 104 may be a liquid crystal display (LCD), a light-emitting diode (LED) display or other display devices with display capabilities. The display 104 may display according to the image processing result provided by the image processing device 103 .

[0047] In some examples, the display 104 can display in real time the image captured by the camera element of the endoscope 102 and the recognition result of the image processing device 103 performing image recognition (such as lesion recognition). In other examples, the display 104 can also display the three-dimensional model obtained by the image processing device 103 performing three-dimensional reconstruction based on the image.

[0048] It should be noted that Figure 1 This does not constitute a limitation on the endoscope system. In actual applications, the endoscope system may include more or fewer components than shown in the figure, or a combination of certain components, or different components. For example, the endoscope system may also include a power supply device, an AI processing device specifically used to execute an artificial intelligence (AI) algorithm, etc., which is not limited in this application.

[0049] To solve the problem of low detection efficiency during endoscopic examination in related technologies, please refer to Figure 2 , Figure 2 A schematic diagram of the implementation process of an auxiliary detection method of the digestive tract provided in an embodiment of the present application is shown. The method can be applied to the aforementioned image processing device 103 and can be suitable for situations where the detection efficiency during endoscopic examination needs to be improved.

[0050] Specifically, the above-mentioned auxiliary detection method of the digestive tract may include the following steps S201 to S205.

[0051] Step S201, obtaining a historical three-dimensional model of the digestive tract.

[0052] In the embodiments of the present application, the digestive tract refers to any part of the digestive system, including but not limited to the intestines, esophagus, and throat.

[0053] In an embodiment of the present application, the historical three-dimensional model can be obtained by three-dimensional reconstruction based on historical digestive tract images. The historical digestive tract images can refer to images obtained by photographing the digestive tract through the camera element of the endoscope 102 during a previous endoscopic examination (e.g., a previous endoscopic examination). Based on the historical digestive tract images, a three-dimensional reconstruction can be performed to obtain a historical three-dimensional model of the digestive tract. The historical three-dimensional model can show the condition of the digestive tract during previous endoscopic examinations.

[0054] It should be understood that the historical three-dimensional model may be stored in the memory of the image processing device 103 or may be input to the image processing device 103 by a user, and this application does not impose any limitation on this.

[0055] In the implementation of the present application, the historical three-dimensional model may be marked with historical location information and historical cleanliness information at multiple locations of the digestive tract. The location point refers to a point inside the digestive tract, and may refer to a marking point on a specific part of the digestive tract. Taking the intestine as an example, the location point may be a corresponding marking point on each intestinal segment.

[0056] The historical position information is used to characterize the position of each position point on the digestive tract during the process of collecting historical digestive tract images. The historical position information can be represented in the form of absolute coordinates, relative coordinates, etc. In the implementation of the present application, the historical position information can be obtained by identifying the historical digestive tract images, or can be obtained by real-time recording by the built-in positioning module of the endoscope 102, and the present application does not limit this.

[0057] Historical cleanliness information is used to characterize the cleanliness of each location point during the acquisition of historical digestive tract images. Historical cleanliness information can be expressed as a score or rating. Taking the intestine as an example, the cleanliness of the location points can be scored according to the Boston Bowel Preparation Scale (BBPS). The Boston Scale rates cleanliness from poor to good as 0, 1, 2, and 3 points. Please refer to Figure 3 , Figure 3 A corresponds to a score of 0, indicating that the entire intestinal mucosa cannot be observed due to solid or liquid feces that cannot be removed; B corresponds to a score of 1, indicating that stains, turbid liquid, and residual feces make part of the mucosa unable to be observed; C corresponds to a score of 2, and D corresponds to a score of 3. These two situations indicate that the intestine is relatively clean and the doctor can perform an endoscopic examination. Historical cleanliness information can be obtained by identifying historical digestive tract images. The recognition method can use a convolutional neural network model, a fully connected neural network model, etc., which is not limited by this application.

[0058] Step S202, obtaining a current digestive tract image photographed at the current position point.

[0059] The current digestive tract image is the image currently captured by the imaging element of the endoscope 102, and the current position point may represent the point in the digestive tract currently captured by the imaging element of the endoscope 102. It is understandable that the endoscope 102 may move in the patient's body, and the current digestive tract image may specifically include images continuously captured during the movement of the endoscope 102.

[0060] Step S203, determining the current position information corresponding to the current digestive tract image in the digestive tract.

[0061] The current position information may represent the position of the current position point. Similarly, the current position information may be obtained based on the current digestive tract image recognition, or based on the built-in positioning module of the endoscope 102 .

[0062] Step S204: determining target location information associated with the current location information from the historical location information, and determining historical cleanliness information associated with the target location information.

[0063] Specifically, the image processing device 103 may determine the current position point corresponding to the position point during the past endoscopic examination based on the current position information, and then determine the historical position information of the position point in the historical position information, and use it as the target position information associated with the current position information.

[0064] After obtaining the target position information, the image processing device 103 may use the historical cleanliness information at the same position point as the historical cleanliness information associated with the target position information.

[0065] Step S205: marking the position points corresponding to the target position information in the target three-dimensional model of the digestive tract.

[0066] The target 3D model may be the aforementioned historical 3D model or a new 3D model, which is not limited in this application. When the target 3D model is a new 3D model, the new 3D model is different from the historical 3D model and can be used to display the condition of the digestive tract during this endoscopic examination.

[0067] In the implementation of the present application, the image processing device 103 may mark the position points corresponding to the target position information in the target three-dimensional model by using a specific style of identification. The specific style of identification may include but is not limited to: identification of a specific color and identification of a specific pattern.

[0068] In an embodiment of the present application, the target three-dimensional model is also marked with auxiliary information obtained according to the historical cleanliness information associated with the target position information. The auxiliary information can be used to identify the cleanliness of the position point corresponding to the target position information, or whether the position point corresponding to the target position information has been inspected during the past endoscopic examination.

[0069] Please refer to Figure 4 , Figure 4 FIG. 1 is a schematic diagram of the intestinal structure to which the auxiliary detection method of this embodiment is applied. Figure 4 As shown, between the ascending colon 401 and the transverse colon 403, there is the right flexure of the colon 402; between the transverse colon 403 and the descending colon-sigmoid colon 405, there is the left flexure of the colon 404; in addition, the intestine also includes the appendix, cecum, ileum, rectum and anal canal (not shown in the figure).

[0070] As an example, Figure 5 As shown, the position point corresponding to the target position information can be marked in the target three-dimensional model of the intestine by a dot mark 51. Then, the uninspected area can be determined based on the historical cleanliness information. For example, based on the historical cleanliness information corresponding to the position point, if the cleanliness score is lower than the score threshold, it is regarded as an uninspected area, and an identification box 52 is generated in the target three-dimensional model as auxiliary information, and the area within the identification box 52 is the uninspected area.

[0071] As another example, Figure 6 As shown, the position point corresponding to the target position information can be marked in the target three-dimensional model of the intestine by a dot mark 61, and then an information box 62 is generated in the target three-dimensional model as auxiliary information, and the information box 62 can display the cleanliness score corresponding to the historical cleanliness information. It can be understood that when the cleanliness score is lower than a certain threshold, such as the BBPS cleanliness score of the intestinal segment is lower than 2 points, the position is an uninspected area.

[0072] In practical applications, the marking method of location points and auxiliary information can also be adjusted according to the needs of doctors or patients, and this application does not impose any restrictions on this.

[0073] In the implementation of the present application, during the first endoscopic examination, a historical three-dimensional model is obtained by performing three-dimensional reconstruction based on the historical digestive tract image, and the historical position information and historical cleanliness information of the digestive tract part are marked, so as to first detect the digestive tract part that can be detected. During the second endoscopic examination, the current digestive tract image is obtained to determine the current position information corresponding to the current digestive tract image in the digestive tract, and the target position information associated with the current position information and the historical cleanliness information associated with the target position information are determined from the historical position information, and then the position point corresponding to the target position information is marked in the target three-dimensional model. Since the target three-dimensional model is marked with auxiliary information obtained according to the historical cleanliness information associated with the target position information, the doctor can obtain the historical cleanliness information of the corresponding position point during the first endoscopic examination based on the auxiliary information in the target three-dimensional model, and then skip the digestive tract part that has been detected during the first endoscopic examination, without the need to re-detect the entire digestive tract, thereby improving the detection efficiency of endoscopic detection.

[0074] In some embodiments, the target three-dimensional model may be obtained based on a historical three-dimensional model, or may be obtained based on a three-dimensional model reconstructed from a digestive tract image acquired during the current examination.

[0075] Specifically, in the target three-dimensional model of the digestive tract, before marking the position point corresponding to the target position information, the image processing device 103 can register the first three-dimensional model according to the current position information and the historical position information to obtain the target three-dimensional model. The first three-dimensional model can be a historical three-dimensional model, or a three-dimensional model obtained by three-dimensional reconstruction based on the digestive tract image acquired during the current inspection (including the current digestive tract image acquired in real time, and the digestive tract image acquired before this during the current inspection).

[0076] The goal of registration is to solve the mapping relationship between the two models. In short, since the digestive tract such as the intestines will move, the digestive tract will have certain changes in length and spatial position. In the implementation of the present application, through registration, the change relationship between the digestive tract during the past endoscopic examination presented by the historical three-dimensional model and the current digestive tract can be determined to obtain the target three-dimensional model.

[0077] Taking the first three-dimensional model as the historical three-dimensional model as an example, through registration, the position points on the historical three-dimensional model can be aligned to the new position points to obtain the target three-dimensional model. In other words, a new three-dimensional model can be obtained by adjusting the old three-dimensional model established during the previous endoscopic examination. Correspondingly, if the historical cleanliness information at a certain position point in the historical three-dimensional model does not meet the preset cleanliness conditions, for example, the corresponding cleanliness score is lower than the score threshold, then the position point corresponding to the position point in the new three-dimensional model is a position point that has not been inspected.

[0078] Taking the first 3D model as a 3D model obtained by 3D reconstruction based on the current digestive tract image as an example, through registration, the position points on the first 3D model can be aligned to the historical 3D model to know whether each position point on the first 3D model needs to be inspected. Specifically, according to the correspondence between a certain position point in the first 3D model and a position point on the historical 3D model, if the historical cleanliness information does not meet the preset cleanliness condition, for example, the corresponding cleanliness score is lower than the score threshold, then the position point in the first 3D model is a position point that has not been inspected.

[0079] Specifically, in some embodiments of the present application, the current digestive tract image and the historical digestive tract image are both images captured by the endoscope 102. Figure 7 The above-mentioned registration of the first three-dimensional model to obtain the target three-dimensional model may include the following steps S701 to S703.

[0080] Step S701, determining a first moving speed according to historical position information.

[0081] The first moving speed is the moving speed of the endoscope 102 when capturing historical digestive tract images.

[0082] Step S702: determine a second moving speed according to the current position information.

[0083] The second moving speed is the moving speed of the endoscope 102 when capturing the current digestive tract image.

[0084] Step S703: adjusting the first three-dimensional model according to the first moving speed and the second moving speed to obtain a target three-dimensional model.

[0085] Specifically, the endoscope 102 can move slowly in the digestive tract under the doctor's operation, and collect digestive tract images during the movement. During the previous endoscopic examination, the endoscope 102 moved in the digestive tract at a first moving speed, and based on the historical position information and the collection time corresponding to the historical position information, the first moving speed of the endoscope 102 when taking the historical digestive tract image can be calculated. Similarly, during the current endoscopic examination, the endoscope 102 moves in the digestive tract at a second moving speed, and based on the current position information and the collection time corresponding to the current position information, the second moving speed of the endoscope 102 when taking the current digestive tract image can be calculated.

[0086] Taking the first three-dimensional model as the historical three-dimensional model as an example, the image processing device 103 can convert the first moving speed and the second moving speed in equal proportion, align the position point on the historical three-dimensional model to the new position point, and obtain the target three-dimensional model. Taking intestinal segment A as an example, if the first moving speed of the endoscope 102 passing through intestinal segment A is v1, and the second moving speed of the endoscope 102 passing through intestinal segment A is v2, then the part of intestinal segment A with an original length of s1 in the historical three-dimensional model can be adjusted to the current length s2 according to the scaling ratio of v2 / v1 to obtain the target three-dimensional model, so that the length of intestinal segment A on the target three-dimensional model matches the current actual length of intestinal segment A.

[0087] Taking the first three-dimensional model as a three-dimensional model obtained by three-dimensional reconstruction based on the current digestive tract image as an example, the image processing device 103 can perform mapping in equal proportion based on the first moving speed and the second moving speed. Taking the intestinal segment A as an example, if the first moving speed of the endoscope 102 passing through the intestinal segment A is v1, and the second moving speed of the endoscope 102 passing through the intestinal segment A is v2, then the part of the intestinal segment A on the first three-dimensional model and the part of the intestinal segment A in the historical three-dimensional model can be mapped to each other according to the scaling ratio of v2 / v1.

[0088] In other embodiments of this application, please refer to Figure 8 The above-mentioned registration of the first three-dimensional model to obtain the target three-dimensional model may also include the following steps S801 to S803.

[0089] Step S801, determining a first length of a digestive tract segment in the digestive tract when the historical digestive tract image is captured according to the historical position information.

[0090] Step S802: determining a second length of the digestive tract segment when the current digestive tract image is captured according to the current position information.

[0091] Step S803: adjusting the first three-dimensional model according to the first length and the second length to obtain a target three-dimensional model.

[0092] The digestive tract segment is a part of the digestive tract. In the embodiments of the present application, the digestive tract segment can specifically refer to a part of the digestive tract. For example, taking the digestive tract as the intestine as an example, the digestive tract segment can refer to Figure 4 The intestinal segments include the middle ascending colon 401, the right flexure of the colon 402, and the transverse colon 403. Of course, the digestive tract segment can also be a segment formed by any two points, and this application does not limit this.

[0093] In an embodiment of the present application, based on the historical position information, the coordinates of the starting point and the end point on the digestive tract segment during the previous endoscopic examination can be determined, and then the first length of the digestive tract segment when the historical digestive tract image is taken can be calculated. Similarly, based on the current position information, the coordinates of the starting point and the end point on the digestive tract segment during the current endoscopic examination can be determined, and then the second length of the digestive tract segment when the current digestive tract image is taken can be calculated.

[0094] Taking the first three-dimensional model as the historical three-dimensional model as an example, based on the first length and the second length, the conversion can be performed in proportion, and the digestive tract segment on the historical three-dimensional model can be aligned to the new position to obtain the target three-dimensional model. Taking intestinal segment A as an example, if the first length of intestinal segment A when the historical digestive tract image is taken is s1, and the second length when the current digestive tract image is taken is s2, then the part of intestinal segment A in the historical three-dimensional model can be adjusted to the current second length s2 according to the scaling ratio of s2 / s1, so that the length of the part of intestinal segment A on the target three-dimensional model matches the current actual length of intestinal segment A.

[0095] Taking the first three-dimensional model as a three-dimensional model obtained by three-dimensional reconstruction based on the current digestive tract image as an example, the image processing device 103 can perform mapping in equal proportion based on the first length and the second length. Taking the intestinal segment A as an example, if the first length of the endoscope 102 passing through the intestinal segment A is s1 and the second length is s2, the part of the intestinal segment A on the first three-dimensional model and the part of the intestinal segment A in the historical three-dimensional model can be mapped to each other according to the scaling ratio of s2 / s1.

[0096] In this way, the obtained target three-dimensional model can provide doctors and patients with more accurate auxiliary detection guidance.

[0097] It should be noted that, after acquiring the current digestive tract image captured at the current position point, the image processing device 103 can detect whether the endoscope 102 has passed through the digestive tract segment.

[0098] For example, whether the endoscope 102 has passed through the digestive tract segment can be detected based on whether the end point of the digestive tract segment is identified. For another example, whether the endoscope 102 has passed through the digestive tract segment can be determined based on the relative positional relationship between the coordinates of the end point of the digestive tract segment and the current coordinates of the endoscope 102.

[0099] If the endoscope 102 has passed through the digestive tract segment, it is convenient to calculate the first length and the second length. Figure 8 In the manner shown, the registration is performed according to the length of the digestive tract segment. If the endoscope 102 does not pass through the digestive tract segment, the Figure 7 In the manner shown, registration is performed according to the moving speed. In this way, the effectiveness of registration can be guaranteed, and registration failure due to failure to identify the complete digestive tract segment can be avoided.

[0100] It should also be noted that endoscopic examination usually includes two stages: advancing the scope and retracting the scope. In actual applications, the above-mentioned alignment process can be implemented based on the current position information collected during the advancement of the scope during this endoscopic examination, or it can be performed in real time based on the current position information collected during the retraction of the scope during this endoscopic examination. This application does not impose any restrictions on this.

[0101] In addition, in the case where the first three-dimensional model is a three-dimensional model obtained by three-dimensionally reconstructing the current digestive tract image, before marking the position point corresponding to the target position information, the image processing device 103 can perform three-dimensional reconstruction based on the current digestive tract image to obtain the first three-dimensional model.

[0102] Among them, the three-dimensional reconstruction method can adopt an image parallax-based reconstruction method, a deep learning-based method, or other existing three-dimensional reconstruction algorithms, which is not limited in this application.

[0103] In order to make the target three-dimensional model more accurate, the first three-dimensional model may be a model obtained based on images collected during the advancement of the endoscope 102, and the current digestive tract image may also include an image captured during the withdrawal of the endoscope 102. After acquiring the current digestive tract image captured at the current position point, the auxiliary detection method of the digestive tract further includes: optimizing the aforementioned target three-dimensional model according to the current digestive tract image captured during the withdrawal of the endoscope 102.

[0104] That is to say, three-dimensional reconstruction and registration can be performed based on the images collected during the advancement of the endoscope 102 to obtain a target three-dimensional model, and then the target three-dimensional model can be further optimized based on the images collected during the withdrawal of the endoscope 102.

[0105] Since the endoscopic examination process generally occurs during the process of withdrawing the endoscope, the movement speed of the endoscope 102 during the process of withdrawing the endoscope is slower than that during the process of advancing the endoscope, so a clearer current digestive tract image can be collected during the process of withdrawing the endoscope. In view of this, based on the clearer current digestive tract image collected during the process of withdrawing the endoscope, the target three-dimensional model can be optimized to make the target three-dimensional model more detailed and accurate.

[0106] Similarly, the historical three-dimensional model can also be obtained by optimizing the initial three-dimensional model based on the historical digestive tract images collected during the retraction process. The initial three-dimensional model is obtained by three-dimensional reconstruction based on the historical digestive tract images collected during the advancement process.

[0107] It is understandable that, whether the historical three-dimensional model is registered or the three-dimensional reconstruction is directly performed based on the current digestive tract image, a target three-dimensional model different from the historical three-dimensional model will be obtained. At this time, in order to mark the position point corresponding to the target position information, the image processing device 103 can align the target three-dimensional model with the historical three-dimensional model, and then mark the position point corresponding to the target position information in the target three-dimensional model according to the alignment result.

[0108] Specifically, by aligning the target 3D model with the historical 3D model, the position of the position point corresponding to the target position information in the historical 3D model can be obtained in the target 3D model, or in other words, the mapping relationship between the positions of the historical 3D model and the target 3D model can be obtained. Based on the mapping relationship, the position point corresponding to the target position information can be marked in the target 3D model, and the auxiliary information can be further marked, so that the doctor can confirm whether the current position point has been detected in the past endoscopic examination based on the auxiliary information and the marking of the position point.

[0109] In some other implementations of the present application, the image processing device 103 may also use the historical three-dimensional model as the target three-dimensional model, and then mark the position points corresponding to the target position information in the target three-dimensional model.

[0110] At this time, the image processing device 103 does not need to generate a new target three-dimensional model, but instead uses the existing historical three-dimensional model to mark the position points corresponding to the target position information on the historical three-dimensional model. This can reduce the high computational complexity caused by three-dimensional reconstruction of the image processing device 103.

[0111] It should be noted that the historical three-dimensional model may also be marked with the proportion of the target digestive tract part in the digestive tract, and the target digestive tract part is the digestive tract part whose corresponding historical cleanliness information meets the preset cleanliness condition.

[0112] Among them, the digestive tract parts that meet the preset cleanliness conditions are also the digestive tract parts that can be examined by endoscopy. Taking the intestine as an example, if the BBPS cleanliness score corresponding to the historical cleanliness information of the intestinal segment is 2 or 3 points, then the intestinal segment can be confirmed as the target digestive tract part.

[0113] The proportion of the target digestive tract part in the digestive tract can specifically refer to the length proportion or the number proportion. The length proportion is the ratio between the length of the target digestive tract part and the total length of the digestive tract. The number proportion is the ratio between the number of the target digestive tract part and the total number of all digestive tract parts in the digestive tract.

[0114] The higher the percentage, the cleaner the digestive tract is and the more adequate the patient's preoperative preparation is; the lower the percentage, the less clean the digestive tract is and the less adequate the patient's preoperative preparation is. Therefore, by marking the percentage in the historical 3D model, doctors can be reminded of the patient's preoperative preparation status, making it easier for doctors to provide preoperative preparation suggestions to patients, so that patients can be more adequately prepared for this endoscopic examination.

[0115] In some embodiments of the present application, the image processing device 103 may also perform a cleanliness assessment based on the current digestive tract image to obtain current cleanliness information corresponding to the current digestive tract image in the digestive tract.

[0116] The method of cleanliness assessment can refer to the description of historical cleanliness information in the previous text, which is not described in detail in this application. Through cleanliness assessment, the image processing device 103 can provide current cleanliness information to assist doctors in determining whether the current cleanliness of the digestive tract meets the requirements of endoscopic examination.

[0117] After obtaining the target three-dimensional model, the image processing device 103 may also send the target three-dimensional model to the display 104. The display 104 may display the target three-dimensional model to visually provide the doctor or patient with the current examination status.

[0118] As a specific example but not limitation, the image processing device 103 can obtain the historical digestive tract images taken by the camera element of the endoscope 102 and the historical position information recorded by the positioning module when the patient undergoes the first endoscopic examination. Then, by scoring the cleanliness of the historical digestive tract images, the corresponding historical cleanliness information is obtained. Based on the three-dimensional reconstruction of the historical digestive tract images, the image processing device 103 can obtain a historical three-dimensional model and mark the historical position information and historical cleanliness information at each position in the historical three-dimensional model. In this process, three-dimensional reconstruction can be performed when the endoscope is inserted during the endoscope examination, and the three-dimensional model can be updated and optimized when the endoscope is withdrawn. During the withdrawal process, the doctor can check the position where the historical cleanliness information meets the preset cleanliness conditions.

[0119] When the patient undergoes a second endoscopic examination, in response to the doctor's operating instructions, the image processing device 103 can read the historical three-dimensional model obtained during the first endoscopic examination stored in the memory. At the same time, the image processing device 103 can obtain the current digestive tract image captured by the camera element of the endoscope 102 and the current position information recorded by the positioning module when the patient undergoes a second endoscopic examination, identify the position point, determine the target position information associated with the current position information, and determine the historical cleanliness information associated with the target position information, and mark it in the target three-dimensional model. The doctor can choose whether to skip the area that has been detected during the first endoscopic examination (that is, the position point corresponding to the historical cleanliness information that meets the preset cleanliness condition) according to the marking of the target position information and auxiliary information in the target three-dimensional model, find the undetected area for detection (that is, the position point corresponding to the historical cleanliness information that does not meet the preset cleanliness condition), thereby improving the efficiency of the endoscopic examination.

[0120] It should be noted that, for the sake of simplicity of description, the aforementioned method embodiments are all expressed as a series of action combinations, but those skilled in the art should know that the present application is not limited to the described order of actions, because according to the present application, certain steps can be performed in other orders.

[0121] like Fig. 9 Shown is a schematic structural diagram of an auxiliary detection device 900 for the digestive tract provided in an embodiment of the present application. The auxiliary detection device 900 for the digestive tract is configured on the image processing device 103.

[0122] Specifically, the auxiliary detection device 900 for the digestive tract may include:

[0123] A model acquisition unit 901 is used to acquire a historical three-dimensional model of the digestive tract, wherein the historical three-dimensional model is obtained by three-dimensional reconstruction based on historical digestive tract images, and the historical three-dimensional model is marked with historical position information and historical cleanliness information at multiple positions of the digestive tract;

[0124] An image acquisition unit 902 is used to acquire a current digestive tract image obtained by photographing a current position point;

[0125] A first determining unit 903 is used to determine current position information corresponding to the current digestive tract image in the digestive tract;

[0126] A second determining unit 904 is used to determine the target location information associated with the current location information from the historical location information, and determine the historical cleanliness information associated with the target location information;

[0127] The model marking unit 905 is used to mark the position points corresponding to the target position information in the target three-dimensional model of the digestive tract, and the target three-dimensional model is marked with auxiliary information obtained according to the historical cleanliness information associated with the target position information.

[0128] In some embodiments of the present application, the auxiliary detection device 900 of the digestive tract may include a model processing unit, which is used to: align the first three-dimensional model according to the current position information and the historical position information to obtain the target three-dimensional model; the first three-dimensional model is the historical three-dimensional model or a three-dimensional model obtained by three-dimensional reconstruction based on the current digestive tract image.

[0129] In some embodiments of the present application, the current digestive tract image and the historical digestive tract image are both images captured by an endoscope 102; the above-mentioned model processing unit can be specifically used to: determine a first moving speed based on the historical position information, the first moving speed being the moving speed of the endoscope 102 when capturing the historical digestive tract image; determine a second moving speed based on the current position information, the second moving speed being the moving speed of the endoscope 102 when capturing the current digestive tract image; and adjust the first three-dimensional model based on the first moving speed and the second moving speed to obtain the target three-dimensional model.

[0130] In some embodiments of the present application, the above-mentioned model processing unit can also be specifically used to: determine, based on the historical position information, a first length of the digestive tract segment in the digestive tract when the historical digestive tract image is taken; determine, based on the current position information, a second length of the digestive tract segment when the current digestive tract image is taken; and adjust the first three-dimensional model based on the first length and the second length to obtain the target three-dimensional model.

[0131] In some embodiments of the present application, the current digestive tract image is an image captured by the endoscope 102; the above-mentioned model processing unit can also be specifically used to detect whether the endoscope 102 has passed through the digestive tract segment.

[0132] In some embodiments of the present application, the model marking unit 905 may be specifically used to: align the target three-dimensional model with the historical three-dimensional model; and mark the position point corresponding to the target position information in the target three-dimensional model according to the alignment result.

[0133] In some implementations of the present application, the model marking unit 905 may be specifically used to: use the historical three-dimensional model as the target three-dimensional model; and mark the position points corresponding to the target position information in the target three-dimensional model.

[0134] In some embodiments of the present application, the auxiliary detection device 900 for the digestive tract may include a cleanliness assessment unit for: performing a cleanliness assessment based on the current digestive tract image to obtain current cleanliness information corresponding to the current digestive tract image in the digestive tract.

[0135] In some embodiments of the present application, the auxiliary detection device 900 for the digestive tract may include a communication unit for sending the target three-dimensional model to the display 104 .

[0136] In some embodiments of the present application, the target three-dimensional model is a model obtained based on images collected during the advancement of the endoscope 102, and the current digestive tract image is an image taken during the withdrawal of the endoscope 102; the above-mentioned model processing unit can also be specifically used to: optimize the target three-dimensional model according to the current digestive tract image.

[0137] In some embodiments of the present application, the historical three-dimensional model also marks the proportion of the target digestive tract site in the digestive tract, and the target digestive tract site is the digestive tract site whose corresponding historical cleanliness information meets a preset cleanliness condition.

[0138] It should be noted that, for the convenience and simplicity of description, the specific working process of the auxiliary detection device 900 of the digestive tract can be referred to Figures 1 to 8 The corresponding process of the method will not be repeated here.

[0139] like Fig.10 , which is a schematic diagram of an image processing device 103 provided in an embodiment of the present application. Specifically, the image processing device 103 may include: a processor 1030, a memory 1031, and a computer program 1032 stored in the memory 1031 and executable on the processor 1030, such as an auxiliary detection program for the digestive tract. When the processor 1030 executes the computer program 1032, the steps in the above-mentioned auxiliary detection method embodiments of the digestive tract are implemented, such as Figure 2 Alternatively, when the processor 1030 executes the computer program 1032, the functions of each module / unit in the above-mentioned device embodiments are realized, for example Fig. 9 The functions of the model acquisition unit 901, the image acquisition unit 902, the first determination unit 903, the second determination unit 904 and the model marking unit 905 are shown.

[0140] The computer program may be divided into one or more modules / units, which are stored in the memory 1031 and executed by the processor 1030 to complete the present application. The one or more modules / units may be a series of computer program instruction segments capable of completing specific functions, which are used to describe the execution process of the computer program in the image processing device.

[0141] For example, the computer program can be divided into: a model acquisition unit, an image acquisition unit, a first determination unit, a second determination unit, and a model marking unit. The specific functions of each unit are as follows: a model acquisition unit, used to acquire a historical three-dimensional model of the digestive tract, the historical three-dimensional model is obtained by three-dimensional reconstruction based on historical digestive tract images, and the historical three-dimensional model is marked with historical position information and historical cleanliness information at multiple positions of the digestive tract; an image acquisition unit, used to acquire a current digestive tract image obtained by photographing a current position point; a first determination unit, used to determine the current position information corresponding to the current digestive tract image in the digestive tract; a second determination unit, used to determine the target position information associated with the current position information from the historical position information, and determine the historical cleanliness information associated with the target position information; a model marking unit, used to mark the position point corresponding to the target position information in the target three-dimensional model of the digestive tract, and the target three-dimensional model is marked with auxiliary information obtained according to the historical cleanliness information associated with the target position information.

[0142] The image processing device 103 may include, but is not limited to, a processor 1030 and a memory 1031. Those skilled in the art will appreciate that Fig.10 It is only an example of an image processing device and does not constitute a limitation of the image processing device. It may include more or fewer components than shown in the figure, or a combination of certain components, or different components. For example, the image processing device may also include input and output devices, network access devices, buses, etc.

[0143] The processor 1030 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), off-the-shelf programmable gate arrays or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc.

[0144] The memory 1031 may be an internal storage unit of the image processing device, such as a hard disk or memory of the image processing device. The memory 1031 may also be an external storage device of the image processing device, such as a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (SecureDigital, SD) card, a flash card (Flash Card), etc. equipped on the image processing device. Further, the memory 1031 may also include both an internal storage unit of the image processing device and an external storage device. The memory 1031 is used to store the computer program and other programs and data required by the image processing device. The memory 1031 may also be used to temporarily store data that has been output or is to be output.

[0145] It should be noted that, for the convenience and brevity of description, the structure of the above-mentioned image processing device 103 can also refer to the specific description of the structure in the method embodiment, and will not be repeated here.

[0146] The technicians in the relevant field can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example for illustration. In practical applications, the above-mentioned function allocation can be completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiment can be integrated in a processing unit, or each unit can exist physically separately, or two or more units can be integrated in one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units. In addition, the specific names of the functional units and modules are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of this application. The specific working process of the units and modules in the above-mentioned system can refer to the corresponding process in the aforementioned method embodiment, which will not be repeated here.

[0147] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0148] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0149] In the embodiments provided in the present application, it should be understood that the disclosed devices / equipment and methods can be implemented in other ways. For example, the device / equipment embodiments described above are merely schematic, for example, the division of the modules or units is only a logical function division, and there may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0150] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0151] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0152] If the integrated module / unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present application implements all or part of the processes in the above-mentioned embodiment method, and can also be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium, and the computer program can implement the steps of the above-mentioned various method embodiments when executed by the processor. Among them, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording medium, U disk, mobile hard disk, disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electric carrier signal, telecommunication signal and software distribution medium. It should be noted that the content contained in the computer-readable medium can be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, computer-readable media do not include electric carrier signals and telecommunication signals.

[0153] The embodiments described above are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, a person skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. Such modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application, and should all be included in the protection scope of the present application.

Claims

1. A digestive tract auxiliary detection method, characterized in that: include: Acquire a historical three-dimensional model of the digestive tract, the historical three-dimensional model being obtained by three-dimensional reconstruction based on historical digestive tract images, the historical three-dimensional model being marked with historical position information and historical cleanliness information at multiple position points of the digestive tract; Obtaining a current digestive tract image obtained by photographing the current position point; Determining current position information corresponding to the current digestive tract image in the digestive tract; Determining target location information associated with the current location information from the historical location information, and determining the historical cleanliness information associated with the target location information; In the target three-dimensional model of the digestive tract, the position points corresponding to the target position information are marked, and the target three-dimensional model is marked with auxiliary information obtained according to the historical cleanliness information associated with the target position information.

2. The auxiliary detection method of the digestive tract according to claim 1, characterized in that: In the target three-dimensional model of the digestive tract, before marking the position point corresponding to the target position information, the auxiliary detection method of the digestive tract further includes: According to the current position information and the historical position information, the first three-dimensional model is registered to obtain the target three-dimensional model; the first three-dimensional model is the historical three-dimensional model or a three-dimensional model obtained by three-dimensional reconstruction based on the current digestive tract image.

3. The auxiliary detection method of the digestive tract according to claim 2, characterized in that: The current digestive tract image and the historical digestive tract image are both images taken by endoscope; The registering the first three-dimensional model according to the current position information and the historical position information to obtain the target three-dimensional model includes: Determine a first moving speed according to the historical position information, where the first moving speed is the moving speed of the endoscope when taking the historical digestive tract image; Determine a second moving speed according to the current position information, where the second moving speed is the moving speed of the endoscope when taking the current digestive tract image; The first three-dimensional model is adjusted according to the first moving speed and the second moving speed to obtain the target three-dimensional model.

4. The auxiliary detection method of the digestive tract according to claim 2, characterized in that: The registering the first three-dimensional model according to the current position information and the historical position information to obtain the target three-dimensional model includes: determining, according to the historical position information, a first length of the digestive tract segment in the digestive tract when the historical digestive tract image is taken; determining, according to the current position information, a second length of the digestive tract segment when the current digestive tract image is captured; The first three-dimensional model is adjusted according to the first length and the second length to obtain the target three-dimensional model.

5. The auxiliary detection method of the digestive tract according to claim 4, characterized in that: The current digestive tract image is an image taken by an endoscope; After acquiring the current digestive tract image obtained by photographing the current position point, the auxiliary detection method of the digestive tract further includes: Detecting whether the endoscope has passed through the digestive tract segment.

6. The auxiliary detection method of the digestive tract according to any one of claims 1 to 5, characterized in that: The step of marking the position points corresponding to the target position information in the target three-dimensional model of the digestive tract includes: Using the historical three-dimensional model as the target three-dimensional model; In the target three-dimensional model, the position points corresponding to the target position information are marked.

7. The auxiliary detection method of the digestive tract according to any one of claims 1 to 5, characterized in that: The target three-dimensional model is a model obtained based on images collected during the endoscope advancement process, and the current digestive tract image is an image taken during the endoscope withdrawal process; After acquiring the current digestive tract image obtained by photographing the current position point, the auxiliary detection method of the digestive tract further includes: The target three-dimensional model is optimized according to the current digestive tract image.

8. An auxiliary detection device for the digestive tract, characterized in that: The auxiliary detection device for the digestive tract, configured in the image processing device, comprises: A model acquisition unit, used to acquire a historical three-dimensional model of the digestive tract, wherein the historical three-dimensional model is obtained by three-dimensional reconstruction based on historical digestive tract images, and the historical three-dimensional model is marked with historical position information and historical cleanliness information at multiple position points of the digestive tract; An image acquisition unit, used to acquire a current digestive tract image obtained by photographing a current position point; A first determining unit, configured to determine current position information corresponding to the current digestive tract image in the digestive tract; A second determining unit is used to determine the target location information associated with the current location information from the historical location information, and determine the historical cleanliness information associated with the target location information; A model marking unit is used to mark the position points corresponding to the target position information in the target three-dimensional model of the digestive tract, and the target three-dimensional model is marked with auxiliary information obtained according to the historical cleanliness information associated with the target position information.

9. An image processing device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps of the auxiliary detection method of the digestive tract as described in any one of claims 1 to 7 are implemented.

10. An endoscope system, characterized in that: include: Light source equipment, endoscopes, image processing equipment and displays; Wherein, the image processing device is used to execute the steps of the auxiliary detection method of the digestive tract as described in any one of claims 1 to 7.

Images