Information prompting method, image processing method, device, equipment and system

Abstract

The invention is suitable for the technical field of medical treatment, and provides an information prompting method, an image processing method, a device, equipment and a system. The information prompting method comprises the following steps: displaying a first alimentary canal image in the process of checking the alimentary canal by using an endoscope; displaying a prompt graph corresponding to a target area in the first alimentary canal image at the image boundary of the first alimentary canal image, the target area including non-effective information; and in response to the operation of the user on the endoscope, updating and displaying a second alimentary canal image, and updating the prompt graph based on the target area condition contained in the second alimentary canal image. According to the embodiment of the invention, the inspection quality of endoscopic inspection can be improved.

Description

Technical Field

[0001] The present application belongs to the field of medical technology, and in particular relates to an information prompting method, an image processing method, a device, a equipment and a system. Background Art

[0002] An endoscope is a diagnostic and auxiliary treatment device that is widely used in clinical practice. The device can enter the human body through natural orifices or surgical incisions, clearly and realistically display the anatomical structure, and assist doctors in seeing lesions in the human body that cannot be displayed by X-rays.

[0003] Endoscopic examination is a technique for examining the human body through an endoscope. The accuracy of the examination results is closely related to the quality of the examination. At present, the clinical indicators for ensuring the quality of endoscopic examinations mainly focus on the anatomical site and the examination time. At the same time, doctors are required to perform spiral retraction during endoscopic examinations to reduce the risk of missed diagnosis. However, whether this method can obtain the information required for endoscopic examinations still depends on the doctor's experience, so it is difficult for related technologies to control the quality of endoscopic examinations. Summary of the invention

[0004] The embodiments of the present application provide an information prompting method, an image processing method, an apparatus, a device and a system, which can assist doctors in improving the quality of endoscopic examinations.

[0005] A first aspect of an embodiment of the present application provides an information prompt method, comprising: displaying a first digestive tract image during an endoscope-based digestive tract examination; displaying a prompt graphic corresponding to a target area in the first digestive tract image at an image boundary of the first digestive tract image, wherein the target area contains non-valid information; updating and displaying a second digestive tract image in response to a user's operation of the endoscope, and updating the prompt graphic based on the target area contained in the second digestive tract image.

[0006] In some embodiments of the first aspect, before displaying a prompt graphic corresponding to a target area in the first digestive tract image at an image boundary of the first digestive tract image, the information prompt method further includes: dividing the first digestive tract image into multiple image areas; and determining the target area from the multiple image areas.

[0007] In some embodiments of the first aspect, dividing the first digestive tract image into multiple image areas includes: identifying the digestive tract cavity in the first digestive tract image to obtain the digestive tract cavity position; calculating the length of a line between the digestive tract cavity position and the image center of the first digestive tract image; if the length is greater than a length threshold, dividing the first digestive tract image into multiple quadrants according to a first coordinate system, each quadrant corresponding to an image area among the multiple image areas, wherein the origin of the first coordinate system is the image center, and the horizontal and vertical axes of the first coordinate system are symmetrically distributed along the line.

[0008] In some embodiments of the first aspect, after calculating the length of the line between the intestinal cavity position and the image center of the first digestive tract image, the information prompt method further includes: if the length is less than or equal to the length threshold, dividing the first digestive tract image into multiple quadrants according to a second coordinate system, each quadrant corresponding to one image area among the multiple image areas, wherein the second coordinate system is the image coordinate system of the first digestive tract image.

[0009] In some embodiments of the first aspect, determining the target area from the multiple image areas includes: performing digestive tract wall detection in each of the multiple image areas, and taking the image area where the digestive tract wall is not detected as the target area; or, performing digestive tract cavity detection in each of the multiple image areas, and taking the image area including the center of the digestive tract cavity as the target area; or, performing digestive tract wall detection in each of the multiple image areas, calculating the digestive tract wall proportion corresponding to each of the multiple image areas, and taking the image area where the digestive tract wall proportion is lower than a proportion threshold as the target area, wherein the digestive tract wall proportion is the ratio of the area between the digestive tract wall area and the corresponding entire image area.

[0010] In some embodiments of the first aspect, the updating of the prompt graphic based on the target area contained in the second digestive tract image includes: determining the digestive tract segment contained in the second digestive tract image; if the digestive tract segment contained in the second digestive tract image is the same as the digestive tract segment contained in the first digestive tract image, updating the prompt graphic based on the target area contained in the second digestive tract image.

[0011] In some embodiments of the first aspect, after determining the digestive tract segment contained in the second digestive tract image, the information prompt method further includes: if the digestive tract segment contained in the second digestive tract image is different from the digestive tract segment contained in the first digestive tract image, generating a new prompt graphic based on the target area contained in the second digestive tract image.

[0012] In some embodiments of the first aspect, the updating of the prompt graphic based on the target area contained in the second digestive tract image includes: if information of the digestive tract wall in the target area is detected according to the second digestive tract image, clearing the prompt graphic.

[0013] In some embodiments of the first aspect, the prompt graphic is updated based on the target area contained in the second digestive tract image, including: if information on the digestive tract wall in a partial sub-area of ​​the target area is detected based on the second digestive tract image, then part of the graphic in the prompt graphic is cleared, and the partial graphic corresponds to the partial sub-area.

[0014] In some embodiments of the first aspect, before updating the prompt graphic based on the target area situation contained in the second digestive tract image, the information prompt method further includes: the information prompt method further includes: detecting whether the second digestive tract image is an information frame.

[0015] In some embodiments of the first aspect, after the prompt graphic is updated based on the target area included in the second digestive tract image, the information prompt method further includes: determining the inspection quality score of the digestive tract inspection using the endoscope according to the number of the prompt graphics and / or the proportion of the image area corresponding to the prompt graphics in the image areas included in all the information frames.

[0016] In some embodiments of the first aspect, the inspection quality score of the digestive tract inspection using the endoscope is determined based on the number of the prompt graphics and / or the proportion of the image area corresponding to the prompt graphics in the image areas included in all the information frames, including: for each digestive tract segment, an initial score of the digestive tract segment is determined based on the number of the prompt graphics and / or the proportion of the image area corresponding to the prompt graphics in the image areas included in all the information frames; and the inspection quality score is determined based on the initial scores of each digestive tract segment.

[0017] A second aspect of an embodiment of the present application provides an image processing method, including: obtaining a first digestive tract image collected during an endoscopic examination of the digestive tract; identifying the digestive tract cavity in the first digestive tract image to obtain a position of the digestive tract cavity; calculating the length of a line between the digestive tract cavity position and the image center of the first digestive tract image; if the length is greater than a length threshold, dividing the first digestive tract image into a plurality of quadrants according to a first coordinate system, wherein the origin of the first coordinate system is the image center, the horizontal and vertical axes of the first coordinate system are symmetrically distributed along the line, and the plurality of quadrants are used to evaluate the inspection quality of the endoscopic inspection process.

[0018] An information prompt device provided in the third aspect of an embodiment of the present application includes: an image display unit, used to display a first digestive tract image during an examination of the digestive tract using an endoscope; a prompt graphic display unit, used to display a prompt graphic corresponding to a target area in the first digestive tract image at an image boundary of the first digestive tract image, wherein the target area contains non-valid information; an update unit, used to update and display a second digestive tract image in response to a user's operation of the endoscope, and to update the prompt graphic based on the target area contained in the second digestive tract image.

[0019] An information prompting device provided in the fourth aspect of an embodiment of the present application includes: an image acquisition unit, used to acquire a first digestive tract image collected during an endoscopic examination of the digestive tract; an identification unit, used to identify the digestive tract cavity in the first digestive tract image to obtain the digestive tract cavity position; a calculation unit, used to calculate the length of a line between the digestive tract cavity position and the image center of the first digestive tract image; an image division unit, used to divide the first digestive tract image into a plurality of quadrants according to a first coordinate system if the length is greater than a length threshold, wherein the origin of the first coordinate system is the image center, the horizontal and vertical axes of the first coordinate system are symmetrically distributed along the line, and the plurality of quadrants are used to evaluate the inspection quality of the endoscopic inspection process.

[0020] A fifth aspect of an embodiment of the present application provides an image processing device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the above-mentioned information prompt method when executing the computer program, or implements the steps of the above-mentioned image processing method when executing the computer program.

[0021] A sixth aspect of an embodiment of the present application provides an endoscope system, comprising: a light source, an endoscope, an image processing device and a display; wherein the image processing device is used to execute the steps of the information prompt method as described in the first aspect, or the image processing device is used to execute the steps of the image processing method as described in the second aspect.

[0022] A seventh aspect of an embodiment of the present application provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, it implements the steps of the above-mentioned information prompt method, or when the computer program is executed by a processor, it implements the steps of the above-mentioned image processing method.

[0023] An eighth aspect of an embodiment of the present application provides a computer program product, which, when the computer program product runs on an image processing device, enables the image processing device to execute the steps of the above-mentioned information prompt method, or, when the computer program product runs on an image processing device, enables the image processing device to execute the steps of the above-mentioned image processing method.

[0024] In an embodiment of the present application, on the one hand, in the process of examining the digestive tract using an endoscope, a first digestive tract image is displayed, and a prompt graphic corresponding to a target area in the first digestive tract image is displayed at the image boundary of the first digestive tract image. Since the target area contains invalid information, the doctor can be prompted with the digestive tract area that has not been effectively inspected. In response to the user's operation of the endoscope, the second digestive tract image is updated and displayed, and the prompt graphic is updated based on the target area contained in the second digestive tract image. The prompt graphic can be updated based on the information obtained by the doctor on the target area, and the prompt graphic can assist the doctor in performing a complete digestive tract examination, thereby helping to improve the quality of endoscopic examination.

[0025] On the other hand, the digestive tract cavity is identified in the first digestive tract image to obtain the position of the digestive tract cavity, and the length of the line between the position of the digestive tract cavity and the image center of the first digestive tract image is calculated. If the length is greater than the length threshold, the first digestive tract image is divided into multiple quadrants according to the first coordinate system to perform quality assessment through multiple quadrants. Since the origin of the first coordinate system is the image center, the horizontal and vertical axes of the first coordinate system are symmetrically distributed along the line, so that the digestive tract cavity can be distributed in the same quadrant as much as possible, avoiding the problem of insufficient accuracy of the quality assessment result caused by taking an area containing a large amount of digestive tract wall as the digestive tract cavity, which helps to accurately achieve quality assessment. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0027] Figure 1 is a schematic structural diagram of an endoscope system provided in an embodiment of the present application;

[0028] Figure 2 This is a schematic diagram of an implementation flow of an information prompt method provided in an embodiment of the present application;

[0029] Figure 3 This is a first schematic diagram of image division and display prompt graphics provided by an embodiment of the present application;

[0030] Figure 4 is a second schematic diagram of a display prompt graphic provided in an embodiment of the present application;

[0031] Figure 5 It is a schematic diagram of a specific implementation process of image segmentation provided in an embodiment of the present application;

[0032] Figure 6 is a second schematic diagram of image division provided in an embodiment of the present application;

[0033] Figure 7 and Figure 8 It is a schematic diagram of the process of processing intestinal images provided in an embodiment of the present application;

[0034] Fig. 9 It is a schematic diagram of an implementation flow of an image processing method provided in an embodiment of the present application;

[0035] Fig.10 It is a structural schematic diagram of an information prompting device provided in an embodiment of the present application;

[0036] Fig.11 is a structural schematic diagram of an image processing device provided in an embodiment of the present application;

[0037] Fig.12 It is a structural schematic diagram of the image processing device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0038] In order to make the purpose, technical solutions and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not intended to limit the present application. Based on the embodiments of the present application, all other embodiments obtained by those skilled in the art without making creative work are protected by the present application.

[0039] It should be understood that when used in the present specification and the appended claims, the term "comprising" indicates the presence of described features, wholes, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, components and / or combinations thereof.

[0040] In the description of the present application specification and the appended claims, the terms "first", "second", "third", etc. are only used to distinguish the descriptions and cannot be understood as indicating or implying relative importance.

[0041] References to "one embodiment" or "some embodiments" etc. described in the specification of this application mean that one or more embodiments of the present application include specific features, structures or characteristics described in conjunction with the embodiment. Therefore, the statements "in one embodiment", "in some embodiments", "in some other embodiments", "in some other embodiments", etc. that appear in different places in this specification do not necessarily refer to the same embodiment, but mean "one or more but not all embodiments", unless otherwise specifically emphasized in other ways. The terms "including", "comprising", "having" and their variations all mean "including but not limited to", unless otherwise specifically emphasized in other ways.

[0042] Endoscopic examination is a technique for examining the human body through an endoscope. The accuracy of the examination results is closely related to the quality of the examination. At present, the clinical indicators for ensuring the quality of endoscopic examinations mainly focus on the anatomical site and examination time. At the same time, doctors are required to perform spiral retraction during endoscopic examinations to reduce the risk of missed diagnosis. However, whether this method can obtain the information required for endoscopic examinations still depends on the doctor's experience. For example, when retracting the spiral endoscope, the doctor's operation is not smooth enough, sometimes fast and sometimes slow. Although the examination time meets the requirements, it is still easy to miss some areas. Therefore, it is difficult for related technologies to control the quality of endoscopic examinations.

[0043] In view of this, the present application proposes an information prompt method and an image processing method. On the one hand, it can assist doctors to improve the completeness of endoscopic examination through graphic prompts to improve the inspection quality of endoscopic examination. On the other hand, through a specific quadrant division method, the inspection quality assessment after endoscopic examination can be made more accurate and the inspection efficiency can be improved.

[0044] In order to illustrate the technical solution of the present application, a specific embodiment is provided below for illustration.

[0045] Please refer to Figure 1 , Figure 1 1 is a schematic diagram of an endoscope system provided in an embodiment of the present application. The endoscope system may include: a light source device 101 , an endoscope 102 (or referred to as a scope), an image processing device 103 and a display 104 .

[0046] The light source device 101 may include a light source and an optical element, wherein the light source may generate a light beam of a specific wavelength, and the optical element includes but is not limited to a collimating lens, a focusing lens, etc. The light source device 101 may be used to converge the light beam generated by the light source and inject it into the end face of the light guide of the endoscope 102, so that the light beam is emitted from the front end of the endoscope 102, and provides illumination light for the endoscope 102.

[0047] The endoscope 102 may include an insertion portion in an elongated shape. The insertion portion may be equipped with a built-in imaging element. The endoscope 102 may capture images of the digestive tract through the imaging element, and feed the captured images back to the image processing device 103. More specifically, the doctor may insert the insertion portion into the patient's body, and capture images of the patient's body through the imaging element. It should be understood that the endoscope 102 also includes an operating portion, and the doctor may control the insertion portion through the operating portion.

[0048] The image processing device 103 may be a computer or a dedicated processing device for an endoscope system. The image processing device 103 can perform image processing based on the image collected by the endoscope 102 and send the image processing result to the display 104. It is understood that image processing includes but is not limited to: image recognition, image cropping, and image enhancement.

[0049] The display 104 may be a liquid crystal display (LCD), a light-emitting diode (LED) display or other display devices with display capabilities. The display 104 may display according to the image processing result provided by the image processing device 103 .

[0050] In some examples, the display 104 can display in real time the image captured by the camera element of the endoscope 102 and the recognition result of the image processing device 103 performing image recognition (such as lesion recognition). In other examples, the display 104 can also display the three-dimensional model obtained by the image processing device 103 performing three-dimensional reconstruction based on the image.

[0051] In an embodiment of the present application, the display 104 may also be used to display a prompt graphic. The prompt graphic may be used to prompt the doctor of a specific area to be examined. Specifically, the prompt graphic may indicate the direction of the target area containing invalid information relative to the endoscope, so that the doctor can change the field of view of the imaging element by operating the endoscope 102 to collect valid information of the target area.

[0052] It should be noted that, in some implementations, the display 104 may be a part of the image processing device 103. It should also be noted that: Figure 1 This does not constitute a limitation on the endoscope system. In actual applications, the endoscope system may include more or fewer components than shown in the figure, or a combination of certain components, or different components. For example, the endoscope system may also include a power supply device, an AI processing device specifically used to execute an artificial intelligence (AI) algorithm, etc., which is not limited in this application.

[0053] Please refer to Figure 2 , Figure 2 A schematic diagram of the implementation flow of an information prompt method provided in an embodiment of the present application is shown. The method can be applied in the aforementioned endoscope system, for example, applied to the image processing device 103, and can be applied to situations where the quality of endoscopic examination needs to be improved.

[0054] Specifically, the above information prompting method may include the following steps S201 to S203.

[0055] Step S201, during the process of examining the digestive tract using an endoscope, a first digestive tract image is displayed.

[0056] In the embodiments of the present application, the digestive tract may be the intestine, esophagus, throat, or other digestive tracts within the digestive system, and the present application does not impose any limitation on this.

[0057] In the embodiment of the present application, the doctor can operate through the aforementioned operating unit to control the camera element on the endoscope 102 to photograph the digestive tract, thereby realizing the spiral endoscope inspection. The first digestive tract image is an image collected by the endoscope 102, and can specifically refer to an image collected in real time by the camera element of the endoscope 102. In the process of inspecting the digestive tract using the endoscope 102, the endoscope system can display the first digestive tract image in real time through the display 104. In this way, the doctor can perform endoscopic inspection by observing the first digestive tract image.

[0058] Step S202: displaying a prompt graphic corresponding to the target area in the first digestive tract image at the image boundary of the first digestive tract image.

[0059] Specifically, the information required for endoscopic examination can be called valid information, and other information other than the valid information can be called non-valid information. As an embodiment of the present application, considering that most gastrointestinal endoscopic examinations require the detection of the digestive tract wall, the digestive tract wall can be selected as valid information, and the content outside the digestive tract wall can be selected as non-valid information, such as the digestive tract cavity and foreign matter. Taking colonoscopy as an example, since colonoscopy usually needs to examine the condition of the intestinal wall, the intestinal wall can be called valid information, and the content outside the intestinal wall can be collectively referred to as non-valid information. At this time, non-valid information includes but is not limited to information such as the intestinal cavity or foreign matter.

[0060] The target area is an image area in the first digestive tract image, and the target area may contain non-valid information. In order to improve the quality of the doctor's examination, the embodiment of the present application will provide a visual image prompt for the target area to inform the doctor of the area that needs to be detected in the current digestive tract image. That is, the doctor is prompted to continue testing for the valid information content that has not been detected, so as to improve the detection quality.

[0061] As an optional embodiment of the present application, in order to improve the efficiency of prompting doctors, in the embodiment of the present application, among the multiple areas included in the digestive tract image, the area whose effective information area ratio is lower than the preset threshold (or the area whose ineffective information area ratio is higher than the preset threshold) can be used as the target area in the embodiment of the present application, thereby screening out the area that needs the doctor's attention and detection.

[0062] As another optional embodiment of the present application, in order to improve the efficiency of prompting doctors, in the embodiment of the present application, a region containing specific non-valid information among multiple regions contained in the digestive tract image can be used as a target region. For example, a region containing the digestive tract cavity and / or a region for rejecting foreign matter can be used as a target region.

[0063] As an optional embodiment of the present application, one or more of the above embodiments may be flexibly selected in combination to select the target area according to actual needs.

[0064] In order to avoid blocking the first digestive tract image and thus preventing the doctor from effectively performing endoscopic examination, in an embodiment of the present application, a prompt graphic corresponding to the target area in the first digestive tract image may be displayed at the image boundary of the first digestive tract image. The prompt graphic may be used to indicate the position of the target area, and specifically may be used to indicate the position of the digestive tract area corresponding to the target area relative to the endoscope.

[0065] It is understandable that the style of the prompt graphic can be selected according to the actual situation. For example, please refer to Figure 3 and Figure 4 , for the target area (the quadrant containing the intestinal cavity) in the same first digestive tract image, Figure 3In the present invention, according to the quadrant boundary of the quadrant where the target area is located, the area within the quadrant boundary can be filled with a specific color (black in the figure, but can also be blue, red or other colors in actual application) at the image boundary of the first digestive tract image to serve as a prompt graphic. Figure 4 In the embodiment, an arrow pattern may be displayed at the image boundary of the first digestive tract image as a prompt graphic, and the arrow direction corresponding to the arrow pattern is the direction from the image center of the first digestive tract image to the target area.

[0066] Step S203, in response to the user's operation on the endoscope, the second digestive tract image is updated and displayed, and the prompt graphic is updated based on the target area contained in the second digestive tract image.

[0067] In the embodiments of the present application, the user's operation of the endoscope 102 includes, but is not limited to, operations such as adjusting the direction of the endoscope through the operating unit, and the doctor's operation of advancing and retracting the endoscope by pushing or pulling the endoscope, as well as other operations that may cause changes in the endoscope's field of view. Through this operation, the endoscope's field of view may change, and a new image may be collected. The new image collected is the second digestive tract image. In other words, the second digestive tract image is a new digestive tract image collected by the doctor's operation after the first digestive tract image during the same endoscopic examination.

[0068] To facilitate endoscopic examinations by doctors, in response to user operations on the endoscope, the endoscope system may update and display the second digestive tract image via the display 104 , that is, the previously displayed first digestive tract image may be replaced with the newly acquired second digestive tract image.

[0069] At the same time, as the endoscope field of view changes, the valid information and ineffectiveness contained in the target area in the second digestive tract image may change. Among them, some valid information that is not displayed in the first digestive tract image may be detected in the second digestive tract image as the doctor operates. That is, it means that this part of the valid information has been effectively detected at this time, so the graphic prompt for this part of the valid information can be cancelled. Based on this, the embodiment of the present application can update the prompt graphics based on the target area contained in the second digestive tract image. For example, all prompt graphics can be eliminated, or, based on the detection of valid information in the target area, the prompt graphic content corresponding to the newly detected valid information is eliminated, and the prompt graphics corresponding to the remaining part of the target area continue to be displayed (that is, part of the prompt graphics is eliminated at this time), and so on. In this way, the update of the prompt graphics can reflect the changes in the number, area and / or proportion of the areas that have not been effectively detected. The more prompt graphics are eliminated, the more areas the doctor has checked for valid information, which means that the more complete the endoscopic examination is and the higher the inspection quality is. The prompt graphics that have not been eliminated can continue to prompt the doctor to continue to detect the valid information (such as the digestive tract wall) that has not been detected in the current target area to further improve the inspection quality.

[0070] In an embodiment of the present application, during the process of using an endoscope to examine the digestive tract, by displaying a first digestive tract image and displaying a prompt graphic corresponding to a target area in the first digestive tract image at the image boundary of the first digestive tract image, since the target area contains invalid information, the doctor can be prompted of the digestive tract area that has not been effectively inspected. In response to the user's operation of the endoscope, the second digestive tract image is updated and displayed, and the prompt graphic is updated based on the target area contained in the second digestive tract image. The prompt graphic can be updated based on the information obtained by the doctor from the target area, and the prompt graphic can assist the doctor in performing a complete digestive tract examination, which helps to improve the quality of endoscopic examination.

[0071] Specifically, before step S102, the first digestive tract image may be divided into a plurality of image regions, so as to determine the target region from the plurality of image regions.

[0072] In the implementation of the present application, the method of dividing the image area can be selected according to the actual situation. For example, the image segmentation can be performed by equal division, threshold segmentation or image recognition to obtain multiple image areas. For each image area, it can be determined whether it belongs to the target area by determining whether it contains valid information or whether it contains invalid information. After the target area is determined, a prompt graphic can be generated and displayed based on the target area.

[0073] like Figure 5As shown, the present application proposes a method for dividing image regions, including the following steps S501 to S503. Figure 2 The update process shown is the same as Figure 2 The illustrated embodiments are applied in combination, so as to achieve the division of the image area before step S102.

[0074] Step S501 : identifying the digestive tract cavity in the first digestive tract image to obtain the digestive tract cavity position.

[0075] The digestive tract cavity refers to the cavity part of the digestive tract, for example, the intestinal cavity, the esophageal cavity, etc. By identifying the digestive tract cavity in the first digestive tract image, the digestive tract cavity position of the digestive tract cavity in the first digestive tract image can be obtained.

[0076] Specifically, in some optional embodiments, the recognition of the digestive tract cavity can be achieved by using deep learning methods such as Faster-RCNN (Faster-Region-Convolutional Neural Networks) and YOLO (You Only Look Once). The recognition results of these methods can be the minimum circumscribed rectangle of the digestive tract cavity. At this time, the image coordinates of the center point in the minimum circumscribed rectangle (that is, the center of the digestive tract cavity) can be used as the position of the digestive tract cavity. Some non-deep learning methods can also be used to identify the digestive tract cavity, which is not limited here.

[0077] Step S502, calculating the length of the line between the digestive tract cavity position and the image center of the first digestive tract image.

[0078] Specifically, the line between the digestive tract cavity position and the image center of the first digestive tract image can represent the direction of the digestive tract cavity relative to the endoscope. Based on the image coordinates of the digestive tract cavity position and the image center, the Euclidean distance between the digestive tract cavity position and the image center can be calculated to obtain the length of the line between the digestive tract cavity position and the image center.

[0079] Step S503: if the length is greater than the length threshold, the first digestive tract image is divided into a plurality of quadrants according to the first coordinate system.

[0080] Each quadrant corresponds to one image region among the multiple image regions.

[0081] In the implementation of the present application, the origin of the first coordinate system is the center of the image, and the horizontal and vertical axes of the first coordinate system are symmetrically distributed along the connecting line. In other words, the first coordinate system can be established with 45° to the left and right of the connecting line as the horizontal and vertical axes, and the first digestive tract image can be divided into multiple image areas according to the first coordinate system.

[0082] For easier understanding, please refer to Figure 3 The intestinal cavity diagram shown in FIG. 1 can be divided into the following: according to the digestive tract cavity position 31 and the image center 32, the length of the line connecting the digestive tract cavity position 31 and the image center 32 is greater than the length threshold (the range shown by the dotted circle), and the following can be divided into the following: Figure 3 In this way, when the digestive tract cavity is far from the center of the image, the digestive tract cavity can be divided into the same quadrant as much as possible.

[0083] Furthermore, since the digestive tract cavity is usually non-valid information, combined with the update process described above, this quadrant division method can make the prompt graphic more accurately reflect the position of the intestinal cavity, thereby dividing the non-valid information into the same quadrant as much as possible. When updating the prompt graphic, the amount of information that needs to be detected in the second digestive tract image is also less, which helps to improve the efficiency of endoscopic examinations. At the same time, the subsequent statistics on the overall endoscopic examination quality are more accurate and effective, which improves the accuracy and effectiveness of endoscopic examination quality control.

[0084] Correspondingly, if the length is less than or equal to the length threshold, the first digestive tract image may be divided into a plurality of quadrants according to the second coordinate system. Similarly, each quadrant corresponds to one of the plurality of image regions.

[0085] The second coordinate system is the image coordinate system of the first digestive tract image. In other words, the first digestive tract image can be divided into a plurality of image regions with the image center of the first digestive tract image as the origin and the right cross as the horizontal and vertical coordinate axes.

[0086] For easier understanding, please refer to Figure 6 The intestinal cavity image shown in the figure can be divided into the following according to the digestive tract cavity position 61 and the image center 62 according to the second coordinate system: Figure 6 In this way, when the digestive tract cavity is close to the center of the image, each quadrant can contain as much effective information as possible.

[0087] After completing the division of the image area, the target area can be determined by identifying effective information or ineffective information.

[0088] Specifically, in some embodiments of the present application, the digestive tract wall detection may be performed in each of the multiple image regions, and the image region where the digestive tract wall is not detected is used as the target region.

[0089] For example, by performing intestinal wall detection on an intestinal image, if the intestinal wall is detected in an image region, the image region contains valid information and does not belong to the target region. If the intestinal wall is not detected, the image region contains invalid information and belongs to the target region.

[0090] In other embodiments, the digestive tract cavity detection may be performed in each of the multiple image regions, and the image region including the center of the digestive tract cavity is used as the target region.

[0091] For example, Figure 3 In the intestinal lumen diagram shown, the quadrant where the center of the intestinal lumen is located can be used as the target area.

[0092] In order to more accurately determine the target area, in other embodiments, the digestive tract wall detection can be performed in each of the multiple image areas, the digestive tract wall proportion corresponding to each of the multiple image areas is calculated, and the image area where the digestive tract wall proportion is lower than the proportion threshold is used as the target area.

[0093] The digestive tract wall ratio is the ratio of the area of ​​the digestive tract wall to the area of ​​the corresponding entire image area. Figure 3 For example, for each quadrant, the area ratio of the intestinal wall in the quadrant can be calculated. If the area ratio is lower than the ratio threshold, it means that there is less valid information in the quadrant and it belongs to the target area.

[0094] It is understandable that other methods of determining the target area through analysis of valid information or invalid information are also applicable to the present application, and the present application does not limit this.

[0095] Considering that the endoscope may pass through different digestive tract segments during the endoscopic examination, and different digestive tract segments usually need to be analyzed separately, in step S203, the digestive tract segment included in the second digestive tract image may be determined first. If the digestive tract segment included in the second digestive tract image is the same as the digestive tract segment included in the first digestive tract image, the prompt graphic is updated based on the target area included in the second digestive tract image.

[0096] The digestive tract segments can be divided according to actual conditions. Usually, they can be divided according to the location of the digestive tract. For example, the intestine can be divided into different intestinal segments such as the ascending colon and the transverse colon, and each intestinal segment can be regarded as a digestive tract segment.

[0097] If the digestive tract segment included in the second digestive tract image is the same as the digestive tract segment included in the first digestive tract image, the second digestive tract image can be used to analyze the target area, and based on the target area, the prompt graphic obtained through the first digestive tract image can be updated.

[0098] Specifically, in some embodiments of the present application, if information of the digestive tract wall in the target area is detected based on the second digestive tract image, the prompt graphic can be cleared.

[0099] Taking into account that the process of updating the endoscope field of view is relatively slow, the second digestive tract image may not be able to completely cover the target area each time. In some embodiments of the present application, if information on the digestive tract wall in a partial sub-area of ​​the target area is detected based on the second digestive tract image, part of the graphics in the prompt graphic are cleared, and the partial graphics correspond to the partial sub-area.

[0100] In other words, the graphics corresponding to the sub-regions in the prompt graphics where valid information is detected can be eliminated, and the prompt graphics of the remaining parts can be retained to guide the doctor to continue to update the visual field and examine the remaining parts. Figure 3 The area of ​​black filling can be reduced in Figure 4 The direction indicated by the arrow can be updated to adapt to the remaining area to be inspected in the target area.

[0101] Correspondingly, if the digestive tract segment included in the second digestive tract image is different from the digestive tract segment included in the first digestive tract image, a new prompt graphic may be generated based on the target area included in the second digestive tract image.

[0102] Specifically, the second digestive tract image can be processed by the image division and target area determination method described above, and a new prompt graphic can be generated based on the target area determined by the second digestive tract image to guide the doctor to conduct a comprehensive examination of the new digestive tract segment.

[0103] Of course, after acquiring the second digestive tract image, it is possible to first detect whether the second digestive tract image is an information frame. Specifically, a deep learning convolutional neural network (CNN), a Transformer model, and other methods can be used to detect whether the second digestive tract image contains information, and then determine whether the second digestive tract image is an information frame. In this way, images with high image clarity and containing valid information can be retained as information frames, and prompt graphics can be updated based on the information frames, which helps to avoid the occurrence of erroneous updates.

[0104] After the update is completed, the inspection quality score of the digestive tract inspection using an endoscope can be determined based on the number of prompt graphics and / or the proportion of the image area corresponding to the prompt graphics in the image area included in all information frames.

[0105] Among them, the examination quality score represents the inspection quality of the endoscopic examination.

[0106] Taking the aforementioned quadrant as an example, the total number of quadrants corresponding to the prompt graphics in all information frames can be compared with the total number of quadrants included in all information frames to obtain the above-mentioned proportion. It can be understood that the number of prompt graphics and the above-mentioned proportion can reflect the number and area of ​​the target areas included. The more the number of target areas and the larger the area, the more effective information the doctor has missed and the lower the inspection quality. Therefore, the inspection quality score of the digestive tract inspection using an endoscope can be determined by the number of prompt graphics and the above-mentioned proportion.

[0107] For easier understanding, please refer to Figure 7 and Figure 8 , the technical solution of the present application is explained by taking the digestive tract as the intestine. During the endoscopic examination process, the intestinal part of the input intestinal image can be identified. If it is a newly detected intestinal part, the first coordinate system or the second coordinate system can be used to divide the quadrants according to the length of the line connecting the intestinal cavity position and the image center, and effective information detection (i.e., intestinal wall recognition) is performed in each quadrant to determine the quadrant belonging to the target area and generate a corresponding prompt graphic. In response to the user's operation of the endoscope, when a new intestinal image arrives, it can be identified whether it belongs to an information frame. If the new intestinal image is an information frame, the effective information in the target area can be identified, and the prompt graphic of the corresponding quadrant is eliminated when the intestinal wall of the target area is identified.

[0108] In the implementation of the present application, the orientation of the area that has not been effectively detected (i.e., the target area) can be prompted during the process of withdrawing the scope, ensuring comprehensive detection of the digestive tract wall, reducing repeated detection of the detected direction and omission of the undetected direction, and helping to reduce the endoscopic examination time and improve the inspection quality. For digestive tract images, the coordinate system is determined according to the position of the digestive tract cavity, which can make more full use of the effective information in the image and fit the clinical practice. In addition, compared with the parameters such as lens speed and inspection time as indicators of quality evaluation, the present application strongly correlates the quality of withdrawing the scope with the comprehensiveness of the digestive tract observation, and reflects the doctor's inspection quality by the number of uneliminated prompt graphics and the proportion of the digestive tract wall in the information frame. The inspection quality score can be used to generate a diagnostic report, making the inspection quality assessment more accurate. At the same time, the method provided by the present application can be applied to endoscopic examinations of pipelines such as colorectal endoscopes, small intestinal endoscopes, capsule endoscopes, and pharyngoscopes, with higher scene adaptability.

[0109] It should also be understood that the digestive tract images (such as the first digestive tract image and the second digestive tract image) mentioned in the present application can be images collected in real time or images collected in the past. Figure 2 The scheme shown can occur in real time when the doctor operates the endoscope system to adapt to clinical applications and help the doctor to completely examine the digestive tract during a single examination. Figure 2The scheme shown may also be based on images or videos collected by the doctor through an endoscope in advance, which occurs when an examination quality assessment is needed, for example, it can be applied in medical teaching scenarios or in scenarios where the doctor reviews the endoscopic examination process.

[0110] Please refer to Fig. 9 , Fig. 9 A schematic diagram of the implementation flow of an image processing method provided in an embodiment of the present application is shown. The method can be applied in the aforementioned endoscope system, for example, on the image processing device 103, and can be applied to situations where the quality of endoscopic examination needs to be improved.

[0111] Specifically, the above image processing method may include the following steps S901 to S904.

[0112] Step S901, obtaining a first digestive tract image collected during the process of endoscope examination of the digestive tract.

[0113] Step S902: Identify the digestive tract cavity in the first digestive tract image to obtain the digestive tract cavity position.

[0114] Step S903, calculating the length of the line between the digestive tract cavity position and the image center of the first digestive tract image.

[0115] Step S904: if the length is greater than the length threshold, the first digestive tract image is divided into a plurality of quadrants according to the first coordinate system.

[0116] The origin of the first coordinate system is the center of the image, and the horizontal and vertical axes of the first coordinate system are symmetrically distributed along the connecting line. In addition, multiple quadrants can be used to evaluate the inspection quality of the endoscope inspection process.

[0117] It should be noted that the specific implementation of steps S901 to S904 can refer to Figures 2 to 8 Moreover, after step S904, the target area can also be determined according to the quadrant and a prompt graphic can be generated, and then the prompt graphic can be updated based on the second digestive tract image. The specific update method can also be referred to Figures 2 to 8 The description of this application will not be repeated here.

[0118] In an embodiment of the present application, the digestive tract cavity is identified in the first digestive tract image to obtain the position of the digestive tract cavity, and the length of the line between the position of the digestive tract cavity and the image center of the first digestive tract image is calculated. If the length is greater than the length threshold, the first digestive tract image is divided into multiple quadrants according to the first coordinate system to perform quality assessment through multiple quadrants. Since the origin of the first coordinate system is the image center, the horizontal and vertical axes of the first coordinate system are symmetrically distributed along the line, so that the digestive tract cavity can be distributed in the same quadrant as much as possible, avoiding the problem of insufficient accuracy of the quality assessment result caused by taking an area containing a large amount of digestive tract wall as the digestive tract cavity, which helps to accurately achieve quality assessment.

[0119] For the subsequent updating process of prompt graphics, this division method can also reduce the area that needs to re-test valid information and improve the efficiency of endoscopic examination.

[0120] It should be noted that, for the sake of simplicity of description, the aforementioned method embodiments are all expressed as a series of action combinations, but those skilled in the art should know that the present application is not limited to the described order of actions, because according to the present application, certain steps can be performed in other orders.

[0121] like Fig.10 FIG. 1 is a schematic diagram of the structure of an information prompting device 1000 provided in an embodiment of the present application. The information prompting device 1000 is configured on an image processing device 103 .

[0122] Specifically, the information prompting device 1000 may include:

[0123] The image display unit 1001 is used to display a first digestive tract image during the process of examining the digestive tract using an endoscope;

[0124] A prompt graphic display unit 1002 is used to display a prompt graphic corresponding to a target area in the first digestive tract image at an image boundary of the first digestive tract image, wherein the target area contains non-valid information;

[0125] The updating unit 1003 is used to update the display of the second digestive tract image in response to the user's operation on the endoscope, and update the prompt graphic based on the target area contained in the second digestive tract image.

[0126] In some embodiments of the present application, the prompt graphic display unit 1002 may also be specifically used to: divide the first digestive tract image into a plurality of image regions; and determine the target region from the plurality of image regions.

[0127] In some embodiments of the present application, the prompt graphic display unit 1002 can also be specifically used to: identify the digestive tract cavity in the first digestive tract image to obtain the digestive tract cavity position; calculate the length of the line between the digestive tract cavity position and the image center of the first digestive tract image; if the length is greater than a length threshold, the first digestive tract image is divided into multiple quadrants according to a first coordinate system, each quadrant corresponding to one of the multiple image areas, wherein the origin of the first coordinate system is the image center, and the horizontal and vertical axes of the first coordinate system are symmetrically distributed along the line.

[0128] In some embodiments of the present application, the prompt graphic display unit 1002 can also be specifically used for: if the length is less than or equal to the length threshold, dividing the first digestive tract image into multiple quadrants according to a second coordinate system, each quadrant corresponding to one image area among the multiple image areas, wherein the second coordinate system is the image coordinate system of the first digestive tract image.

[0129] In some embodiments of the present application, the prompt graphic display unit 1002 may also be specifically used to: perform digestive tract wall detection in each of the multiple image areas, and use the image area where the digestive tract wall is not detected as the target area; or, perform digestive tract cavity detection in each of the multiple image areas, and use the image area containing the center of the digestive tract cavity as the target area; or, perform digestive tract wall detection in each of the multiple image areas, calculate the digestive tract wall proportion corresponding to each of the multiple image areas, and use the image area where the digestive tract wall proportion is lower than a proportion threshold as the target area, wherein the digestive tract wall proportion is the ratio of the area between the digestive tract wall area and the corresponding entire image area.

[0130] In some embodiments of the present application, the above-mentioned updating unit 1003 can be specifically used to: determine the digestive tract segment contained in the second digestive tract image; if the digestive tract segment contained in the second digestive tract image is the same as the digestive tract segment contained in the first digestive tract image, then based on the target area contained in the second digestive tract image, update the prompt graphic.

[0131] In some embodiments of the present application, the updating unit 1003 may also be specifically used to generate a new prompt graphic based on the target area contained in the second digestive tract image if the digestive tract segment contained in the second digestive tract image is different from the digestive tract segment contained in the first digestive tract image.

[0132] In some embodiments of the present application, the updating unit 1003 may be specifically configured to: if information of the digestive tract wall in the target area is detected according to the second digestive tract image, clear the prompt graphic.

[0133] In some embodiments of the present application, the above-mentioned updating unit 1003 can be specifically used for: if information of the digestive tract wall in a partial sub-region of the target region is detected according to the second digestive tract image, then clearing part of the graphics in the prompt graphics, and the partial graphics correspond to the partial sub-region.

[0134] In some implementations of the present application, the updating unit 1003 may also be specifically used to detect whether the second digestive tract image is an information frame.

[0135] In some embodiments of the present application, the above-mentioned information prompting device 1000 may also include a quality assessment unit for determining the inspection quality score of the digestive tract inspection using the endoscope based on the number of the prompting graphics and / or the proportion of the image area corresponding to the prompting graphics in the image area included in all the information frames.

[0136] In some embodiments of the present application, the above-mentioned quality assessment unit can be specifically used to: for each digestive tract segment, determine the initial score of the digestive tract segment according to the number of the prompt graphics and / or the proportion of the image area corresponding to the prompt graphics in the image area included in all the information frames; determine the inspection quality score according to the initial scores of each digestive tract segment.

[0137] It should be noted that, for the convenience and simplicity of description, the specific working process of the above-mentioned information prompting device 1000 can refer to Figures 1 to 9 The corresponding process of the method will not be repeated here.

[0138] like Fig.11 FIG. 1 is a schematic diagram of the structure of an image processing apparatus 1100 provided in an embodiment of the present application. The image processing apparatus 1100 is configured on the image processing device 103 .

[0139] Specifically, the image processing device 1100 may include:

[0140] An image acquisition unit 1101 is used to acquire a first digestive tract image acquired during the process of endoscope examination of the digestive tract;

[0141] An identification unit 1102 is used to identify the digestive tract cavity in the first digestive tract image to obtain the digestive tract cavity position;

[0142] A calculation unit 1103, configured to calculate the length of a line between the digestive tract cavity position and the image center of the first digestive tract image;

[0143] The image division unit 1104 is used to divide the first digestive tract image into multiple quadrants according to a first coordinate system if the length is greater than a length threshold, wherein the origin of the first coordinate system is the center of the image, and the horizontal and vertical axes of the first coordinate system are symmetrically distributed along the connecting line, and the multiple quadrants are used to evaluate the inspection quality of the endoscopic inspection process.

[0144] It should be noted that, for the convenience and simplicity of description, the specific working process of the above-mentioned image processing device 1100 can refer to Figures 1 to 9 The corresponding process of the method will not be repeated here.

[0145] like Fig.12 , which is a schematic diagram of an image processing device 103 provided in an embodiment of the present application. Specifically, the image processing device 103 may include: a processor 1030, a memory 1031, and a computer program 1032 stored in the memory 1031 and executable on the processor 1030, such as an information prompting program. When the processor 1030 executes the computer program 1032, the steps in the above-mentioned information prompting method embodiments are implemented, such as Figure 2 Alternatively, when the processor 1030 executes the computer program 1032, the functions of each module / unit in the above-mentioned device embodiments are realized, for example Fig.10 The functions of the image display unit 1001, the prompt graphic display unit 1002 and the update unit 1003 are shown.

[0146] Alternatively, when the processor 1030 executes the computer program 1032, the steps in the above-mentioned various image processing method embodiments are implemented, for example Fig. 9 Alternatively, when the processor 1030 executes the computer program 1032, the functions of each module / unit in the above-mentioned device embodiments are realized, for example Fig.11 The functions of the image acquisition unit 1101, the recognition unit 1102, the calculation unit 1103 and the image division unit 1104 are shown.

[0147] The computer program may be divided into one or more modules / units, which are stored in the memory 1031 and executed by the processor 1030 to complete the present application. The one or more modules / units may be a series of computer program instruction segments capable of completing specific functions, which are used to describe the execution process of the computer program in the image processing device 103.

[0148] For example, the computer program can be divided into: an image display unit, a prompt graphic display unit, and an update unit. The specific functions of each unit are as follows: an image display unit, used to display a first digestive tract image during the process of using an endoscope to examine the digestive tract; a prompt graphic display unit, used to display a prompt graphic corresponding to a target area in the first digestive tract image at the image boundary of the first digestive tract image, wherein the target area contains non-valid information; an update unit, used to update and display a second digestive tract image in response to a user's operation of the endoscope, and update the prompt graphic based on the target area contained in the second digestive tract image.

[0149] For another example, the computer program can be divided into: an image acquisition unit, a recognition unit, a calculation unit, and an image division unit. The specific functions of each unit are as follows: an image acquisition unit, used to acquire a first digestive tract image collected during the endoscope inspection of the digestive tract; a recognition unit, used to identify the digestive tract cavity in the first digestive tract image and obtain the digestive tract cavity position; a calculation unit, used to calculate the length of the line between the digestive tract cavity position and the image center of the first digestive tract image; an image division unit, used to divide the first digestive tract image into a plurality of quadrants according to a first coordinate system if the length is greater than a length threshold, wherein the origin of the first coordinate system is the image center, the horizontal and vertical axes of the first coordinate system are symmetrically distributed along the line, and the plurality of quadrants are used to evaluate the inspection quality of the endoscope inspection process.

[0150] The image processing device 103 may include, but is not limited to, a processor 1030 and a memory 1031. Those skilled in the art will appreciate that Fig.12 It is only an example of the image processing device 103 and does not constitute a limitation of the image processing device 103. It may include more or fewer components than shown in the figure, or a combination of certain components, or different components. For example, the image processing device 103 may also include input and output devices, network access devices, buses, etc.

[0151] The processor 1030 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), off-the-shelf programmable gate arrays or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc.

[0152] The memory 1031 may be an internal storage unit of the image processing device 103, such as a hard disk or memory of the image processing device 103. The memory 1031 may also be an external storage device of the image processing device 103, such as a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), etc. equipped on the image processing device 103. Further, the memory 1031 may also include both an internal storage unit of the image processing device 103 and an external storage device. The memory 1031 is used to store the computer program and other programs and data required by the image processing device 103. The memory 1031 may also be used to temporarily store data that has been output or is to be output.

[0153] It should be noted that, for the convenience and brevity of description, the structure of the above-mentioned image processing device 103 can also refer to the specific description of the structure in the method embodiment, and will not be repeated here.

[0154] The technicians in the relevant field can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example for illustration. In practical applications, the above-mentioned function allocation can be completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiment can be integrated in a processing unit, or each unit can exist physically separately, or two or more units can be integrated in one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units. In addition, the specific names of the functional units and modules are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of this application. The specific working process of the units and modules in the above-mentioned system can refer to the corresponding process in the aforementioned method embodiment, which will not be repeated here.

[0155] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0156] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0157] In the embodiments provided in the present application, it should be understood that the disclosed devices / equipment and methods can be implemented in other ways. For example, the device / equipment embodiments described above are merely schematic, for example, the division of the modules or units is only a logical function division, and there may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0158] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0159] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0160] If the integrated module / unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present application implements all or part of the processes in the above-mentioned embodiment method, and can also be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium, and the computer program can implement the steps of the above-mentioned various method embodiments when executed by the processor. Among them, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording medium, U disk, mobile hard disk, disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electric carrier signal, telecommunication signal and software distribution medium. It should be noted that the content contained in the computer-readable medium can be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, computer-readable media do not include electric carrier signals and telecommunication signals.

[0161] The embodiments described above are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, a person skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. Such modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application, and should all be included in the protection scope of the present application.

Claims

1. An information prompting method, characterized in that: include: During the process of examining the digestive tract using an endoscope, displaying a first digestive tract image; Displaying a prompt graphic corresponding to a target area in the first digestive tract image at an image boundary of the first digestive tract image, wherein the target area contains non-valid information; In response to the user's operation on the endoscope, the second digestive tract image is updated and displayed, and based on the target area contained in the second digestive tract image, the prompt graphic is updated.

2. The information prompting method according to claim 1, characterized in that: Before displaying a prompt graphic corresponding to a target area in the first digestive tract image at an image boundary of the first digestive tract image, the information prompt method further includes: dividing the first digestive tract image into a plurality of image regions; The target area is determined from the multiple image areas.

3. The information prompting method according to claim 2, characterized in that: The step of dividing the first digestive tract image into a plurality of image regions comprises: Identifying the digestive tract cavity in the first digestive tract image to obtain the digestive tract cavity position; Calculating the length of a line between the digestive tract cavity position and the image center of the first digestive tract image; If the length is greater than the length threshold, the first digestive tract image is divided into multiple quadrants according to a first coordinate system, each quadrant corresponding to an image area among the multiple image areas, wherein the origin of the first coordinate system is the center of the image, and the horizontal and vertical axes of the first coordinate system are symmetrically distributed along the connecting line.

4. The information prompting method according to claim 3, characterized in that: After calculating the length of the line between the digestive tract cavity position and the image center of the first digestive tract image, the information prompting method further includes: If the length is less than or equal to the length threshold, the first digestive tract image is divided into multiple quadrants according to a second coordinate system, each quadrant corresponding to an image area among the multiple image areas, wherein the second coordinate system is the image coordinate system of the first digestive tract image.

5. The information prompting method according to claim 2, characterized in that: The step of determining the target area from the multiple image areas comprises: Performing digestive tract wall detection in each of the plurality of image regions, and taking the image region where the digestive tract wall is not detected as the target region; Alternatively, digestive tract cavity detection is performed in each of the plurality of image regions, and an image region including the center of the digestive tract cavity is used as the target region; Alternatively, digestive tract wall detection is performed in each of the multiple image areas, and the digestive tract wall proportion corresponding to each of the multiple image areas is calculated, and the image area where the digestive tract wall proportion is lower than a proportion threshold is used as the target area, wherein the digestive tract wall proportion is the ratio of the area between the digestive tract wall area and the corresponding entire image area.

6. The information prompting method according to any one of claims 1 to 5, characterized in that: The updating of the prompt graphic based on the target area situation included in the second digestive tract image includes: determining a digestive tract segment included in the second digestive tract image; If the digestive tract segment included in the second digestive tract image is the same as the digestive tract segment included in the first digestive tract image, the prompt graphic is updated based on the target area included in the second digestive tract image.

7. The information prompting method according to claim 6, characterized in that: After determining the digestive tract segment included in the second digestive tract image, the information prompting method further includes: If the digestive tract segment included in the second digestive tract image is different from the digestive tract segment included in the first digestive tract image, a new prompt graphic is generated based on the target area included in the second digestive tract image.

8. The information prompting method according to any one of claims 1 to 5, characterized in that: The updating of the prompt graphic based on the target area situation included in the second digestive tract image includes: If information on the digestive tract wall in a partial sub-region of the target region is detected according to the second digestive tract image, a partial graphic in the prompt graphic is deleted, and the partial graphic corresponds to the partial sub-region.

9. An image processing method, characterized in that: include: Acquire a first digestive tract image acquired during the process of endoscopic examination of the digestive tract; Identifying the digestive tract cavity in the first digestive tract image to obtain the digestive tract cavity position; Calculating the length of a line between the digestive tract cavity position and the image center of the first digestive tract image; If the length is greater than the length threshold, the first digestive tract image is divided into multiple quadrants according to a first coordinate system, wherein the origin of the first coordinate system is the center of the image, the horizontal and vertical axes of the first coordinate system are symmetrically distributed along the connecting line, and the multiple quadrants are used to evaluate the inspection quality of the endoscopic inspection process.

10. An information prompting device, characterized in that: include: An image display unit, used to display a first digestive tract image during the process of examining the digestive tract using an endoscope; a prompt graphic display unit, configured to display a prompt graphic corresponding to a target area in the first digestive tract image at an image boundary of the first digestive tract image, wherein the target area contains non-valid information; An updating unit is used to update and display the second digestive tract image in response to the user's operation on the endoscope, and to update the prompt graphic based on the target area contained in the second digestive tract image.

11. An image processing device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps of the information prompt method according to any one of claims 1 to 8 are implemented, or when the processor executes the computer program, the steps of the image processing method according to claim 9 are implemented.

12. An endoscope system, characterized in that: include: Light sources, endoscopes, image processing equipment, and displays; The image processing device is used to execute the steps of the information prompt method according to any one of claims 1 to 8, or the image processing device is used to execute the steps of the image processing method according to claim 9.

Images