Intestine segment recognition method and device, image processing device, and endoscope system
By analyzing the volume and distribution data of fluid in endoscopic images, combined with body position and rotation angle, the problem of inaccurate intestinal segment identification during colonoscopy was solved, improving the accuracy of intestinal segment identification and the precision of lesion localization.
Patent Information
- Application Number
- CN202410056635.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-01-15
- Publication Date
- 2026-01-09
- Estimated Expiration
- 2044-01-15
AI Technical Summary
During colonoscopy, current technology struggles to accurately identify the ascending colon, transverse colon, and descending colon, leading to inaccurate lesion localization.
By acquiring colon images captured by an endoscope, analyzing the volume and distribution of fluid, and combining this with the patient's position and the rotation angle of the endoscope, the identification result of the intestinal segment is determined.
It improves the accuracy of intestinal segment identification, enabling more accurate identification of the intestinal segment to which the current frame belongs, and improves the localization effect of lesions.
Smart Images

Figure CN120107630B_ABST
Abstract
Description
TECHNICAL FIELD
[0001] The present application belongs to the technical field of image recognition, and particularly relates to an intestinal segment identification method and device, an image processing apparatus, and an endoscope system. BACKGROUND
[0002] In a colonoscopy, an operator generally inserts an endoscope from the anus of a subject into the colon through the rectum, completes the insertion operation, and then starts the withdrawal of the endoscope. During the withdrawal of the endoscope, the intestinal wall tissue is observed through the endoscope to detect a lesion.
[0003] The colon can be divided into four intestinal segments, namely, ascending colon, transverse colon, descending colon, and sigmoid colon. The sigmoid colon is connected to the rectum and has a very obvious difference in shape from other intestinal segments, while the differences in shape among the ascending colon, the transverse colon, and the descending colon are relatively small. This makes it difficult for the operator to accurately locate the intestinal segment where the endoscope is located according to the image captured by the endoscope, which is not conducive to the positioning of the lesion. In the related art, image recognition is performed on the image captured by the endoscope to assist in locating the intestinal segment. However, due to the small differences in shape, the accuracy of the recognition still needs to be improved. SUMMARY
[0004] Embodiments of the present application provide an intestinal segment identification method and device, an image processing apparatus, and an endoscope system, which can solve the problem that the accuracy of the colonoscopy image recognition needs to be improved in the related art.
[0005] In a first aspect, embodiments of the present application provide an intestinal segment identification method, which includes: obtaining a target image frame captured by an endoscope based on the colon; and determining a first intestinal segment identification result of the target image frame according to the amount of fluid and distribution data of the liquid in the target image frame. The intestinal segment identification result includes the ascending colon, the transverse colon, or the descending colon.
[0006] In a possible implementation manner of the first aspect, the ascending colon, the transverse colon, and the descending colon each have an associated amount-of-fluid range and a direction range. Correspondingly, determining the first intestinal segment identification result of the target image frame according to the amount of fluid and the distribution data of the liquid in the target image frame includes: in a case where the amount of fluid of the liquid in the target image frame belongs to the amount-of-fluid range associated with the target intestinal segment and the distribution data belongs to the direction range associated with the target intestinal segment, determining that the first intestinal segment identification result is the target intestinal segment, and the target intestinal segment is one of the ascending colon, the transverse colon, and the descending colon.
[0007] In a possible implementation manner of the first aspect, before determining that the first intestinal segment identification result is the target intestinal segment, the method further includes: determining the amount-of-fluid range and the direction range associated with the ascending colon, the transverse colon, and the descending colon according to the body position of the subject of the endoscope.
[0008] In a possible implementation manner of the first aspect, after determining the fluid volume range and the direction range of the ascending colon, the transverse colon and the descending colon according to the body position of the examination object of the endoscope, the method further includes: adjusting the direction range of the ascending colon, the transverse colon and the descending colon according to a rotation angle of the endoscope.
[0009] In a possible implementation manner of the first aspect, the determination of the fluid volume range and the direction range of the ascending colon, the transverse colon and the descending colon according to the body position of the examination object of the endoscope includes: in a case where the body position is left lateral decubitus, the direction range of the ascending colon includes a first direction range, the direction range of the transverse colon includes a second direction range, and the direction range of the descending colon includes a third direction range; and the first direction range, the second direction range and the third direction range are independent of each other.
[0010] In a possible implementation manner of the first aspect, the first direction range in angle is 66° to 225°, the upper limit of the fluid volume range of the transverse colon is a first preset value, the second direction range in angle is 354.5° to 360° and 0° to 65.5°, and the lower limit of the fluid volume range of the descending colon is a second preset value. The third direction range in angle is 225.5° to 354°.
[0011] In a possible implementation manner of the first aspect, the determination of the fluid volume range and the direction range of the ascending colon, the transverse colon and the descending colon according to the body position of the examination object of the endoscope includes: in a case where the body position is right lateral decubitus, the third direction range in angle is 66° to 225°, the upper limit of the fluid volume range of the transverse colon is a first preset value, the second direction range in angle is 354.5° to 360° and 0° to 65.5°, the lower limit of the fluid volume range of the ascending colon is a second preset value, and the first direction range in angle is 225.5° to 354°.
[0012] In a possible implementation manner of the first aspect, the method further includes: obtaining a second intestinal segment recognition result and a third intestinal segment recognition result of the target image frame, the second intestinal segment recognition result being obtained according to a shape of an intestinal cavity in the target image frame, and the third intestinal segment recognition result being obtained according to a bending of an intestinal tract in the target image frame; and determining a comprehensive intestinal segment recognition result of the target image frame according to at least the first intestinal segment recognition result, the second intestinal segment recognition result and the third intestinal segment recognition result.
[0013] In a possible implementation manner of the first aspect, determining the comprehensive intestinal tract recognition result of the target image frame according to the first intestinal tract recognition result, the second intestinal tract recognition result and the third intestinal tract recognition result comprises: determining a first recognition result, a second recognition result and a third recognition result of the intestinal tract according to the first intestinal tract recognition result, the second intestinal tract recognition result and the third intestinal tract recognition result of each image frame in the first image frame respectively; the first image frame comprises a plurality of historical image frames in a preset time range ending at a target time and the target image frame, and the target time is a collection time of the target image frame; and determining the comprehensive intestinal tract recognition result of the target image frame according to the first recognition result, the second recognition result and the third recognition result.
[0014] In a possible implementation manner of the first aspect, the preset time range comprises: a first time range, a second time range and a third time range; determining the first recognition result, the second recognition result and the third recognition result of the intestinal tract according to the first intestinal tract recognition result, the second intestinal tract recognition result and the third intestinal tract recognition result of each image frame in the first image frame comprises: when the first intestinal tract recognition results of at least a first number of continuous image frames in the first image frame in the first time range are the same, taking the same first intestinal tract recognition result as the first recognition result; when the second intestinal tract recognition results of at least a second number of continuous image frames in the first image frame in the second time range are the same, taking the same second intestinal tract recognition result as the second recognition result; and when the third intestinal tract recognition results of at least a third number of continuous image frames in the first image frame in the third time range are the same, taking the same third intestinal tract recognition result as the third recognition result.
[0015] In a possible implementation manner of the first aspect, the first time range and the second time range are both smaller than the third time range.
[0016] In a possible implementation manner of the first aspect, the first number and the second number are both greater than the third number.
[0017] In a possible implementation manner of the first aspect, the first intestinal tract recognition result further comprises other, and determining the comprehensive intestinal tract recognition result of the target image frame according to the first recognition result, the second recognition result and the third recognition result comprises: in a case where the first recognition result and the third recognition result are both matched with the second recognition result, determining that the comprehensive intestinal tract recognition result is the second recognition result; or in a case where one of the first recognition result and the third recognition result is matched with the second recognition result, and the unmatched recognition result is other, determining that the comprehensive intestinal tract recognition result is the second recognition result.
[0018] In a possible implementation manner of the first aspect, the first intestinal tract recognition result further comprises other, and determining the comprehensive intestinal tract recognition result of the target image frame according to the first recognition result, the second recognition result and the third recognition result comprises: in a case where the first recognition result is other, determining the comprehensive intestinal tract recognition result according to the second recognition result and the third recognition result; and in a case where the first recognition result is not other, determining the comprehensive intestinal tract recognition result as the first recognition result.
[0019] In a possible implementation manner of the first aspect, determining the comprehensive intestinal tract recognition result of the target image frame according to at least the first intestinal tract recognition result, the second intestinal tract recognition result and the third intestinal tract recognition result comprises: determining the comprehensive intestinal tract recognition result of the target image frame according to only the first intestinal tract recognition result, the second intestinal tract recognition result and the third intestinal tract recognition result of the target image frame.
[0020] In a possible implementation manner of the first aspect, the first intestinal tract recognition result further comprises other, and determining the comprehensive intestinal tract recognition result of the target image frame according to only the first intestinal tract recognition result, the second intestinal tract recognition result and the third intestinal tract recognition result of the target image frame comprises: in a case where the first intestinal tract recognition result and the third intestinal tract recognition result both match the second intestinal tract recognition result, determining the comprehensive intestinal tract recognition result as the second intestinal tract recognition result; or in a case where one of the first intestinal tract recognition result and the third intestinal tract recognition result matches the second intestinal tract recognition result, and the unmatched recognition result is other, determining the comprehensive intestinal tract recognition result as the second intestinal tract recognition result.
[0021] In a possible implementation manner of the first aspect, the first intestinal tract recognition result further comprises other, and determining the comprehensive intestinal tract recognition result of the target image frame according to only the first intestinal tract recognition result, the second intestinal tract recognition result and the third intestinal tract recognition result of the target image frame comprises: in a case where the first intestinal tract recognition result is other, determining the comprehensive intestinal tract recognition result according to the second intestinal tract recognition result and the third intestinal tract recognition result; and in a case where the first intestinal tract recognition result is not other, determining the comprehensive intestinal tract recognition result as the first intestinal tract recognition result.
[0022] In a possible implementation manner of the first aspect, before determining the first intestinal tract recognition result of the target image frame according to the amount of liquid accumulation of the liquid in the target image frame and the distribution data of the liquid, the method further comprises: inputting the target image frame into a pre-trained liquid recognition model to obtain the amount of liquid accumulation of the liquid in the target image frame and the distribution data of the liquid in the target image frame, wherein the liquid recognition model is obtained by training liquid recognition samples, and the liquid recognition samples are partially or entirely liquid-containing sample images carrying labels associated with the amount of liquid accumulation and the distribution data of the liquid.
[0023] In a possible implementation manner of the first aspect, the first intestinal segment identification result of the target image frame is determined according to the liquid accumulation amount of the liquid in the target image frame and the distribution data of the liquid, and the determining includes: inputting the target image frame into a pre-trained liquid identification model to obtain the first intestinal segment identification result of the target image frame; the liquid identification model is obtained by training liquid identification samples, and sample images of the liquid identification samples partially or entirely exist liquid and carry labels associated with the liquid accumulation amount and the distribution data of the liquid.
[0024] In a possible implementation manner of the first aspect, the method further includes: obtaining a second intestinal segment identification result of the target image frame, the second intestinal segment identification result being obtained according to a shape of an intestinal cavity in the target image frame; and determining a comprehensive intestinal segment identification result of the target image frame according to at least the first intestinal segment identification result and the second intestinal segment identification result.
[0025] In a possible implementation manner of the first aspect, the determining the comprehensive intestinal segment identification result of the target image frame according to at least the first intestinal segment identification result and the second intestinal segment identification result includes: determining a first identification result and a second identification result of an intestinal segment respectively according to the first intestinal segment identification result and the second intestinal segment identification result of each image frame in a first image frame, the first image frame including a plurality of historical image frames in a preset time range ending at a target time and the target image frame, the target time being a collection time of the target image frame; and determining the comprehensive intestinal segment identification result of the target image frame according to the first identification result and the second identification result.
[0026] In a possible implementation manner of the first aspect, the preset time range includes a first time range and a second time range; the determining the first identification result and the second identification result of the intestinal segment respectively according to the first intestinal segment identification result and the second intestinal segment identification result of each image frame in the first image frame includes: when the first intestinal segment identification results of at least a first number of continuous image frames in the first image frame in the first time range are the same, taking the same first intestinal segment identification result as the first identification result; and when the second intestinal segment identification results of at least a second number of continuous image frames in the first image frame in the second time range are the same, taking the same second intestinal segment identification result as the second identification result.
[0027] In a possible implementation manner of the first aspect, the determining the comprehensive intestinal segment identification result of the target image frame according to the first identification result and the second identification result includes: in a case where the first identification result matches the second identification result, determining that the comprehensive intestinal segment identification result is the first identification result.
[0028] In a possible implementation manner of the first aspect, the first intestinal segment identification result further comprises other, and determining the comprehensive intestinal segment identification result of the target image frame according to the first identification result and the second identification result comprises: in a case where one of the first identification result and the second identification result is other, determining the comprehensive intestinal segment identification result as the intestinal segment identification result other than the other.
[0029] In a possible implementation manner of the first aspect, determining the comprehensive intestinal segment identification result of the target image frame according to at least the first intestinal segment identification result and the second intestinal segment identification result comprises: determining the comprehensive intestinal segment identification result of the target image frame according to only the first intestinal segment identification result and the second intestinal segment identification result of the target image frame.
[0030] In a possible implementation manner of the first aspect, the first intestinal segment identification result further comprises other, and determining the comprehensive intestinal segment identification result of the target image frame according to only the first intestinal segment identification result and the second intestinal segment identification result of the target image frame comprises: in a case where the first intestinal segment identification result matches the second intestinal segment identification result, determining the comprehensive intestinal segment identification result as the first intestinal segment identification result.
[0031] In a possible implementation manner of the first aspect, the first intestinal segment identification result further comprises other, and determining the comprehensive intestinal segment identification result of the target image frame according to only the first intestinal segment identification result and the second intestinal segment identification result of the target image frame comprises: in a case where one of the first intestinal segment identification result and the second intestinal segment identification result is other, determining the comprehensive intestinal segment identification result as the intestinal segment identification result other than the other.
[0032] In a second aspect, an embodiment of the present application provides an intestinal segment identification device, comprising: an acquisition module configured to acquire a target image frame based on a colon shot by an endoscope; and an identification module configured to determine a first intestinal segment identification result of the target image frame according to liquid accumulation amount and distribution data of liquid in the target image frame, the intestinal segment identification result comprising ascending colon, transverse colon or descending colon.
[0033] In a third aspect, an embodiment of the present application provides an image processing device, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, and the processor implements the intestinal segment identification method in any one of the first aspect when executing the computer program.
[0034] In a fourth aspect, an embodiment of the present application provides an endoscope system, comprising a light source device, a display device, the image processing device in the third aspect, and an endoscope.
[0035] In a fifth aspect, an embodiment of the present application provides a computer readable storage medium, storing a computer program, and the computer program is executable on a processor to implement the intestinal segment identification method in any one of the first aspect.
[0036] In a sixth aspect, an embodiment of the present application provides a computer program product, which, when running on a terminal device, causes the terminal device to perform the intestinal segment identification method in any one of the first aspect.
[0037] Compared with the prior art, the embodiment of the present application has the beneficial effect that: by identifying the liquid in the current frame from the colonoscope, the liquid identification result of the current frame is obtained according to the proportion and distribution of the liquid; and the intestinal segment identification result of the current frame is determined at least according to the liquid identification result of the current frame. The liquid in the intestinal tract is generally the secretion of the intestinal wall tissue and generally exists in the colon bag. Influenced by gravity and the body position of the examination object, the amount and distribution of the liquid in different intestinal segments have relatively obvious differences, so that the liquid identification result can more accurately identify the intestinal segment to which the current frame belongs. The liquid identification result is introduced into the intestinal segment identification, which can effectively improve the accuracy of intestinal segment identification. BRIEF DESCRIPTION OF DRAWINGS
[0038] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed to be used in the embodiments or the prior art description will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.
[0039] Figure 1 is a structural schematic diagram of an image processing device provided by an embodiment of the present application;
[0040] Figure 2 is a structural schematic diagram of an endoscope system provided by an embodiment of the present application;
[0041] Figure 3 is a flow schematic diagram of an intestinal segment identification method provided by an embodiment of the present application;
[0042] Figure 4 is a schematic diagram of the liquid distribution in each intestinal segment and the endoscope tip posture under a left lateral body position provided by an embodiment of the present application;
[0043] Figure 5 is an example diagram of the liquid existing in the ascending colon under a left lateral body position provided by an embodiment of the present application;
[0044] Figure 6 is an example diagram of the liquid existing in the transverse colon under a left lateral body position provided by an embodiment of the present application;
[0045] Figure 7 is an example diagram of the liquid existing in the descending colon under a left lateral body position provided by an embodiment of the present application;
[0046] Figure 8 is a flowchart of an intestinal segment identification method provided by another embodiment of the present application;
[0047] Figure 9 is a flowchart of an intestinal segment identification method provided by another embodiment of the present application;
[0048] Figure 10 is a flowchart of an intestinal segment identification method provided by another embodiment of the present application;
[0049] Figure 11 is a structural diagram of an intestinal segment identification device provided by an embodiment of the present application. DETAILED DESCRIPTION
[0050] In the following description, for the purposes of explanation and not limitation, specific details are set forth, such as particular system configurations, techniques, etc., in order to provide a thorough understanding of the embodiments of the present application. However, it will be apparent to those skilled in the art that the present application can be practiced in other embodiments that depart from these specific details. In other instances, detailed descriptions of well-known systems, devices, circuits, and methods are omitted so as not to obscure the description of the present application with unnecessary detail.
[0051] It should be understood that the term "comprises" when used in this specification and the appended claims, specifies the presence of stated features, integers, steps, operations, elements, and / or components, but does not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof.
[0052] It should also be understood that the term "and / or" when used in this specification and the appended claims, means any one or more of the associated listed items can be present, and includes multiples of any one or more of the associated listed items.
[0053] As used in this specification and the appended claims, the term "if' can, in some instances, be interpreted as meaning "when," or "once," or "in response to a determination," or "in response to detecting." Similarly, the phrase "if a determination" or "if detecting [the recited condition or event]" can, in some instances, be interpreted to mean "once a determination" or "in response to a determination" or "once detecting [the recited condition or event]" or "in response to detecting [the recited condition or event]."
[0054] In addition, in the description of the specification and the appended claims, the terms "first," "second," "third," etc. are used only to distinguish descriptions, and cannot be understood as indicating or implying relative importance.
[0055] Reference within the specification of this application to "one embodiment" or "some embodiments" means that a particular feature, structure, or characteristic described in connection with the embodiment is included in at least one embodiment of the application. The appearances of the phrases "in one embodiment" or "in some embodiments" in various places in the specification are not necessarily all referring to the same embodiment, however, unless otherwise specifically stated. The terms "including," "containing," "having," and variations thereof mean "including but not limited to," unless expressly specified otherwise.
[0056] The intestinal segment recognition method provided by the embodiments of the present application can be applied to an image processing device. The image processing device can work independently, and can be specifically an electronic device with a computing function, including but not limited to a server, a server cluster, a workstation, a notebook computer, a desktop computer, and the like. Alternatively, the image processing device can be part of an endoscope system. The embodiments of the present application do not limit the specific type of the image processing device.
[0057] Figure 1 A block diagram of part of the structure of the image processing device provided by the embodiments of the present application is shown. Referring to Figure 1 , the image processing device includes a processor 10, a memory 20, a bus 30, and a communication device 60. The processor 10 and the memory 20 are connected to each other through the bus 30, and an input device 40, an output device 50, and the communication device 60 are also connected to the bus 30. Those skilled in the art can understand that Figure 1 The structure of the image processing device shown in the specification does not constitute a limitation on the image processing device, and can include more or fewer components than shown, or combine certain components, or different component arrangements.
[0058] The specific components of the image processing device will be described below. Figure 1 The specific components of the image processing device will be described below.
[0059] The processor 10 is a control center of the image processing device, and can execute various functions and process data by running programs stored in the memory 20. The processor 10 can be a central processing unit (CPU), and can also be other general-purpose processors, digital signal processors (DSPs), application specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs) or other programmable logic devices, discrete gates or transistor logic components, discrete hardware components, etc. The general-purpose processor can be a microprocessor or can also be any conventional processor. In some embodiments, the processor 10 can include an artificial intelligence (AI) processor for processing computing operations related to machine learning.
[0060] The memory 20 is configured to store operating systems, application programs, boot loaders, data, and other programs, such as program codes of computer programs, etc. The memory 20 can also be used to temporarily store data required for executing programs and generated data. The memory 20 can include a high-speed random access memory, and can also include a non-volatile memory, such as a flash memory, a hard disk, a multimedia card, a card-type memory, etc. The memory 20 can include a storage unit arranged inside the image processing device, such as a hard disk of the image processing device, and / or a detachable external storage unit, such as a mobile hard disk, a U disk, a smart media card (SMC), a secure digital (SD) card, etc.
[0061] The input device 40 can include at least one of a keyboard, a mouse, a touch panel, a joystick, etc., and is configured to collect input operations of a user to generate corresponding input signals.
[0062] The output device 50 is configured to output information to be provided to a user. The output device 50 generally includes a display, and can optionally be a liquid crystal display (LCD), an organic light-emitting diode (OLED), etc. In addition, the output device can further include a speaker.
[0063] The communication device 60 can include a modem, a network card, etc., and is configured to establish a network connection with other devices and communicate with each other.
[0064] In a case where the image processing apparatus is part of an endoscope system, at least one of the input device 40, the output device 50, and the communication device 60 can be omitted.
[0065] The intestinal segment recognition method provided by the embodiments of the present application can be implemented as a computer software program. For example, the embodiments of the present application provide a computer program product including a computer program carried on a computer readable medium, the computer program containing program codes for executing the method shown in the flowchart. In such embodiments, the computer program can be downloaded and installed from the network through the communication device 60, and / or installed from the detachable external storage unit. When the computer program is executed by the processor 10, various functions defined in the intestinal segment recognition method provided by the embodiments of the present application are implemented.
[0066] Figure 2 A block diagram of part of the structure of an endoscope system provided by the embodiments of the present application is shown. Referring to Figure 2 , the endoscope system includes an image processing apparatus 100, an endoscope 200, a light source apparatus 300, and a display apparatus 400. The endoscope 200 is connected to the image processing apparatus 100 and the light source apparatus 300 respectively, and the image processing apparatus 100 is further connected to the display apparatus 400. Those skilled in the art can understand that Figure 2 The structure of the endoscope system shown in the above
[0067] The endoscope 200 can include a tip, an insertion tube, and an operation portion. During an examination, the tip and the insertion tube can be inserted into a natural cavity of a human body under the control of an operator, and the operator can control the position and posture of the tip and the insertion tube in the human body by operating the operation portion. The tip of the endoscope 200 is provided with an image sensor for collecting images, such as a Charge Coupled Device (CCD) sensor, a Complementary Metal Oxide Semiconductor (CMOS) sensor, etc. The images collected by the image sensor are conducted through the scope body and input to the image processing apparatus 100.
[0068] The light source device 300 is used to provide light source for the image sensor. The light source device 300 shown in the figure is a device independent of the endoscope 200, and the light generated by the device is transmitted to the tip through the light guide fiber in the scope.
[0069] The display device 400 can be a liquid crystal display, an organic light emitting diode display, etc., and is used to display the display content output by the image processing device 100.
[0070] In order to facilitate understanding of the intestinal segment identification method provided in the embodiments of the present application, the colonoscopy is specifically described below.
[0071] In the present application, the site of the colonoscopy includes the large intestine. The large intestine is a part of the lower digestive tract, including the cecum, the colon and the rectum. The cecum is the beginning of the large intestine, located in the right iliac fossa, connected to the ileum on the left, and connected to the ascending colon on the top. The colon surrounds the empty ileum, which can be divided into four parts: the ascending colon, the transverse colon, the descending colon and the sigmoid colon. The ascending colon is the part of the cecum continuing upward, bending to the left to form the transverse colon below the right lobe of the liver. The left end of the transverse colon reaches the lower part of the spleen, and the part bending downward to the left iliac crest is called the descending colon. The part of the colon below the left iliac crest plane is located in the lower abdomen and the pelvic cavity, the intestine is curved, which is called the sigmoid colon, and continues to the rectum at the third sacral vertebra level. The rectum is located in the pelvic cavity, and the part below the pelvic diaphragm is also called the anal canal. The turning part between the ascending colon and the transverse colon is the right colon of the colon, also known as the hepatic flexure. The turning part between the transverse colon and the descending colon is the left colon of the colon, also known as the splenic flexure. The colon and the cecum have a characteristic structure of the colon pouch, which is a cystic protrusion separated by a transverse groove and bulging outward.
[0072] The process of colonoscopy generally includes two parts of entering the scope and withdrawing the scope, and the operator mainly checks the intestinal wall during the withdrawing process. The main component of the large intestine is the colon, so the colonoscopy can also be called the colonoscopy. In a relatively ideal observation condition, the sigmoid colon is slender, curved and has a large degree of freedom, the colon pouch is shallow, and the intestinal cavity is circular; the descending colon has a relatively fixed shape, which is a short straight tunnel, and is circular or triangular; the transverse colon is long, and appears as an inverted triangle under the scope; the ascending colon is short and straight, the colon pouch is wide, and the intestinal cavity is large and triangular. It can be seen that the shape difference between the sigmoid colon and other intestinal segments is more obvious, while the shape difference between the ascending colon, the transverse colon and the descending colon is smaller. In addition, the individual differences of the objects of the endoscopic examination, the examination operation and other factors often make it more difficult to distinguish.
[0073] Figure 3 A flowchart of the intestinal segment identification method provided in an embodiment of the present application is shown, which is used as an example and is not limited. The method can be applied to the image processing device described above.
[0074] S1: Obtain a target image frame based on the endoscope shooting the colon.
[0075] The embodiment can be applied in the process of enteroscopy. The image processing device can receive a real-time captured video stream from an endoscope and sequentially process image frames in the video stream. A target image frame is a currently processed image frame. To ensure real-time performance, a time length between a capturing time of the target image frame and a starting processing time is very short. Alternatively, the embodiment can be applied to processing of an enteroscopy video obtained by completed enteroscopy. The target image frame is a currently processed image frame.
[0076] S2: determining a liquid volume of the liquid in the target image frame and distribution data of the liquid in the target image frame.
[0077] The enteroscopy segment recognition method provided in the embodiment uses the liquid in the image frame to recognize enteroscopy segments. The main recognized objects include ascending colon, transverse colon, and descending colon. Under normal circumstances, the colon will secrete liquid to protect the intestinal wall and lubricate the feces. The liquid is generally referred to as intestinal cleansing fluid. Due to the action of gravity, the intestinal cleansing fluid is generally stored in the colon bag. Digestive tract diseases can cause changes in the properties, volume, and the like of the intestinal cleansing fluid.
[0078] Due to the presence of liquid in the intestinal tract, the liquid can adversely affect the observation of the intestinal wall. In the operation guidelines of enteroscopy, it is generally required to suck the residual liquid during the insertion of the endoscope. However, in actual applications, due to a large number of examinations, the time of a single enteroscopy is very limited, and the operator usually does not have time to completely suck the residual liquid during the insertion of the endoscope, but performs the sucking during the withdrawal of the endoscope, so that the liquid can still be identified in the image during the withdrawal of the endoscope.
[0079] In some embodiments, a conventional image processing, such as edge detection, contour processing, or the like, can be used to identify which pixels in the target image frame belong to a liquid region, to determine the liquid volume of the liquid in the target image frame according to the size of the liquid region, and to determine the distribution data of the liquid in the target image frame according to the position of the liquid region in the target image frame. The distribution data is used to represent the direction of the liquid in the target image frame. The direction can be represented by a direction of a center point of the liquid region, or can be represented by a direction range of the entire liquid region. The direction can be represented by a specific angle, or can be represented by other manners, such as a point clock direction, which is not limited herein.
[0080] In some embodiments, the liquid in the target image frame can be identified by using a deep learning method. Specifically, the target image frame is input into a pre-trained liquid identification model to obtain the liquid volume and the distribution data of the liquid in the target image frame. The liquid identification model is trained using liquid identification samples, some of which contain liquid or all of which contain liquid. In addition to the sample images, the liquid identification samples can carry labels associated with the liquid volume and the distribution data of the liquid as supervision information. The specific content of the label is related to the output of the model. If the model outputs the liquid volume and the distribution data of the liquid in the target image frame, the label can include the liquid region in the corresponding sample, the liquid volume, and the distribution data of the liquid in the target image frame, etc. The sample images containing liquid are non-information frames that need to be removed for common models used for intestinal segment identification (such as models based on intestinal cavity morphology), and cannot be used as training samples, but in this application, they are used as samples for training the liquid identification model.
[0081] S3: determining the first intestinal segment identification result of the target image frame according to the liquid volume and the distribution data of the liquid in the target image frame.
[0082] In the embodiments of the present application, the intestinal segment identification result includes ascending colon, transverse colon, descending colon, or other, which means that the intestinal segment cannot be identified according to the liquid. Ascending colon, transverse colon, and descending colon each have an associated liquid volume range and a direction range. If the liquid volume of the liquid in the target image frame belongs to the liquid volume range associated with the target intestinal segment, and the distribution data belongs to the direction range associated with the target intestinal segment, the first intestinal segment identification result is determined to be the target intestinal segment, and the target intestinal segment is one of the ascending colon, the transverse colon, and the descending colon. In other cases, the first intestinal segment identification result is determined to be other.
[0083] The adjacent intestinal segments of the colon are connected, and the liquid in the intestinal segment flows under the action of gravity as the body position of the examination object changes. Combined with the change in the posture of the image sensor caused by the movement of the endoscope tip in the intestinal tract, the liquid distribution characteristics associated with different intestinal segments can be analyzed, which can include the liquid volume range and the direction range. The liquid distribution characteristics associated with each intestinal segment are different for different body positions of the examination object. The liquid volume range and the direction range of the liquid in each intestinal segment under each body position can be analyzed and stored in advance, and then the liquid distribution characteristics associated with each intestinal segment are determined according to the body position of the examination object of the endoscope when the target image frame is collected.
[0084] The following is illustrated by way of example in conjunction with the drawings. Without considering the rotation of the endoscope lens, and when the body position of the examination object is left lateral decubitus, the liquid distribution in each intestinal segment and the posture of the endoscope tip are as shown in Figure 4 Figure 4 As can be seen, when in the left lateral position, the descending colon is located at the lower side, under the action of gravity, most of the liquid in the transverse colon and the ascending colon flows into the descending colon at the lower side, and the transverse colon has little liquid due to its proximity to the vertical. An example of the liquid in the ascending colon, the transverse colon and the descending colon is shown in FIG. 8. Figures 5-7 As can be seen, when in the left lateral position, the descending colon is located at the lower side, under the action of gravity, most of the liquid in the transverse colon and the ascending colon flows into the descending colon at the lower side, and the transverse colon has little liquid due to its proximity to the vertical. An example of the liquid in the ascending colon, the transverse colon and the descending colon is shown in FIG. 8.
[0085] In some embodiments, when in the left lateral position, the direction range associated with the ascending colon includes a first direction range, the direction range associated with the transverse colon includes a second direction range, and the direction range associated with the descending colon includes a third direction range; and the first direction range, the second direction range and the third direction range are independent of each other. That is, in the embodiments of the present application, the direction ranges corresponding to the three intestinal segments are independent of each other, without overlapping or intersecting ranges, to ensure the unique and accurate identification of each intestinal segment. The specific range of the first direction range, the second direction range and the third direction range is not limited here.
[0086] In some embodiments, when in the left lateral position, the first direction range associated with the ascending colon is 66° to 225° in terms of angle, the upper limit of the liquid volume range associated with the transverse colon is a first preset value, the second direction range associated with the transverse colon includes 354.5° to 360° and 0° to 65.5° in terms of angle, and the lower limit of the liquid volume range associated with the descending colon is a second preset value, and the third direction range associated with the descending colon is 225.5° to 354° in terms of angle. Here, the angle refers to the direction angle, and the value range is [0°, 360°), with the positive direction of the x-axis of the plane coordinate system being 0° and increasing in the counterclockwise direction.
[0087] For example, the direction range can also be represented as: the first direction range associated with the ascending colon is 7:30 o'clock to 00:48 o'clock, the second direction range associated with the transverse colon is 00:49 o'clock to 3:11 o'clock, and the third direction range associated with the descending colon is 3:12 o'clock to 7:29 o'clock.
[0088] By analogy, when in the right lateral position, the ascending colon is located at the lower side, under the action of gravity, most of the liquid in the transverse colon and the descending colon flows into the descending colon at the lower side, and the transverse colon has little liquid due to its proximity to the vertical. The liquid distribution characteristics in the following intestinal segments can be obtained in combination with the posture of the endoscope tip.
[0089] In some embodiments, when the body position is right lateral decubitus, the third direction range associated with the descending colon is 66° to 225° in angle, the upper limit of the fluid volume range associated with the transverse colon is a first preset value, the second direction range associated with the transverse colon includes 354.5° to 360° and 0° to 65.5° in angle, and the lower limit of the fluid volume range associated with the ascending colon is a second preset value, and the first direction range associated with the ascending colon is 225.5° to 354° in angle.
[0090] In the actual process of enteroscopy, the lens of the endoscope can be rotated, for example, the operator can control the lens of the endoscope to rotate around the central axis, causing the image sensor to rotate and the pose to change, thereby affecting the distribution data of the liquid in the captured image frame. In some alternative embodiments, in order to adapt to the rotation of the lens and the rotation of the liquid distribution in the image frame, different direction ranges are stored for different lens rotation angles. However, this will cause a waste of resources, so generally a direction range under a certain rotation angle is pre-stored, and then the pre-stored direction range is called in the process of identifying the intestinal segment, and the direction range is adjusted as needed according to the actual rotation angle. In this case, the following specific embodiments can be referred to:
[0091] In some embodiments, when the body position is left lateral decubitus and the rotation angle of the endoscope is 0°, the first direction range associated with the ascending colon is 66° to 225° in angle, the upper limit of the fluid volume range associated with the transverse colon is a first preset value, the second direction range associated with the transverse colon includes 354.5° to 360° and 0° to 65.5° in angle, and the lower limit of the fluid volume range associated with the descending colon is a second preset value, and the third direction range associated with the descending colon is 225.5° to 354° in angle. Here, the angle refers to the direction angle, and its value range is [0°, 360°), with the positive direction of the x-axis of the plane coordinate system being 0° and increasing along the counterclockwise direction.
[0092] In some embodiments, when the body position is right lateral decubitus and the rotation angle of the endoscope is 0°, the third direction range associated with the descending colon is 66° to 225° in angle, the upper limit of the fluid volume range associated with the transverse colon is a first preset value, the second direction range associated with the transverse colon includes 354.5° to 360° and 0° to 65.5° in angle, and the lower limit of the fluid volume range associated with the ascending colon is a second preset value, and the first direction range associated with the ascending colon is 225.5° to 354° in angle.
[0093] The rotation angle of the endoscope can range from 0° to 360°, with the positive direction of the x-axis of the plane coordinate system being 0° and increasing in the counterclockwise direction. Due to the periodicity of rotation, the rotation angle exceeding the range can be summed with an integer multiple of 360° to return to the value range.
[0094] As an optional embodiment of the present application, to avoid the influence of the rotation of the endoscope around the central axis on the first intestinal segment recognition result, one of the target image frame and the direction range of each intestinal segment can be selected for adjustment to eliminate the influence of the rotation of the endoscope. Specifically, the target image frame can not be rotated, but the direction ranges associated with the ascending colon, transverse colon and descending colon can be adjusted according to the rotation angle of the endoscope; or the target image frame can be rotated according to the rotation angle of the endoscope, but the direction ranges associated with the ascending colon, transverse colon and descending colon can not be adjusted. Alternatively, both the target image frame and the direction ranges associated with each intestinal segment can be adjusted, so that the final effect can eliminate the influence of the rotation of the endoscope. Examples include:
[0095] In the case where the body position is left lateral decubitus and the rotation angle of the endoscope is not 0°, the first, second and third direction ranges in the case where the body position is left lateral decubitus and the rotation angle of the endoscope is 0° are adjusted according to the rotation angle.
[0096] In the case where the body position is right lateral decubitus and the rotation angle of the endoscope is not 0°, the first, second and third direction ranges in the case where the body position is right lateral decubitus and the rotation angle of the endoscope is 0° are adjusted according to the rotation angle.
[0097] In each of the above embodiments, the first preset value is generally smaller than the second preset value. When the amount of fluid is identified as the ratio of the number of pixels in the liquid region to the total number of pixels in the image frame, the first preset value and the second preset value can be represented by the ratio accordingly; when the resolution of the image frame is fixed, the amount of fluid can be represented by the number of pixels in the liquid region, and the first preset value and the second preset value can be represented by the number accordingly.
[0098] Optionally, the liquid recognition model can implement both liquid recognition and intestinal segment recognition, in which case the liquid recognition model can output the first intestinal segment recognition result of the target image frame. In this case, the label can include the intestinal segment to which the corresponding sample image belongs. In addition, the label can include information such as the liquid region in the corresponding sample, the amount of fluid of the liquid, and the distribution data of the liquid in the target image frame to assist the training of part of the modules in the liquid recognition model. In addition, the liquid recognition model can be trained for different body positions, and the use process can select the corresponding liquid recognition model according to the body position of the examination object.
[0099] In other optional embodiments, the recognition of the liquid and the intestinal segment can be implemented by different models.
[0100] As an optional embodiment of the present application, if the first intestinal segment recognition result is used as the final intestinal segment recognition result of the target image frame, when the first intestinal segment recognition result is other, the first intestinal segment recognition result of the previous image frame can be used as the first intestinal segment recognition result of the target image frame.
[0101] By identifying the liquid in the current frame from the colon endoscope, the liquid recognition result of the current frame is obtained according to the proportion and distribution of the liquid; and the intestinal segment recognition result of the current frame is determined according to at least the liquid recognition result of the current frame. The liquid in the intestinal tract is generally the secretion of the intestinal wall tissue, and generally exists in the colon bag. Affected by gravity and the body position of the examination object, the amount and distribution of the liquid in different intestinal segments have relatively obvious differences, so the liquid recognition result can more accurately identify the intestinal segment to which the current frame belongs. By introducing the liquid recognition result into the intestinal segment recognition, the accuracy of the intestinal segment recognition can be effectively improved.
[0102] In actual applications, water accumulation does not exist in all positions of each intestinal segment of the large intestine. Therefore, in order to improve the accuracy and reliability of the intestinal segment recognition in actual applications, the recognition of the intestinal cavity and the bend can be introduced synchronously, and the water accumulation and the recognition of the intestinal cavity and the bend are comprehensively analyzed to analyze the intestinal segment photographed by the actual target image frame. At this time, the first intestinal segment recognition result is used as an intermediate result of the intestinal segment recognition to assist in judging the final recognition result (i.e., the comprehensive intestinal segment recognition result). For details, refer to Figure 8 , which shows a flowchart of an intestinal segment recognition method provided by another embodiment of the present application. As an example but not limitation, the method can be applied to the image processing device described above. The present embodiment is a further extension of the embodiment shown in Figure 3 , and the same parts are not repeated.
[0103] S11: Obtain a target image frame photographed by an endoscope on a colon.
[0104] S12: Determine the amount of liquid accumulation and the distribution data of the liquid in the target image frame.
[0105] S13: Determine the first intestinal segment recognition result of the target image frame according to the amount of liquid accumulation and the distribution data of the liquid in the target image frame.
[0106] Wherein, S11 to S13 respectively correspond to S1 to S3 in the embodiment shown in Figure 3 , and for details, refer to the related description of the embodiment shown in Figure 3 , which is not repeated here.
[0107] S14: Obtain the second intestinal segment recognition result and the third intestinal segment recognition result of the target image frame.
[0108] The second intestinal segment recognition result is obtained according to the shape of the intestinal cavity in the target image frame, and the third intestinal segment recognition result is obtained according to the inflection of the intestinal tract in the target image frame. The shape of the intestinal cavity of different intestinal segments can refer to the related description in the foregoing. The shape of the inflection is obviously different from the shape of the non-inflection, and the shape and color of the hepatic flexure and the splenic flexure also differ, according to which whether the target image frame is at an inflection and which inflection it is at can be recognized. The second intestinal segment recognition result and the third intestinal segment recognition result can be obtained by using traditional image recognition or deep learning.
[0109] Like the first intestinal segment recognition result, the type of the second intestinal segment result includes ascending colon, transverse colon, descending colon, and other. The type of the third intestinal segment result includes hepatic flexure, splenic flexure, and other.
[0110] S15: determining a comprehensive intestinal segment recognition result of the target image frame according to at least the first intestinal segment recognition result, the second intestinal segment recognition result, and the third intestinal segment recognition result.
[0111] The second intestinal segment recognition result and the third intestinal segment recognition result are introduced in this embodiment, and are comprehensively judged together with the first intestinal segment recognition result.
[0112] The judgment can be made according to the intestinal tract recognition result of a single frame, and the second intestinal segment recognition result based on the shape of the intestinal cavity has a higher priority. Specifically, in the case where the first intestinal segment recognition result, the third intestinal segment recognition result, and the second intestinal segment recognition result all match, it is determined that the comprehensive intestinal segment recognition result is the second intestinal segment recognition result; or in the case where one of the first intestinal segment recognition result and the third intestinal segment recognition result matches the second intestinal segment recognition result, and the unmatched recognition result is other, it is determined that the comprehensive intestinal segment recognition result is the second intestinal segment recognition result. Here, the matching, for the first intestinal segment recognition result and the second intestinal segment recognition result, means that they are the same; for the second intestinal segment recognition result and the third intestinal segment recognition result, the mapping result of the third intestinal segment recognition result can be obtained according to a pre-defined mapping relationship, and the mapping result is the same as the second intestinal segment recognition result. For example, the mapping result of the hepatic flexure is transverse colon, the mapping result of the splenic flexure is descending colon, and the mapping result of other is still other.
[0113] The judgment can be made according to the intestinal tract recognition result of a single frame, and the first intestinal segment recognition result based on the liquid has a higher priority. Specifically, in the case where the first intestinal segment recognition result is other, the comprehensive intestinal segment recognition result is determined according to the second intestinal segment recognition result and the third intestinal segment recognition result; in the case where the first intestinal segment recognition result is not other, it is determined that the comprehensive intestinal segment recognition result is the first intestinal segment recognition result.
[0114] In a case that there are two different intestinal segment recognition results in the three intestinal segment recognition results, and neither of them is the other, it can be considered that the clear intestinal segment recognition result of the target image frame cannot be directly given, and the comprehensive intestinal segment recognition result of the previous image frame is taken as the comprehensive intestinal segment recognition result of the target image frame, or, in all intestinal segment recognition results that are not the other, the one with the highest priority is selected as the comprehensive intestinal segment recognition result of the target image frame.
[0115] The intestinal tract recognition results of consecutive multiple frames can be judged. Specifically, the first recognition result, the second recognition result and the third recognition result of the intestinal segment can be determined according to the first intestinal segment recognition result, the second intestinal segment recognition result and the third intestinal segment recognition result of each image frame in the first image frame; the first image frame includes multiple historical image frames and the target image frame within a preset time range ending at a target time, and the target time is the collection time of the target image frame; and the comprehensive intestinal segment recognition result of the target image frame is determined according to the first recognition result, the second recognition result and the third recognition result. Since the image sensor generally takes pictures at a fixed frame rate, the target time here can be represented by the serial number of the target image frame, and correspondingly, the time range can be represented by the number of frames.
[0116] The preset time range includes a first time range, a second time range and a third time range, and the specific determination criteria of the first recognition result, the second recognition result and the third recognition result can include:
[0117] When the first intestinal segment recognition results of at least a first number of consecutive image frames in the first image frame within the first time range are the same, the same first intestinal segment recognition result is taken as the first recognition result.
[0118] When the second intestinal segment recognition results of at least a second number of consecutive image frames in the first image frame within the second time range are the same, the same second intestinal segment recognition result is taken as the second recognition result.
[0119] When the third intestinal segment recognition results of at least a third number of consecutive image frames in the first image frame within the third time range are the same, the same third intestinal segment recognition result is taken as the third recognition result.
[0120] If the above criteria cannot be met, the corresponding recognition result can be the other. Optionally, the first time range and the second time range are both smaller than the third time range. The first number and the second number are both greater than the third number. Optionally, the first time range is approximately equal to the second time range, and the first number is approximately equal to the second number. Here, approximately equal means that the values of the two are close, and more specifically, the absolute value of the difference between the two is less than a set threshold.
[0121] The manner of judging in combination of the first identification result, the second identification result and the third identification result can refer to the aforementioned comprehensive judgment using the intestinal tract identification result of a single frame.
[0122] The second identification result based on the intestinal cavity shape can be set to have a higher priority. Specifically, in a case where the first identification result and the third identification result are both matched with the second identification result, the comprehensive intestinal tract identification result is determined as the second identification result; or in a case where one of the first identification result and the third identification result is matched with the second identification result, and the unmatched identification result is other, the comprehensive intestinal tract identification result is determined as the second identification result.
[0123] The first identification result based on the liquid can be set to have a higher priority. Specifically, in a case where the first identification result is other, the comprehensive intestinal tract identification result is determined according to the second identification result and the third identification result; in a case where the first identification result is not other, the comprehensive intestinal tract identification result is determined as the first identification result.
[0124] In a case where there are two different identification results in the three identification results, and neither of them is other, it can be considered that it is difficult to directly give an explicit intestinal tract identification result of the target image frame, and the comprehensive intestinal tract identification result of the previous image frame is taken as the comprehensive intestinal tract identification result of the target image frame, or, in all identification results that are not other, the one with the highest priority is selected as the comprehensive intestinal tract identification result of the target image frame.
[0125] Since even in an intestinal tract with a relatively large amount of liquid, liquid can not be detected in every image frame, for example, liquid often does not exist near a bend, and in addition, there is a case where the operator sucks the liquid in order to observe the intestinal wall, therefore, the embodiment combines the first intestinal tract identification result based on the liquid with the second intestinal tract identification result based on the intestinal cavity shape and the second intestinal tract identification result based on the bend, and comprehensively judges to obtain the comprehensive intestinal tract identification result of the target image frame, which can better adapt to different regions in the colon and further improve the accuracy of intestinal tract identification.
[0126] Figure 9 A flowchart of an intestinal tract identification method provided by another embodiment of the present application is shown, which is an example and is not limited to the method, and can be applied to the image processing device described above. The embodiment is a further extension of the embodiment shown in Figure 3 The same parts of the embodiment are not repeated.
[0127] S21: Obtain a target image frame captured by an endoscope on a colon.
[0128] S22: Determine the amount of liquid accumulation of liquid in the target image frame and the distribution data of the liquid in the target image frame.
[0129] S23: determining a first intestinal segment recognition result of the target image frame according to the amount and distribution data of the liquid in the target image frame.
[0130] S21-S23 respectively correspond to S1-S3 in the embodiment shown in Figure 3 FIG. 1, and specific details can be referred to the related description of the embodiment shown in Figure 3 FIG. 1, which will not be repeated here.
[0131] S24: obtaining a second intestinal segment recognition result of the target image frame.
[0132] The second intestinal segment recognition result is obtained according to the shape of the intestinal cavity in the target image frame. The second intestinal segment recognition result can be obtained by traditional image recognition or deep learning.
[0133] S25: determining a comprehensive intestinal segment recognition result of the target image frame according to at least the first intestinal segment recognition result and the second intestinal segment recognition result.
[0134] The embodiment is mainly different from the embodiment shown in Figure 8 FIG. 1 in that only the second intestinal segment recognition result is introduced, and the third intestinal segment recognition result is not introduced to be comprehensively judged together with the first intestinal segment recognition result.
[0135] The judgment can be made according to the intestinal tract recognition result of a single frame. Specifically, in the case that the first intestinal segment recognition result matches the second intestinal segment recognition result, the comprehensive intestinal segment recognition result is determined to be the first intestinal segment recognition result. The judgment can be made according to the intestinal tract recognition result of a single frame, and the first intestinal segment recognition result based on the liquid has a higher priority. In the case that one of the first intestinal segment recognition result and the second intestinal segment recognition result is other, the comprehensive intestinal segment recognition result is determined to be the other intestinal segment recognition result.
[0136] In the case that the two intestinal segment recognition results are different and neither of them is other, it can be considered that the explicit intestinal segment recognition result of the target image frame cannot be directly given, and the comprehensive intestinal segment recognition result of the previous image frame is taken as the comprehensive intestinal segment recognition result of the target image frame. Alternatively, a preset priority can be referred to, and the intestinal segment recognition result with the highest priority is selected as the comprehensive intestinal segment recognition result of the target image frame.
[0137] The determination can be made according to the intestinal tract recognition results of continuous multiple frames. Specifically, the first recognition result and the second recognition result of the intestinal tract can be respectively determined according to the first intestinal tract recognition result and the second intestinal tract recognition result of each image frame in the first image frame. The first image frame includes multiple historical image frames in a preset time range ending at a target time and the target image frame. The target time is the acquisition time of the target image frame. The comprehensive intestinal tract recognition result of the target image frame is determined according to the first recognition result and the second recognition result.
[0138] The preset time range can include a first time range and a second time range. The specific determination criteria of the first recognition result, the second recognition result and the third recognition result can include:
[0139] When the first intestinal tract recognition results of at least a first number of continuous image frames in the first image frame in the first time range are the same, the same first intestinal tract recognition result is taken as the first recognition result.
[0140] When the second intestinal tract recognition results of at least a second number of continuous image frames in the first image frame in the second time range are the same, the same second intestinal tract recognition result is taken as the second recognition result.
[0141] If the above criteria cannot be met, the corresponding recognition result can be other. Optionally, the first time range is approximately equal to the second time range, and the first number is approximately equal to the second number. Here, approximately equal means that the values of the two are close, and more specifically, the absolute value of the difference between the two is less than a set threshold.
[0142] The manner of combining the first recognition result and the second recognition result for judgment can refer to the aforementioned comprehensive judgment using single-frame intestinal tract recognition result. Specifically, in the case where the first recognition result matches the second recognition result, the comprehensive intestinal tract recognition result is determined as the first recognition result. In the case where one of the first recognition result and the second recognition result is other, the comprehensive intestinal tract recognition result is determined as the other recognition result that is not other.
[0143] In the case where the two recognition results are different and neither of them is other, it can be considered that it is difficult to directly give an explicit intestinal tract recognition result of the target image frame. The comprehensive intestinal tract recognition result of the previous image frame is taken as the comprehensive intestinal tract recognition result of the target image frame, or the recognition result with the highest priority can be selected as the comprehensive intestinal tract recognition result of the target image frame according to a preset priority.
[0144] Through the implementation of the embodiment, the first intestinal tract recognition result based on liquid and the second intestinal tract recognition result based on intestinal cavity morphology are combined to comprehensively determine the comprehensive intestinal tract recognition result of the target image frame.
[0145] Since liquid cannot be detected in each image frame even in a segment with a large amount of liquid, for example, liquid often does not exist near a bend, and in addition, the operator sucks liquid for the purpose of observing the intestinal wall, the embodiment combines the first intestinal segment recognition result based on liquid and the second intestinal segment recognition result based on the intestinal cavity shape, comprehensively judges the comprehensive intestinal segment recognition result of the target image frame, can better adapt to different regions in the colon, and further improves the accuracy of intestinal segment recognition.
[0146] The complete process of the intestinal segment recognition method provided by the embodiment is described below with reference to the accompanying drawings. As shown in Figure 10 The embodiment includes:
[0147] S101: Obtain a target image frame based on the colon shot by an endoscope.
[0148] S102: Identify whether the target image frame is an information frame.
[0149] The purpose of information frame screening is to screen out non-information frames that are not suitable for intestinal segment recognition, such as dynamic blur, mirror blur, highlight reflection, dark screen, etc. In the information frame screening in the related art, image frames with liquid are often discarded as non-information frames with water stains, and are retained as information frames in the embodiment.
[0150] In the case where the target image frame is an information frame, jump to S103; in the case where the target image frame is not an information frame, do not perform intestinal segment recognition on the target image frame.
[0151] S103: Identify whether to enter the mirror withdrawal phase according to the target image frame.
[0152] The recognition of the mirror withdrawal phase can include two parts, one is to recognize the ileocecal part, and the other is to identify whether the trend of screen movement is mirror withdrawal in combination with the target image frame and a plurality of historical frames. The ileocecal part refers to the part where the terminal ileum and the cecum intersect each other, mainly including the ileocecal orifice, ileocecal valve, appendix orifice, etc. Recognizing the ileocecal part from the target image frame means that the tip of the endoscope has reached the part where the large intestine and the small intestine intersect, and under normal circumstances, the subsequent movement is likely to be staying in the ileocecal part or withdrawing the mirror, and will not continue to move forward, so recognizing the ileocecal part means that the mirror insertion is ended and the mirror withdrawal phase is entered. The recognition of the ileocecal part and the trend of screen movement can be realized based on at least one of image recognition and deep learning.
[0153] In the case of determining the entering of the retreat stage by recognizing the ileocecal valve, it means that the ileocecal valve is successfully recognized from the target image frame, that is, the intestinal segment corresponding to the target image frame must be the ascending colon connected with the cecum, and jumping to S104; in the case of determining the entering of the retreat stage by recognizing the moving trend of the image, since the comprehensive intestinal segment recognition result of the target image frame cannot be directly determined like the ileocecal valve recognition, jumping to S105; in the case of determining that the retreat stage is not entered, the intestinal segment recognition is not performed on the target image frame.
[0154] S104: taking the ascending colon as the comprehensive intestinal segment recognition result of the target image frame.
[0155] S105: obtaining the first, second and third intestinal segment recognition results of the target image frame according to the liquid, the intestinal cavity shape and the bending respectively, and determining the comprehensive intestinal segment recognition result of the target image frame according to at least the first intestinal segment recognition result, the second intestinal segment recognition result and the third intestinal segment recognition result.
[0156] The specific process can refer to the related description of the foregoing embodiments. The following is described by taking the judgment according to the intestinal tract recognition results of continuous multiple frames as an example.
[0157] The comprehensive intestinal segment recognition result of the previous image frame can be obtained first.
[0158] Case one: the comprehensive intestinal segment recognition result of the previous image frame is other, which usually means that the intestinal segment photographed by the endoscope is not clear shortly after the entering of the retreat stage is determined by recognizing the moving trend of the image, in this case, if the first intestinal segment recognition result and the second intestinal segment recognition result of n1 continuous frames are the same within a time range with the target time as the end point and t1 seconds as the time length, the comprehensive intestinal segment recognition result of the target image frame is updated to the first intestinal segment recognition result.
[0159] Case two: the comprehensive intestinal segment recognition result of the previous image frame is the ascending colon, which usually means that the entering of the retreat stage is determined by recognizing the ileocecal valve or by recognizing the moving trend of the image, and the intestinal segment photographed by the endoscope has been determined as the ascending colon, in this case, if the three intestinal segment recognition results meet any of the following scenarios a-c, the comprehensive intestinal segment recognition result is updated to the transverse colon, otherwise the existing comprehensive intestinal segment recognition result is kept unchanged.
[0160] Scenario a: the first intestinal segment recognition result of n2 continuous frames is the transverse colon within a time range with the target time as the end point and t2 seconds as the time length; the second intestinal segment recognition result of n3 continuous frames is the transverse colon within a time range with the target time as the end point and t3 seconds as the time length; the third intestinal segment recognition result of n4 continuous frames is the hepatic flexure within a time range with the target time as the end point and t4 seconds as the time length.
[0161] Scenario b: In a time range with the target time as the end point and a time length of t2 seconds, the first intestinal segment recognition result of n2 continuous frames is other; in a time range with the target time as the end point and a time length of t3 seconds, the second intestinal segment recognition result of n3 continuous frames is transverse colon; in a time range with the target time as the end point and a time length of t4 seconds, the third intestinal segment recognition result of n4 continuous frames is hepatic flexure.
[0162] Scenario c: In a time range with the target time as the end point and a time length of t2 seconds, the first intestinal segment recognition result of n2 continuous frames is transverse colon; in a time range with the target time as the end point and a time length of t3 seconds, the second intestinal segment recognition result of n3 continuous frames is transverse colon; in a time range with the target time as the end point and a time length of t4 seconds, the third intestinal segment recognition result of n4 continuous frames is other.
[0163] Case three: The comprehensive intestinal segment recognition result of the previous image frame is transverse colon, which usually means that the recognition of ileocecal recess is adopted to determine the entry into the mirror withdrawal stage, or the recognition of picture moving trend is adopted to determine the entry into the mirror withdrawal stage in the ascending colon, and the comprehensive intestinal segment recognition result has been updated once, or the recognition of picture moving trend is adopted to determine the entry into the mirror withdrawal stage, and it has been determined that the intestinal segment photographed by the endoscope is transverse colon before. In this case, if the three intestinal segment recognition results meet any of the following scenarios d-f, the comprehensive intestinal segment recognition result is updated to descending colon, otherwise the existing comprehensive intestinal segment recognition result remains unchanged.
[0164] Scenario d: In a time range with the target time as the end point and a time length of t5 seconds, the first intestinal segment recognition result of n5 continuous frames is descending colon; in a time range with the target time as the end point and a time length of t6 seconds, the second intestinal segment recognition result of n6 continuous frames is descending colon; in a time range with the target time as the end point and a time length of t7 seconds, the third intestinal segment recognition result of n7 continuous frames is splenic flexure.
[0165] Scenario e: In a time range with the target time as the end point and a time length of t5 seconds, the first intestinal segment recognition result of n5 continuous frames is other; in a time range with the target time as the end point and a time length of t6 seconds, the second intestinal segment recognition result of n6 continuous frames is descending colon; in a time range with the target time as the end point and a time length of t7 seconds, the third intestinal segment recognition result of n7 continuous frames is splenic flexure.
[0166] Scene f: in the time range with the target time as the end point and t5 seconds as the length, there are n5 continuous frames of the first intestinal segment recognition result being descending colon; in the time range with the target time as the end point and t6 seconds as the length, there are n6 continuous frames of the second intestinal segment recognition result being descending colon; in the time range with the target time as the end point and t7 seconds as the length, there are n7 continuous frames of the third intestinal segment recognition result being other.
[0167] wherein, since the continuous frames of the bend are less, and compared with the intestinal segment, the size of the bend part is smaller, and the descending colon is shorter than the transverse colon, generally t2≈t3<t4, t5≈t6<t7, n4≈n7, n2≈n3, n5≈n6, n7<n5<n3.
[0168] After abandoning the target image frame or completing the intestinal segment recognition of the target image frame, the target image frame can be updated, and then the above process is executed again.
[0169] Figure 11 A structure schematic diagram of an intestinal segment recognition device provided by an embodiment of the present application is shown, which comprises an acquisition module 101 and a recognition module 102.
[0170] The acquisition module 101 is configured to acquire a target image frame based on the colon shot by an endoscope.
[0171] The recognition module 102 is configured to determine a first intestinal segment recognition result of the target image frame according to the liquid accumulation amount and distribution data of the liquid in the target image frame, the intestinal segment recognition result comprising ascending colon, transverse colon, descending colon or other.
[0172] Optionally, the ascending colon, the transverse colon and the descending colon each have an associated liquid accumulation amount range and a direction range; correspondingly, the recognition module 102 is configured to determine the first intestinal segment recognition result as a target intestinal segment in the case that the liquid accumulation amount of the liquid in the target image frame belongs to the associated liquid accumulation amount range of the target intestinal segment, and the distribution data belongs to the associated direction range of the target intestinal segment, the target intestinal segment being one of the ascending colon, the transverse colon and the descending colon.
[0173] Optionally, the intestinal segment recognition device further comprises a first determination module (not shown in the figure), configured to determine the associated liquid accumulation amount range and the direction range of the ascending colon, the transverse colon and the descending colon according to the body position of the object of the endoscope.
[0174] Optionally, the intestinal segment recognition device further comprises a rotation module (not shown in the figure), configured to adjust the associated direction range of the ascending colon, the transverse colon and the descending colon according to the rotation angle of the endoscope.
[0175] Optionally, in the case of a left lateral position, the direction range associated with the ascending colon is 66° to 225° in angle, the upper limit of the effusion amount range associated with the transverse colon is a first preset value, the direction range associated with the transverse colon is 354.5° to 360° and 0° to 65.5° in angle, the lower limit of the effusion amount range associated with the descending colon is a second preset value, and the direction range associated with the descending colon is 225.5° to 354° in angle.
[0176] Optionally, in the case of a right lateral position, the direction range associated with the descending colon is 66° to 225° in angle, the upper limit of the effusion amount range associated with the transverse colon is a first preset value, the direction range associated with the transverse colon is 354.5° to 360° and 0° to 65.5° in angle, the lower limit of the effusion amount range associated with the ascending colon is a second preset value, and the direction range associated with the ascending colon is 225.5° to 354° in angle.
[0177] Optionally, the intestinal segment recognition device further comprises a second determination module (not shown in the figure) and a first comprehensive module (not shown in the figure). The second determination module is configured to obtain a second intestinal segment recognition result and a third intestinal segment recognition result of the target image frame. The second intestinal segment recognition result is obtained according to the shape of the intestinal cavity in the target image frame, and the third intestinal segment recognition result is obtained according to the bending of the intestinal tract in the target image frame. The first comprehensive module is configured to determine a comprehensive intestinal segment recognition result of the target image frame according to at least the first intestinal segment recognition result, the second intestinal segment recognition result and the third intestinal segment recognition result.
[0178] Optionally, the second determination module comprises a first determination unit (not shown in the figure) and a first comprehensive unit (not shown in the figure). The first determination unit is configured to determine a first recognition result, a second recognition result and a third recognition result of the intestinal segment according to the first intestinal segment recognition result, the second intestinal segment recognition result and the third intestinal segment recognition result of each image frame in the first image frame, respectively. The first image frame comprises a plurality of historical image frames and the target image frame within a preset time range ending at a target time. The target time is the collection time of the target image frame. The first comprehensive unit is configured to determine a comprehensive intestinal segment recognition result of the target image frame according to the first recognition result, the second recognition result and the third recognition result.
[0179] Optionally, the preset time range comprises a first time range, a second time range and a third time range; the first determining unit is configured to determine the same first intestinal segment recognition result as the first recognition result when the first intestinal segment recognition results of at least a first number of continuous image frames in the first image frames in the first time range are the same; determine the same second intestinal segment recognition result as the second recognition result when the second intestinal segment recognition results of at least a second number of continuous image frames in the first image frames in the second time range are the same; and determine the same third intestinal segment recognition result as the third recognition result when the third intestinal segment recognition results of at least a third number of continuous image frames in the first image frames in the third time range are the same.
[0180] Optionally, the first time range and the second time range are both less than the third time range.
[0181] Optionally, the first number and the second number are both greater than the third number.
[0182] Optionally, the first comprehensive unit is configured to determine the comprehensive intestinal segment recognition result as the second recognition result when the first recognition result and the third recognition result both match the second recognition result, or determine the comprehensive intestinal segment recognition result as the second recognition result when one of the first recognition result and the third recognition result matches the second recognition result and the unmatched recognition result is the other.
[0183] Optionally, the first comprehensive unit is configured to determine the comprehensive intestinal segment recognition result according to the second recognition result and the third recognition result when the first recognition result is the other, and determine the comprehensive intestinal segment recognition result as the first recognition result when the first recognition result is not the other.
[0184] Optionally, the first comprehensive module is configured to determine the comprehensive intestinal segment recognition result of the target image frame according to only the first intestinal segment recognition result, the second intestinal segment recognition result and the third intestinal segment recognition result of the target image frame.
[0185] Optionally, the first comprehensive module is configured to determine the comprehensive intestinal segment recognition result as the second intestinal segment recognition result when the first intestinal segment recognition result and the third intestinal segment recognition result both match the second intestinal segment recognition result, or determine the comprehensive intestinal segment recognition result as the second intestinal segment recognition result when one of the first intestinal segment recognition result and the third intestinal segment recognition result matches the second intestinal segment recognition result and the unmatched recognition result is the other.
[0186] Optionally, the first comprehensive module is configured to determine the comprehensive intestinal segment recognition result according to the second intestinal segment recognition result and the third intestinal segment recognition result when the first intestinal segment recognition result is the other, and determine the comprehensive intestinal segment recognition result as the first intestinal segment recognition result when the first intestinal segment recognition result is not the other.
[0187] Optionally, the first determining module is configured to input the target image frame into a pre-trained liquid recognition model to obtain the liquid volume and the distribution data of the liquid in the target image frame, wherein the liquid recognition model is trained by using liquid recognition samples, and the liquid recognition samples have liquid in part or all of the sample images and carry labels associated with the liquid volume and the distribution data of the liquid.
[0188] Optionally, the recognition module 102 is configured to input the target image frame into a pre-trained liquid recognition model to obtain the first intestinal segment recognition result of the target image frame; the liquid recognition model is trained by using liquid recognition samples, and the liquid recognition samples have liquid in part or all of the sample images and carry labels associated with the liquid volume and the distribution data of the liquid.
[0189] Optionally, the intestinal segment recognition device further comprises a third determining module (not shown in the figure) and a second synthesizing module (not shown in the figure), the third determining module is configured to obtain a second intestinal segment recognition result of the target image frame, and the second intestinal segment recognition result is obtained according to the shape of the intestinal cavity in the target image frame; and the second synthesizing module is configured to determine a comprehensive intestinal segment recognition result of the target image frame according to at least the first intestinal segment recognition result and the second intestinal segment recognition result.
[0190] Optionally, the third determining module comprises a second determining unit (not shown in the figure) and a second synthesizing unit (not shown in the figure), and the second determining unit is configured to determine a first recognition result and a second recognition result of the intestinal segment respectively according to the first intestinal segment recognition result and the second intestinal segment recognition result of each image frame in the first image frame, and the first image frame comprises a plurality of historical image frames and the target image frame in a preset time range ending at a target time, and the target time is the collection time of the target image frame; and the second synthesizing unit is configured to determine the comprehensive intestinal segment recognition result of the target image frame according to the first recognition result and the second recognition result.
[0191] Optionally, the preset time range comprises a first time range and a second time range; and the second determining unit is configured to, when the first intestinal segment recognition results of at least a first number of continuous image frames in the first image frame in the first time range are the same, take the same first intestinal segment recognition result as the first recognition result; and when the second intestinal segment recognition results of at least a second number of continuous image frames in the first image frame in the second time range are the same, take the same second intestinal segment recognition result as the second recognition result.
[0192] Optionally, the second determining unit is configured to, in the case that the first recognition result matches the second recognition result, determine that the comprehensive intestinal segment recognition result is the first recognition result.
[0193] Optionally, the second determining unit is configured to determine the comprehensive intestinal segment recognition result as the other one of the first intestinal segment recognition result and the second intestinal segment recognition result in a case that one of the first intestinal segment recognition result and the second intestinal segment recognition result is other.
[0194] Optionally, the second determining unit is configured to determine the comprehensive intestinal segment recognition result of the target image frame according to the first intestinal segment recognition result and the second intestinal segment recognition result of the target image frame.
[0195] Optionally, the second determining unit is configured to determine the comprehensive intestinal segment recognition result as the first intestinal segment recognition result in a case that the first intestinal segment recognition result matches the second intestinal segment recognition result.
[0196] Optionally, the second determining unit is configured to determine the comprehensive intestinal segment recognition result as the other one of the first intestinal segment recognition result and the second intestinal segment recognition result in a case that one of the first intestinal segment recognition result and the second intestinal segment recognition result is other.
[0197] It should be noted that the information interaction, execution process and the like between the above apparatus / module / unit are based on the same concept as the method embodiments of the present application, and the specific functions and the technical effects brought by the same can be referred to the method embodiments part, and will not be repeated here.
[0198] It should be understood that the size of the serial number of each step in the above embodiments does not mean the execution order, and the execution order of each process should be determined according to its function and inherent logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.
[0199] Those skilled in the art can clearly understand that, for the convenience and brevity of description, only the above division of each functional unit and module is exemplified, and in actual application, the above functions can be completed by different functional units and modules according to needs, that is, the internal structure of the device is divided into different functional units or modules to complete all or part of the functions described above. Each functional unit and module in the embodiments can be integrated in one processing unit, or each unit can exist physically, or two or more units can be integrated in one unit. The above integrated unit can be realized in the form of hardware or software. In addition, the specific names of each functional unit and module are only for easy distinction, and do not limit the protection scope of the present application. The specific working process of the unit and module in the above system can be referred to the corresponding process in the foregoing method embodiments, which will not be repeated here.
[0200] The embodiments of the present application also provide a computer readable storage medium, the computer readable storage medium stores a computer program, and the computer program is executed by a processor to realize the steps in each of the above method embodiments.
[0201] The integrated unit, if implemented in the form of a software function unit and sold or used as an independent product, can be stored in a computer readable storage medium. Based on such understanding, the present application can implement all or part of the processes in the above-mentioned embodiment methods through a computer program to instruct relevant hardware to complete, and the computer program can be stored in a computer readable storage medium. When the computer program is executed by a processor, the steps of each method embodiment described above can be implemented. The computer program includes computer program code, which can be in the form of source code, object code, executable files or some intermediate forms. The computer readable medium can at least include any entity or device capable of carrying the computer program code to the photographing device / electronic device, recording medium, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signal, telecommunication signal and software distribution medium. For example, U disk, mobile hard disk, magnetic disk or optical disk, etc. In some jurisdictions, according to legislation and patent practice, the computer readable medium can not be an electrical carrier signal and a telecommunication signal.
[0202] In the above embodiments, the description of each embodiment has its own focus, and the parts not described or recorded in detail in a certain embodiment can be referred to the relevant description of other embodiments.
[0203] Those of ordinary skill in the art can realize that the units and algorithm steps of each example described in connection with the embodiments disclosed herein can be realized by electronic hardware, or a combination of computer software and electronic hardware. Whether the functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the present application.
[0204] In the embodiments provided by the present application, it should be understood that the disclosed apparatus / network device and method can be implemented in other ways. For example, the above-described apparatus / network device embodiments are merely schematic, for example, the division of the modules or units is only a logical function division, and actual implementation can have another division manner, for example, a plurality of units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the shown or discussed each other can be indirect coupling or communication connection through some interfaces, devices or units, and can be electrical, mechanical or other forms.
[0205] The units described as separate components may or may not be physically separate, and the components displayed as units may or may not be physical units, that is, may be located in one place, or may also be distributed to multiple network units. Part or all of the units can be selected to achieve the purpose of the embodiment scheme according to actual needs.
[0206] The above embodiments are only used to illustrate the technical solutions of the present application, but not limit them; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that the technical solutions recorded in the foregoing embodiments can still be modified, or some technical features can be replaced by equivalents; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application, and should be included in the protection scope of the present application.
Claims
1. A method of identifying a segment of intestine, characterized by, The method comprises: acquiring a target image frame shot by an endoscope on a colon; determining a first intestinal segment identification result of the target image frame according to a liquid accumulation amount and liquid distribution data in the target image frame, the first intestinal segment identification result comprising ascending colon, transverse colon or descending colon; wherein the liquid distribution characteristics associated with each intestinal segment are determined according to the body position of the examination object of the endoscope when the target image frame is collected; the liquid distribution characteristics associated with each intestinal segment under each body position are analyzed and stored in advance.
2. The method of claim 1, wherein, The ascending colon, the transverse colon and the descending colon each have an associated liquid accumulation amount range and a direction range; Correspondingly, the determination of the first intestinal segment identification result of the target image frame according to the liquid accumulation amount and the liquid distribution data in the target image frame comprises: in the case that the liquid accumulation amount in the target image frame belongs to the liquid accumulation amount range associated with a target intestinal segment, and the distribution data belongs to the direction range associated with the target intestinal segment, determining that the first intestinal segment identification result is the target intestinal segment, the target intestinal segment being one of the ascending colon, the transverse colon and the descending colon.
3. The method of claim 2, wherein, before the determination of the first intestinal segment identification result as the target intestinal segment, the method further comprises: determining the liquid accumulation amount range and the direction range associated with the ascending colon, the transverse colon and the descending colon according to the body position of the examination object of the endoscope.
4. The method of claim 3, wherein, after the determination of the liquid accumulation amount range and the direction range associated with the ascending colon, the transverse colon and the descending colon according to the body position of the examination object of the endoscope, the method further comprises: adjusting the direction range associated with the ascending colon, the transverse colon and the descending colon according to the rotation angle of the endoscope.
5. The method of claim 3, wherein, the determination of the liquid accumulation amount range and the direction range associated with the ascending colon, the transverse colon and the descending colon according to the body position of the examination object of the endoscope comprises: in the case that the body position is left lateral position, the direction range associated with the ascending colon comprises a first direction range, the direction range associated with the transverse colon comprises a second direction range, and the direction range associated with the descending colon comprises a third direction range; wherein the first direction range, the second direction range and the third direction range are independent of each other.
6. The method of claim 5, wherein, the first direction range expressed in angle comprises 66° to 225°, the upper limit of the liquid accumulation amount range associated with the transverse colon is a first preset value, the second direction range expressed in angle comprises 354.5° to 360° and 0° to 65.5°, the lower limit of the liquid accumulation amount range associated with the descending colon is a second preset value, and the third direction range expressed in angle comprises 225.5° to 354°. The method further comprises: acquiring a second intestinal segment identification result and a third intestinal segment identification result of the target image frame, the second intestinal segment identification result being obtained according to the shape of the intestinal cavity in the target image frame, and the third intestinal segment identification result being obtained according to the bending of the intestinal tract in the target image frame; 7. The method of claim 1, wherein, Determine a comprehensive intestinal segment identification result of the target image frame according to the first intestinal segment identification result, the second intestinal segment identification result and the third intestinal segment identification result.
8. The method of claim 7, wherein, The determining of the comprehensive intestinal segment identification result of the target image frame according to the first intestinal segment identification result, the second intestinal segment identification result and the third intestinal segment identification result comprises: According to the first intestinal segment identification result, the second intestinal segment identification result and the third intestinal segment identification result of each of the first image frames, respectively determine a first identification result, a second identification result and a third identification result of an intestinal segment; the first image frames include a plurality of historical image frames within a preset time range ending at a target time and the target image frame, and the target time is a collection time of the target image frame; Determine a comprehensive intestinal segment identification result of the target image frame according to the first identification result, the second identification result and the third identification result.
9. The method of claim 8, wherein, The preset time range includes a first time range, a second time range and a third time range; The determining of the first identification result, the second identification result and the third identification result of an intestinal segment according to the first intestinal segment identification result, the second intestinal segment identification result and the third intestinal segment identification result of each of the first image frames comprises: When the first intestinal segment identification result of at least a first number of continuous image frames in the first image frames within the first time range is the same, the same first intestinal segment identification result is taken as the first identification result; When the second intestinal segment identification result of at least a second number of continuous image frames in the first image frames within the second time range is the same, the same second intestinal segment identification result is taken as the second identification result; When the third intestinal segment identification result of at least a third number of continuous image frames in the first image frames within the third time range is the same, the same third intestinal segment identification result is taken as the third identification result.
10. The method of claim 8, wherein, The first intestinal segment identification result further includes others; The determining of the comprehensive intestinal segment identification result of the target image frame according to the first identification result, the second identification result and the third identification result comprises: In a case where the first identification result and the third identification result both match the second identification result, it is determined that the comprehensive intestinal segment identification result is the second identification result; or in a case where one of the first identification result and the third identification result matches the second identification result, and the unmatched identification result is others, it is determined that the comprehensive intestinal segment identification result is the second identification result.
11. The method of any one of claims 1 to 10, wherein, Before the determining of the first intestinal segment identification result of the target image frame according to the liquid accumulation amount and the liquid distribution data in the target image frame, further comprising: inputting the target image frame into a pre-trained liquid recognition model to obtain liquid accumulation amount and liquid distribution data of the liquid in the target image frame, wherein the liquid recognition model is trained by using liquid recognition samples, and the liquid recognition samples have liquid in part or all of sample images and carry labels associated with the liquid accumulation amount and the liquid distribution data.
12. The method of any one of claims 1, 2, and 6-10, wherein, The method further includes determining a first intestinal segment recognition result of the target image frame according to the liquid accumulation amount and the liquid distribution data of the liquid in the target image frame. The method further includes inputting the target image frame into a pre-trained liquid recognition model to obtain the first intestinal segment recognition result of the target image frame, wherein the liquid recognition model is trained by using liquid recognition samples, and the liquid recognition samples have liquid in part or all of sample images and carry labels associated with the liquid accumulation amount and the liquid distribution data.
13. An intestinal segment identification device, characterized by The apparatus includes: an acquisition module configured to acquire a target image frame captured by a colonoscope; a recognition module configured to determine a first intestinal segment recognition result of the target image frame according to liquid accumulation amount and liquid distribution data of the liquid in the target image frame, wherein the first intestinal segment recognition result includes ascending colon, transverse colon, or descending colon, and wherein liquid distribution features associated with each intestinal segment are determined according to a body position of an examination object of the colonoscope when the target image frame is captured, and the liquid distribution features associated with each intestinal segment in each body position are pre-analyzed and stored.
14. An image processing apparatus comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, The processor implements the method of any one of claims 1 to 12 when executing the computer program.
15. An endoscope system characterized by comprising: The apparatus includes: a display device, an image processing device according to claim 14, and a colonoscope.
Citation Information
Patent Citations
Method and equipment for segmenting large intestine effusion and storage medium
CN119832014A
