Endoscope auxiliary diagnosis equipment and diagnosis prompting method thereof

Abstract

The invention relates to the field of medical instruments, and discloses endoscope auxiliary diagnosis equipment and a diagnosis prompting method thereof, and the endoscope auxiliary diagnosis equipment comprises an endoscope, a first processing module, a second processing module and a display module; the first processing module is used for receiving an original image collected by the endoscope and processing the original image to obtain a corresponding real-time image and physiological structure part information; the second processing module processes the real-time image based on the physiological structure part information to obtain image result information; the image result information comprises diagnosis result information and auxiliary reminding information; the display module comprises a picture unit and a prompt unit; the real-time picture area of the picture unit package is used for displaying an original image or a real-time image in real time; the prompt unit comprises a risk prompt area used for displaying diagnosis result information and an auxiliary prompt area used for displaying auxiliary prompt information. According to the invention, the endoscope physician can be reminded in time through the real-time image result information, so that the reliability of the auxiliary diagnosis result is improved.

Description

Technical Field

[0001] This application relates to the technical field of medical devices, and in particular to an endoscopic assisted diagnosis device and a diagnosis prompting method thereof. Background Art

[0002] In recent years, with the rapid development of endoscopic technology and artificial intelligence (AI), the combination of the two provides doctors with a more accurate examination tool. When an endoscopic intelligent system is in use, it usually needs to use the image recognition function of AI to intelligently process the acquired original images to detect suspected lesion areas such as cancer, and then display the processed result data through an external display.

[0003] Although the use of the image recognition function of AI can help doctors quickly identify suspected lesion areas during endoscopic assisted diagnosis, the current endoscopic intelligent systems require two sets of machines or two displays to run simultaneously. Doctors need to pay attention to two screens at the same time, which is likely to cause visual fatigue. Moreover, some irrelevant or incorrect markings may interfere with doctors' attention. In addition, when the endoscope moves quickly (such as operations like advancing the endoscope, retracting the endoscope, and angling), the images in the video stream change too fast, often causing the drift of the auxiliary marking frames, thus affecting doctors' judgment. These problems not only violate the original intention of AI-assisted diagnosis and providing an efficient and convenient diagnosis method for endoscopic physicians, but also may increase the risk of missed diagnosis of lesions. Summary of the Invention

[0004] In view of this, the embodiments of this application provide an endoscopic assisted diagnosis device and a diagnosis prompting method thereof, which can effectively solve problems such as missed diagnosis of lesions caused by endoscopic assisted diagnosis.

[0005] In a first aspect, the embodiments of this application provide an endoscopic assisted diagnosis device, including an endoscope, a first processing module, a second processing module, and a display module;

[0006] The first processing module is configured to receive the original images collected by the endoscope, and process the original images to obtain corresponding real-time images and physiological structure part information described by the real-time images;

[0007] The second processing module is configured to receive the real-time images and the corresponding physiological structure part information, and process the real-time images based on the physiological structure part information to obtain image result information; the image result information includes diagnosis result information and auxiliary reminder information;

[0008] The display module includes a screen unit and a prompt unit; the screen unit includes a real-time screen area for displaying the original image or the real-time image in real time; the prompt unit includes a risk prompt area and an auxiliary prompt area; the risk prompt area is used to display the diagnostic result information, and the auxiliary prompt area is used to display the auxiliary reminder information.

[0009] In some embodiments, the second processing module includes an image processor;

[0010] The image processor is configured to select a corresponding recognition algorithm according to the physiological structure part information, and perform recognition and analysis on the real-time image according to the recognition algorithm to obtain the corresponding image result information; wherein, each recognition algorithm corresponds to at least one piece of physiological structure part information.

[0011] In some embodiments, performing recognition and analysis on the real-time image to obtain the corresponding image result information includes:

[0012] Comparing the real-time image with a preset image library corresponding to the physiological structure part information:

[0013] If the image features of the real-time image are the same as those of at least one preset case image in the preset image library, the diagnostic result information in the image result information is a classic case, and the classic case is the preset lesion, lesion area and lesion coordinates corresponding to the preset case image;

[0014] If the image features of the real-time image are the same as those of the preset healthy image in the preset image library, the diagnostic result information in the image result information is normal;

[0015] If the real-time image is different from all the preset case images and the preset healthy image in the preset image library, the diagnostic result information in the image result information is abnormal.

[0016] In some embodiments, the auxiliary reminder information in the image result information is subject to risk reminder by different types of colors; wherein, the first type of color is used to indicate the occurrence of the classic case or the risk of missed diagnosis; the second type of color is used to indicate the occurrence of the abnormality or the risk of missed diagnosis; the third type of color is used to indicate normal and no risk of missed diagnosis.

[0017] In some embodiments, the endoscopic auxiliary diagnosis device further includes a storage module;

[0018] When the auxiliary reminder information is presented as the first type of color or the second type of color, the real-time image corresponding to the first type of color or the real-time image corresponding to the second type of color is stored as a warning image in the storage module;

[0019] When responding to the manual review request, play back the corresponding warning image in reverse order.

[0020] In some embodiments, the first processing module is further configured to obtain type information of the endoscope and generate a three-dimensional anatomical structure model according to the type information of the endoscope;

[0021] Correspondingly, the screen unit further includes an anatomical structure guidance area, and the anatomical structure guidance area is used to display the three-dimensional anatomical structure model.

[0022] In some embodiments, obtaining the physiological structure part information includes:

[0023] Performing feature extraction on the original image or the real-time image to obtain image features;

[0024] Determining the part position information described in the original image or the real-time image according to the image features;

[0025] Determining the physiological structure part information according to the part position information.

[0026] In a second aspect, an endoscope-assisted diagnosis and prompt method provided by an embodiment of the present application is applicable to the above-mentioned endoscope-assisted diagnosis device, and the method includes:

[0027] Obtaining an original image collected by an endoscope;

[0028] Processing the original image to obtain a real-time image and physiological structure part information;

[0029] Processing the real-time image according to the physiological structure part information to obtain image result information; the image result information is used for auxiliary diagnosis and prompt.

[0030] In some embodiments, comparing the real-time image with a preset picture library corresponding to the physiological structure part information to obtain the diagnosis result information in the image result information; the diagnosis result information includes classic cases, abnormalities, and normals;

[0031] The auxiliary reminder information in the image result information is used for risk prompt through different types of colors; wherein, the first type of color is used to indicate the occurrence of the classic case or the risk of missed diagnosis; the second type of color is used to indicate the occurrence of the abnormality or the risk of missed diagnosis; the third type of color is used to indicate normal and no risk of missed diagnosis.

[0032] In some embodiments, when the auxiliary reminder information is presented in the first type of color or the second type of color, the endoscopist performs a preset operation to re-identify the image result information of the corresponding real-time image; the preset operation includes stretching or adjusting the angle.

[0033] The embodiments of the present application have the following beneficial effects:

[0034] The endoscopic auxiliary diagnosis system of the present application can enable the endoscopist to more clearly and intuitively see the image features corresponding to the physiological structure by displaying the original image or the real-time image in the real-time picture area of the display module; the diagnostic result information and the auxiliary reminder information are intuitively displayed in the prompt area of the display module to reduce the risk of missed diagnosis caused by reasons such as fatigue of the endoscopist, thereby improving the reliability of the auxiliary diagnosis result. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following will briefly introduce the drawings required in the embodiments. It should be understood that the following drawings only show some embodiments of the present application, and therefore should not be regarded as limiting the scope. For those of ordinary skill in the art, without creative efforts, other related drawings can also be obtained based on these drawings.

[0036] Figure 1 Shows a schematic structural diagram of an endoscopic auxiliary diagnosis device according to an embodiment of the present application;

[0037] Figure 2 Shows a schematic diagram of the display interface of an endoscopic auxiliary diagnosis device according to an embodiment of the present application;

[0038] Figure 3 Shows a schematic diagram of a three-dimensional gastroscope model according to an embodiment of the present application;

[0039] Figure 4 Shows a schematic flowchart of an endoscopic auxiliary diagnosis prompt method according to an embodiment of the present application.

[0040] Main element symbol description: 10 - endoscope; 20 - first processing module; 30 - second processing module; 40 - display module; 41 - picture unit; 42 - prompt unit; 43 - inspection unit; 44 - information unit; 50 - storage module. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0041] The following will clearly and completely describe the technical solutions in the embodiments of the present application with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments.

[0042] The components of the embodiments of the present application that are generally described and illustrated in the accompanying drawings herein may be arranged and designed in a variety of different configurations. Accordingly, the following detailed description of the embodiments of the present application provided in the accompanying drawings is not intended to limit the scope of the claimed present application, but is merely representative of selected embodiments of the present application. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present application without creative efforts fall within the scope of protection of the present application.

[0043] Hereinafter, the terms "comprising", "having" and their cognates that may be used in various embodiments of the present application are only intended to denote a specific feature, number, step, operation, element, component or combination of the foregoing items, and should not be construed as precluding the existence of one or more other features, numbers, steps, operations, elements, components or combinations of the foregoing items or the possibility of adding one or more features, numbers, steps, operations, elements, components or combinations of the foregoing items. In addition, the terms "first", "second", "third", etc. are only used for distinguishing descriptions and cannot be construed as indicating or implying relative importance.

[0044] Unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as commonly understood by those of ordinary skill in the art to which the various embodiments of the present application belong. The terms (such as those defined in a general use dictionary) will be construed to have the same meaning as the contextual meaning in the relevant technical field and will not be construed to have an idealized meaning or an overly formal meaning unless clearly defined in the various embodiments of the present application.

[0045] The following will describe in detail some embodiments of the present application with reference to the accompanying drawings. Without conflict, the following embodiments and the features in the embodiments may be combined with each other.

[0046] Although the auxiliary diagnosis device combining endoscopic technology and artificial intelligence AI provides an efficient and convenient tool for endoscopic physicians, due to reasons such as rapid operation, the pictures in the video stream change too fast, often causing the drift of the auxiliary marking frame, thus affecting the doctor's judgment; and the multi-screen display content causes visual fatigue, which also affects the judgment of endoscopic physicians. Therefore, there are currently phenomena of missed diagnosis and judgment deviation in the auxiliary. To improve the reliability of the auxiliary diagnosis result, the present application proposes an endoscopic auxiliary diagnosis device and its diagnosis prompt method.

[0047] The following will describe the endoscopic auxiliary diagnosis device with reference to some specific embodiments.

[0048] Figure 1Shows a schematic structural diagram of an endoscopic-assisted diagnosis device according to an embodiment of the present application. Exemplarily, the endoscopic-assisted diagnosis device includes an endoscope 10, a first processing module 20, a second processing module 30, a display module 40, and a storage module 50.

[0049] In one embodiment, the endoscope 10 includes a camera, and the camera acquires an original image of the physiological structure to be detected. The original image is transmitted in units of frames; for example, the frame format of the original image may include RAW, RGB, YUV, etc.

[0050] In an embodiment, the first processing module 20 includes a field-programmable gate array (FPGA). Exemplarily, the first processing module 20 receives the original image acquired by the endoscope 10 and performs preprocessing such as denoising, enhancement, scaling, or format conversion on the original image through the field-programmable gate array (FPGA) to achieve corrections such as denoising and distortion correction, thereby obtaining a real-time image with less noise and better quality. For example, the first processing module 20 transmits the original image to the display module 40 in the form of a video stream and displays it through the real-time picture area of the display module 40 for the endoscope physician to view in real time; or, the first processing module 20 transmits the real-time image to the display module 40 in the form of a video stream and displays it through the real-time picture area of the display module 40. As Figure 2 shown, the picture unit 41 of the display module 40 includes a real-time picture area, and the real-time picture area displays the original image or the real-time image, so that a dynamic real-time picture is finally displayed.

[0051] In some embodiments, the first processing module 20 is further configured to obtain the type information of the endoscope 10 and generate a three-dimensional anatomical structure model according to the type information of the endoscope 10; the three-dimensional anatomical structure model is displayed in the anatomical structure guidance area of the display module 40. For example, when the type of the endoscope 10 is a gastroscope, as Figure 2 shown, the picture unit 41 of the display module 40 includes an anatomical structure guidance area, and the anatomical structure guidance area displays as Figure 3 shown three-dimensional gastroscope model, covering the esophagus, stomach, and duodenum; when the type of the endoscope 10 is a colonoscope, a three-dimensional colonoscope model is generated, covering the rectum and colon. The anatomical structure guidance area can dynamically display the anatomical structure of the model through the three-dimensional anatomical structure model, enabling the endoscope physician to view the physiological structure site more clearly and intuitively.

[0052] In some embodiments, the field-programmable gate array (FPGA) of the first processing module 20 is further configured to process the original image or the real-time image to determine the physiological structure site information of the real-time image; that is, the field-programmable gate array (FPGA) extracts image features from the real-time image to obtain image features, and then determines the corresponding site position information according to the image features; finally, the physiological structure site information is determined according to the site position information.

[0053] For example, a standard gastroscopy examination usually covers 7 standard physiological structural parts, including the esophagus, cardia, fundus of the stomach, gastric antrum, pylorus, etc.; among them, the position information of the corresponding part of the esophagus: the tube connecting the pharynx and the stomach, and its image characteristics: the mucosa is smooth and pink. The lower end is the esophagogastric junction (EGJ). The position information of the corresponding part of the cardia: the connection between the esophagus and the stomach, and its image characteristics: the mucosal folds transition from the esophagus to the stomach. The radial distribution of the mucosal folds of the cardia can be seen. The position information of the corresponding part of the fundus of the stomach: the uppermost part of the stomach, located on the left side of the cardia; its image characteristics: the place where gas accumulates in the gastric cavity, and there are fewer mucosal folds. The gastric fundus veins can be seen. The position information of the corresponding part of the gastric antrum: the distal end of the stomach, near the pylorus; its image characteristics: the mucosal folds are less than those of the gastric body, and the gastric wall is thinner. The position information of the corresponding part of the pylorus: the connection between the stomach and the duodenum; its image characteristics: a round or star-shaped opening, with a radial distribution of mucosal folds.

[0054] For example, a colonoscopy examination is usually used to examine the large intestine through a colonoscope (endoscope 10), and the standard examination parts include the rectum, sigmoid colon, descending colon, transverse colon, ascending colon, cecum, etc.; among them, the position information of the corresponding part of the rectum: near the anus, with a clear position; its image characteristics: relatively straight in shape on the image. The intestinal wall is thick, rich in blood vessels, and the mucosal folds are obvious.

[0055] In this embodiment, the first processing module 20 is transmitted to the second processing module 30 through a hardware interface or a software interface. Among them, the hardware interfaces include HDMI / DP, PCIe, MIPI, and / or shared memory / DMA, etc.; HDMI / DP is used to transmit high-resolution image data; PCIe is used for high-speed data transmission and is suitable for scenarios that require low latency; MIPI is commonly used in embedded devices and is suitable for small devices such as mobile devices or endoscopes 10; shared memory / DMA directly shares data through the memory and is suitable for high-performance computing scenarios. The software interfaces include API, SDK, network protocols, and message queues; for example, data exchange is performed through API or SDK; the data of the original image and the image result information are transmitted through network protocols such as TCP / IP, gRPC, etc.; message queues (such as ZeroMQ, RabbitMQ) are used to implement asynchronous communication. For example, the real-time image processed by the first processing module 20 is transmitted to the second processing module 30 as a 1080p@60fps video stream through the hardware interface PCIe.

[0056] In this embodiment, the second processing module 30 is used to receive the real-time image and the corresponding physiological structural part information, and process the real-time image based on the physiological structural part information to obtain image result information; the image result information includes diagnostic result information and auxiliary reminder information; among them, such as Figure 2As shown, the prompt unit 42 of the display module 40 includes a risk prompt area and an auxiliary prompt area; the diagnostic result information in the image result information is displayed in the risk prompt area, and the auxiliary reminder information is displayed in the auxiliary prompt area, that is, at the four corners of the real-time screen area.

[0057] In some embodiments, the second processing module 30 includes an image processor (GPU). The image processor (GPU) selects a corresponding recognition algorithm according to the physiological structure part information, and uses the recognition algorithm to perform recognition and analysis on the real-time image to determine whether there is a lesion area and the lesion coordinates of the lesion area; wherein, the recognition algorithm is used for object detection, semantic segmentation and key point detection. For example, algorithms such as YOLO (You Only Look Once), Faster R-CNN, SSD (Single Shot MultiBox Detector) are used to detect key parts such as the cardia and pylorus. Algorithms such as U-Net and DeepLab are used to segment the gastric wall image to distinguish parts such as the fundus of the stomach, the body of the stomach, and the antrum of the stomach. Algorithms such as HRNet are used to locate key points such as the gastric angle and pylorus.

[0058] In some embodiments, the image processor (GPU) obtains image result information, and the diagnostic result information in the image result information includes classic cases, abnormalities, and normals. Exemplarily, the real-time image is compared with a preset image library corresponding to the physiological structure part information: if the image features of the real-time image are the same as those of at least one preset case image in the preset image library, the diagnostic result information in the image result information is the preset lesion corresponding to the preset case image, that is, a classic case. For example, early gastrointestinal cancer (gastric cancer, esophageal cancer, colorectal cancer), peptic ulcer, inflammatory bowel disease, esophageal varices, etc.; if the image features of the real-time image are the same as those of the preset healthy image in the preset image library, the diagnostic result information in the image result information is normal; if the real-time image is different from all preset case images and preset healthy images in the preset image library, the diagnostic result information in the image result information is abnormal.

[0059] In some embodiments, the auxiliary reminder information in the image result information is used for risk prompting by different types of colors. For example, the first type of color is used to indicate the occurrence of a classic case or the risk of missed diagnosis; the second type of color is used to indicate the occurrence of an abnormality or the risk of missed diagnosis; the third type of color is used to indicate normal and no risk of missed diagnosis. When the diagnosis result information is a classic case, the auxiliary reminder information corresponding to the classic case is presented in the first type of color; when the diagnosis result information is an abnormality, the auxiliary reminder information corresponding to the abnormality is presented in the first type of color or the second type of color according to the pixel size of the image features corresponding to the abnormality; when the diagnosis result information is normal, the auxiliary reminder information is presented in the third type of color. For example, the first type of color is red, the second type of color is yellow, and the third type of color is green; when the diagnosis result information is a classic case, the auxiliary reminder information is presented in red; when the diagnosis result information is an abnormality and the pixel size of the corresponding image features is medium (for example, 6*6), the auxiliary reminder information is presented in yellow; when the diagnosis result information is an abnormality and the pixel size of the corresponding image features is small (for example, 3*3), the auxiliary reminder information is presented in red; among them, when the pixel size is 10*10, the pixel size is large.

[0060] In some embodiments, the auxiliary reminder information is a result obtained by judging based on 30-50 consecutive frames of real-time images. After the first type of color and the second type of color are lit for a preset duration (for example, 5s), they automatically jump to the green light, and at the same time, the real-time images corresponding to the first type of color or the second type of color are automatically saved to the storage module 50 as warning images. When the auxiliary reminder information is the first type of color or the second type of color, manual review is required. When the endoscope doctor touches the freeze button, the warning images will be played back in reverse order.

[0061] In some embodiments, when the auxiliary reminder information is the first type of color or the second type of color, the endoscope doctor touches the freeze button, and by stretching or adjusting the angle, re-identifies the image result information of the corresponding real-time image to reduce the probability of missed diagnosis and improve the accuracy and reliability of auxiliary diagnosis.

[0062] In some embodiments, as Figure 2 shown, the display module 40 of the endoscope-assisted diagnosis device further includes an inspection unit 43 and an information unit 44; the inspection unit 43 includes an area of items to be inspected, and the items to be inspected are matched with the three-dimensional anatomical structure model in the anatomical structure guidance area; the area of items to be inspected includes at least two items to be inspected set according to the endoscope inspection guidelines; the information unit 44 includes case information and an endoscope information area; the case information includes patient information; the endoscope information includes endoscope function prompt information, button setting function information, and endoscope channel auxiliary water supply direction information.

[0063] In some embodiments, the items to be examined are set according to the endoscopy guidelines, and the items to be examined are matched with the anatomical structure guiding area items. For each completed examination item, the corresponding position grid in the three-dimensional anatomical structure model in the anatomical structure guiding area is filled and covered. If there are several items missing, the corresponding missing examination items will flash to give a prompt until all examination items are completed. After the examination is over, an examination report (the items missed in this examination, the time taken for each examination item, the number of stored pictures for each examination item, and the number of suspected lesions found for each examination item) will be generated in the area of the items to be examined. The corresponding start examination time, total examination time, and the operation score evaluated by AI will also be recorded in Figure 2 the upper left corner shown in

[0064] Through the mutual cooperation of the first processing module 20 and the second processing module 30, the endoscopic auxiliary diagnosis device of the present application improves the efficiency of image processing; by the real-time display of the original image or the real-time image in the real-time picture area of the display module 40, the endoscopic doctor can more clearly and intuitively see the image features corresponding to the physiological structure; by the different prompt areas of the display module 40, the diagnostic result information and the auxiliary reminder information are intuitively displayed, so as to reduce the risk of missed diagnosis caused by reasons such as the fatigue of the endoscopic doctor, thereby improving the reliability of the auxiliary diagnosis result.

[0065] Figure 4 Fig. shows a schematic flowchart of an endoscopic auxiliary diagnosis prompting method according to an embodiment of the present application. Exemplarily, the endoscopic auxiliary diagnosis prompting method includes:

[0066] S100, acquiring the original image collected by the endoscope 10.

[0067] In this embodiment, the endoscope 10 includes a camera, and the camera acquires the original image of the physiological structure to be detected. The original image is transmitted in real time in units of frames. It can be understood that the real-time transmission is actually the frame image transmitted in the form of a video stream.

[0068] S200, processing the original image to obtain a real-time image and the corresponding physiological structure part information.

[0069] In some embodiments, in order to obtain an image with less noise and better quality, preprocessing such as denoising, enhancement, scaling, or format conversion is performed on the original image to correct the denoising, distortion, etc. of the original image.

[0070] In some embodiments, feature extraction is performed on the original image or the real-time image to obtain image features, and then the part position information is determined according to the image features; finally, the physiological structure part information is determined according to the part position information. By identifying and processing the original image, a real-time image and the corresponding physiological structure part information are obtained, which is convenient for quickly selecting the corresponding recognition algorithm when subsequently identifying suspected lesions intelligently.

[0071] S300 processes the real-time image according to the physiological structure location information to obtain image result information; the image result information is used for auxiliary diagnosis prompts.

[0072] In this embodiment, the image processor (GPU) selects a corresponding recognition algorithm according to the physiological structure location information, and uses the recognition algorithm to recognize and analyze the real-time image to determine whether there is a lesion area and the lesion coordinates of the lesion area, that is, the image result information.

[0073] In some embodiments, the diagnostic result information recognized by the image processor (GPU) includes classic cases, abnormalities, and normal conditions. Exemplarily, the real-time image is compared with a preset image library corresponding to the physiological structure location information: if the image features of the real-time image are the same as those of at least one preset case image in the preset image library, the diagnostic result information in the image result information is the preset lesion corresponding to the preset case image, that is, a classic case; if the image features of the real-time image are the same as those of the preset healthy image in the preset image library, the diagnostic result information in the image result information is normal; if the real-time image is different from all the preset case images and preset healthy images in the preset image library, the diagnostic result information in the image result information is abnormal.

[0074] In some embodiments, the auxiliary reminder information recognized by the image processor (GPU) is used for risk prompts through different types of colors; when the diagnostic result information is a classic case, the auxiliary reminder information corresponding to the classic case is presented in a first type of color; when the diagnostic result information is abnormal, the auxiliary reminder information corresponding to the abnormality is presented in the first type of color or a second type of color according to the pixel size of the image features corresponding to the abnormality; when the diagnostic result information is presented as normal, the auxiliary reminder information is presented in a third type of color. For example, the first type of color is red, the second type of color is yellow, and the third type of color is green; when the diagnostic result information is a classic case, the auxiliary reminder information is presented as red; when the diagnostic result information is abnormal and the pixel size of the corresponding image features is medium (for example, 6*6), the auxiliary reminder information is presented as yellow; when the diagnostic result information is abnormal and the pixel size of the corresponding image features is small (for example, 3*3), the auxiliary reminder information is presented as red; when the diagnostic result information is normal, the auxiliary reminder information is presented as green.

[0075] In some embodiments, when the auxiliary reminder information is the first type of color, that is, red, or the second type of color, that is, yellow, the endoscope physician touches the freeze button, stretches or adjusts the angle, and re-identifies the image result information of the corresponding real-time image to reduce the probability of missed diagnosis and improve the accuracy and reliability of auxiliary diagnosis.

[0076] It can be understood that the optional items in the above embodiments are equally applicable to this embodiment, so they will not be described repeatedly here.

[0077] In several embodiments provided by this application, it should be understood that the disclosed devices and methods can also be implemented in other ways. The device embodiments described above are merely illustrative. For example, the flowcharts and structure diagrams in the accompanying drawings show the possible architectures, functions, and operations of devices, methods, and computer program products according to multiple embodiments of this application. In this regard, each block in the flowchart or block diagram can represent a module, a program segment, or a part of code, and the module, program segment, or part of code contains one or more executable instructions for implementing the specified logical function. It should also be noted that in an alternative implementation, the functions marked in the blocks can occur in a different order from that marked in the accompanying drawings. For example, two consecutive blocks can actually be executed substantially in parallel, and they can sometimes be executed in the reverse order, depending on the functions involved. It should also be noted that each block in the structure diagram and / or flowchart, as well as the combination of blocks in the structure diagram and / or flowchart, can be implemented by a dedicated hardware-based system that executes the specified functions or actions, or can be implemented by a combination of dedicated hardware and computer instructions.

[0078] In addition, in each embodiment of this application, each functional module or unit can be integrated together to form an independent part, or each module can exist separately, or two or more modules can be integrated to form an independent part.

[0079] If the above functions are implemented in the form of software function modules and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on such an understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a part of this technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to enable a computer device (which can be a smart phone, a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in each embodiment of this application.

[0080] As mentioned above, the above are only the specific implementation manners of this application, but the protection scope of this application is not limited thereto. Any person skilled in the art within the technical scope disclosed by this application can easily think of changes or substitutions, which should all be covered by the protection scope of this application.

Claims

1. An endoscope-assisted diagnostic device, characterized in that: It includes an endoscope, a first processing module, a second processing module and a display module; The first processing module is used to receive the original image collected by the endoscope, and process the original image to obtain the corresponding real-time image and physiological structure part information; The second processing module is used to receive the real-time image and the physiological structure part information, and process the real-time image based on the physiological structure part information to obtain image result information; the image result information includes diagnosis result information and auxiliary reminder information; The display module includes a screen unit and a prompt unit; the screen unit includes a real-time screen area, and the real-time screen area is used to display the original image or the real-time image in real time; the prompt unit includes a risk prompt area and an auxiliary prompt area; the risk prompt area is used to display the diagnosis result information, and the auxiliary prompt area is used to display the auxiliary reminder information.

2. The endoscope-assisted diagnostic device according to claim 1, characterized in that: The second processing module includes an image processor; The image processor is used to select a corresponding recognition algorithm according to the physiological structure part information, and perform recognition analysis on the real-time image according to the recognition algorithm to obtain the corresponding image result information; wherein each of the recognition algorithms corresponds to at least one physiological structure part information.

3. The endoscope-assisted diagnostic device according to claim 2, characterized in that: The identifying and analyzing the real-time image to obtain the corresponding image result information includes: Compare the real-time image with a preset library corresponding to the physiological structure part information: If the real-time image has the same image features as at least one preset case image in the preset gallery, the diagnosis result information in the image result information is a classic case, and the classic case is a preset lesion, lesion area and lesion coordinates corresponding to the preset case image; If the image features of the real-time image are the same as those of the preset healthy image in the preset gallery, the diagnosis result information in the image result information is normal; If the real-time image is different from all the preset case images and the preset healthy images in the preset gallery, the diagnosis result information in the image result information is abnormal.

4. The endoscope-assisted diagnostic device according to claim 3, characterized in that: The auxiliary reminder information in the image result information uses different types of colors to provide risk warnings; among them, the first type of color is used to indicate the occurrence of the classic case or the risk of missed diagnosis; the second type of color is used to indicate the occurrence of the abnormality or the risk of missed diagnosis; the third type of color is used to indicate normal and no risk of missed diagnosis.

5. The endoscope-assisted diagnostic device according to claim 4, characterized in that: Also included is a storage module; When the auxiliary reminder information is presented in the first color or the second color, the real-time image corresponding to the first color or the real-time image corresponding to the second color is stored in the storage module as a warning image; In response to a manual review request, the corresponding warning image is played back in reverse order.

6. The endoscope-assisted diagnostic device according to claim 1, characterized in that: The first processing module is further used to obtain type information of the endoscope, and generate a three-dimensional anatomical structure model according to the type information of the endoscope; Correspondingly, the picture unit further includes an anatomical structure guide area, and the anatomical structure guide area is used to display the three-dimensional anatomical structure model.

7. The endoscope-assisted diagnostic device according to claim 1, characterized in that: Obtaining the physiological structure location information includes: Extracting features from the original image or the real-time image to obtain image features; Determine location information of a part described by the original image or the real-time image according to the image features; The physiological structure part information is determined according to the part position information.

8. An endoscope-assisted diagnosis prompting method, characterized in that: Applicable to the endoscope-assisted diagnostic device according to any one of claims 1 to 7, the method comprising: Acquire the original image collected by the endoscope; Processing the original image to obtain a real-time image and physiological structure location information; The real-time image is processed according to the physiological structure part information to obtain image result information; the image result information is used for auxiliary diagnosis prompts.

9. The endoscope-assisted diagnosis prompting method according to claim 8, characterized in that: Comparing the real-time image with a preset library corresponding to the physiological structure part information to obtain the diagnosis result information in the image result information; the diagnosis result information includes classic cases, abnormalities and normals; The auxiliary reminder information in the image result information uses different types of colors to provide risk warnings; among them, the first type of color is used to indicate the occurrence of the classic case or the risk of missed diagnosis; the second type of color is used to indicate the occurrence of the abnormality or the risk of missed diagnosis; the third type of color is used to indicate normal and no risk of missed diagnosis.

10. The endoscope-assisted diagnosis prompting method according to claim 9, characterized in that: include: When the auxiliary reminder information is presented in the first color or the second color, the endoscopist performs a preset operation to re-identify the corresponding image result information of the real-time image; The preset operation includes stretching or adjusting the angle.

Images