Wound surface treatment method based on endoscope and endoscope system

Abstract

The invention discloses a wound surface treatment method based on an endoscope and an endoscope system. The method comprises the following steps: acquiring a first endoscope image after endoscopic submucosal dissection operation; an unclosed wound surface in the first endoscope image is recognized, wound surface information is obtained, the wound surface information at least comprises the size of the wound surface, the position of the wound surface and the shape of the wound surface, the size of the wound surface is used for indicating the length and the width of the unclosed wound surface, and the position of the wound surface is used for indicating the coordinate and the range of the unclosed wound surface in the first endoscope image; the wound shape is used for indicating the edge shape of the unclosed wound; determining a first closing operation scheme for the unclosed wound according to the wound information; and outputting first prompt information according to the first closing operation scheme. According to the application, automatic identification of the wound area can be realized, and the current wound treatment condition is accurately evaluated according to the identified wound information, so that the wound treatment condition is prompted, and the wound treatment efficiency is improved.

Description

Technical Field

[0001] This application relates to the technical field of medical devices, and particularly to a wound treatment method and an endoscope system based on an endoscope. Background Art

[0002] Endoscopic Submucosal Dissection (ESD) is a new minimally invasive method for treating early gastrointestinal cancer and precancerous lesions at present, which can achieve the same effect as surgical operations. It refers to using special instruments such as high-frequency electric knives under an endoscope to gradually dissect the gastrointestinal lesions including early gastrointestinal tumors from the normal submucosa below, so as to achieve the purpose of completely removing the lesions.

[0003] After the lesion is dissected, all visible blood vessels on the wound surface need to be treated with preventive hemostasis. The possible bleeding sites are treated with a thermal coagulation forceps, argon ion coagulation, etc. For those with deeper local dissection or fissures in the muscular layer, metal clips should be applied. However, doctors with less experience may not be able to accurately perform the wound hemostasis operation during wound treatment, and it is easy to have problems such as incomplete wound treatment and abnormal bleeding after surgery. In addition, multiple metal clips will be placed on a slightly larger wound, and the multiple metal clips will block the doctor's vision, affecting the doctor's observation and judgment of the progress of wound treatment, and thus easily leading to operation errors. Summary of the Invention

[0004] Embodiments of this application provide a wound treatment method and an endoscope system based on an endoscope to improve the efficiency of wound treatment.

[0005] In a first aspect, embodiments of this application provide a wound treatment method based on an endoscope, which is applied to a processor of an endoscope system and includes:

[0006] Obtain a first endoscope image after an endoscopic submucosal dissection operation;

[0007] Identify the unclosed wound in the first endoscope image to obtain wound information, where the wound information at least includes: wound size, wound position, and wound shape. The wound size is used to indicate the length and width of the unclosed wound, the wound position is used to indicate the coordinates and range of the unclosed wound in the first endoscope image, and the wound shape is used to indicate the edge shape of the unclosed wound;

[0008] Determine a first closing operation plan for the unclosed wound according to the wound information;

[0009] Output a first prompt message according to the first closing operation plan.

[0010] In a second aspect, an embodiment of the present application provides an endoscope system, which includes: a processor, a display, and an endoscope;

[0011] The endoscope is configured to execute the endoscope image acquisition instruction of the processor; the display is used to display the endoscope image; the processor is configured to implement the method as described in the first aspect.

[0012] In a third aspect, an embodiment of the present application provides an electronic device, including a memory, a processor, and executable program code stored on the memory and executable on the processor. When the processor executes the executable program code, it performs the steps of the method as described in the first aspect.

[0013] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, on which executable program code is stored. The executable program code includes execution instructions for performing the steps of the method as described in the first aspect.

[0014] In a fifth aspect, an embodiment of the present application provides a computer program product. The computer program product includes a non-transitory computer-readable storage medium storing a computer program. The computer program is operable to cause a computer to perform some or all of the steps described in the first aspect of the embodiments of the present application. The computer program product can be a software installation package.

[0015] It can be seen that in the embodiments of the present application, first, a first endoscope image after endoscopic submucosal dissection operation is obtained; then, the unclosed wound surface in the first endoscope image is identified to obtain wound surface information, where the wound surface information at least includes: the size of the wound surface, the position of the wound surface, and the shape of the wound surface. The size of the wound surface is used to indicate the length and width of the unclosed wound surface, the position of the wound surface is used to indicate the coordinates and range of the unclosed wound surface in the first endoscope image, and the shape of the wound surface is used to indicate the edge shape of the unclosed wound surface; then, according to the wound surface information, a first closing operation plan for the unclosed wound surface is determined; finally, a first prompt message is output according to the first closing operation plan. Through the automatic recognition and analysis of the unclosed wound surface in the postoperative endoscope image, the present application realizes an accurate assessment of the current wound surface treatment situation, and then gives a prompt for the wound surface treatment situation, improving the treatment efficiency of the wound surface. Description of the Drawings

[0016] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for the embodiments will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments of the present application. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0017] Figure 1 This is the system architecture diagram of an endoscope system provided by an embodiment of the present application;

[0018] Figure 2 This is the flowchart of a wound treatment method based on an endoscope provided by an embodiment of the present application;

[0019] Figure 3 This is the schematic diagram of the first wound treatment interface provided by an embodiment of the present application;

[0020] Figure 4 This is the position marking diagram of the closing clip provided by an embodiment of the present application;

[0021] Figure 5 This is the schematic diagram of the first prompt information display area provided by an embodiment of the present application;

[0022] Figure 6 This is the schematic diagram of the second wound treatment interface provided by an embodiment of the present application;

[0023] Figure 7 This is the schematic diagram of the second prompt information display area provided by an embodiment of the present application;

[0024] Figure 8 This is the schematic diagram of the third prompt information display area provided by an embodiment of the present application;

[0025] Figure 9 This is the functional unit composition block diagram of a wound treatment device based on an endoscope provided by an embodiment of the present application;

[0026] Figure 10 This is the functional unit composition block diagram of another wound treatment device based on an endoscope provided by an embodiment of the present application;

[0027] Figure 11 This is the structural schematic diagram of an electronic device proposed by an embodiment of the present application. Detailed implementation manners

[0028] In order to enable those skilled in the art to better understand the solution of the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present application.

[0029] In the description and claims of this application and the above-mentioned drawings, terms such as "first" and "second" are used to distinguish different objects, rather than to describe a specific order. In addition, the terms "comprising" and "having" and any variations thereof are intended to cover non-exclusive inclusion. For example, a process, method, system, product or device that includes a series of steps or units is not limited to the listed steps or units, but optionally further includes steps or units not listed, or optionally further includes other steps or units inherent to these processes, methods, products or devices.

[0030] Reference to "embodiment" herein means that a particular feature, structure, or characteristic described in connection with the embodiment can be included in at least one embodiment of this application. The phrase appears in various places in the specification and does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment mutually exclusive with other embodiments. Those skilled in the art will explicitly and implicitly understand that the embodiments described herein can be combined with other embodiments.

[0031] When inexperienced doctors perform wound treatment, they are unable to accurately judge the placement position, direction, and number of metal clips, resulting in incomplete wound treatment and problems such as abnormal postoperative bleeding. Moreover, multiple metal clips are placed on slightly larger wounds. How to use fewer metal clips to achieve complete wound treatment. At the same time, multiple metal clips will block the doctor's view, and there will be errors when observing and judging whether the wound has been completely treated.

[0032] In view of the above problems, the embodiments of this application provide an endoscope-based wound treatment method and an endoscope system. The embodiments of this application will be introduced in detail below with reference to the drawings.

[0033] Please refer to Figure 1 , Figure 1 which is a system architecture diagram of an endoscope system provided by an embodiment of this application. As Figure 1 shown, the endoscope system 100 includes a processor 101, a display 102, and an endoscope 103. Among them, the processor 101, the display 102, and the endoscope 103 are interconnected. The endoscope 103 is used to collect image information of the internal tissues of an observation object such as a human body or an animal through an optical lens or an electronic imaging element at the front end, convert it into an electrical signal or a digital signal, and then transmit it to the processor 101 for processing, and finally display it on the display 102.

[0034] Among them, the processor 101 is the control core of the system. According to the doctor's operations or preset programs, it issues instructions to the display 102 and the endoscope 103. For example, it controls the endoscope 103 to collect and send images, controls the display 102 to display images, etc. The processor 101 can also adjust the display parameters of the display 102 such as brightness and contrast, as well as adjust the illumination brightness and focal length adjustment of the endoscope 103. Specifically, the processor 101 receives image information in the form of electrical signals or digital signals from the endoscope 103, and identifies and analyzes the wound surface based on this image information, and then generates wound surface guidance prompt information; at the same time, it sends the generated guidance prompt information and the identified wound surface image to the display 102, so that the display 102 receives the above information and displays it.

[0035] Specifically, the display 102 includes an image presentation terminal and an operation feedback interface. The image presentation terminal is used to display the image data transmitted by the processor 101 in an intuitive image form, providing the doctor with a clear internal tissue image of the observation object for easy observation and diagnosis. The operation feedback interface is used to cooperate with the processor 101 to feedback the generated guidance prompt information and the like to the doctor.

[0036] Based on this, the present application provides a wound surface treatment method and an endoscope system based on an endoscope. The present application will be described in detail below with reference to the accompanying drawings.

[0037] Please refer to Figure 2 , Figure 2 which is a schematic flowchart of a wound surface treatment method based on an endoscope provided by an embodiment of the present application. As Figure 2 shown, the method includes the following steps:

[0038] S210, obtain a first endoscope image after endoscopic submucosal dissection operation.

[0039] Among them, after the ESD operation is completed, the processor of the endoscope system, or the server of the terminal device connected to the endoscope, sends an image acquisition instruction to the endoscope. Then the endoscope executes the image acquisition instruction to obtain the first endoscope image and transmits it to the instruction sending end. Further, the dynamic process of the wound surface within a preset time period after the operation can also be obtained.

[0040] S220, identify the unclosed wound surface in the first endoscope image to obtain wound surface information.

[0041] Among them, the wound surface information at least includes: the size of the wound surface, the position of the wound surface, and the shape of the wound surface. The size of the wound surface is used to indicate the length and width of the unclosed wound surface. The position of the wound surface is used to indicate the coordinates and range of the unclosed wound surface in the first endoscope image. The shape of the wound surface is used to indicate the edge shape of the unclosed wound surface.

[0042] Among them, the endoscopic image is automatically recognized by a processor or a server, and the mucosal image area, the closed wound area, and the unclosed wound area are recognized. Among them, the mucosal image area, the closed wound area, and the unclosed wound area can be identified and distinguished in the first endoscopic image by different colors. Among them, the identification of each area includes information such as the size, position, and shape of the area. Further, the edge of the unclosed wound area is automatically outlined to display the unclosed area.

[0043] Specifically, the traditional machine learning method can be used for automatic recognition. Specifically, the Otsu method and the level set method are used to segment the image to obtain the wound range. First, the Otsu method is used to perform binary classification on the image according to color to obtain the approximate position of the wound, and this is used as the initial contour. Then, the level set method is used to obtain the fine wound contour. This method has a small amount of calculation, good effects on specific types of mirrors and wound types, and the initial contour can be determined in an interactive manner.

[0044] Specifically, the automatic recognition can also be performed in a deep learning-based manner. Specifically, the UNet network model is used. By collecting wound pictures and the dataset of the wound area and type annotated by experts, the network model is trained, and then the trained model is used to segment the wound area in the real-time endoscopic image to obtain the wound range and the wound type.

[0045] Among them, after the first endoscopic image is obtained, the processor will control the display screen in the display to enter the wound treatment interface. The wound treatment interface includes an endoscopic image display area and a prompt information display area, and the first endoscopic image is displayed in the endoscopic image display area. After the automatic recognition of the first endoscopic image, the endoscopic image marked with the unclosed wound area after recognition replaces the first endoscopic image and is displayed in the endoscopic image display area. Further, the endoscopic image marked with the unclosed wound area after recognition and the first endoscopic image can be jointly displayed in the endoscopic image display area.

[0046] Specifically, please refer to Figure 3 , Figure 3 which is a schematic diagram of the first wound treatment interface provided by the embodiment of the present application. As shown in Figure 3As shown, there can be various endoscopic images in the endoscopic image display area of the wound treatment interface. Exemplarily, there are four endoscopic images, namely A, B, C, and D. Endoscopic image A shows a relatively complete internal tissue surface; there are some black and grayish-white substances at the wound in endoscopic image B, which may be necrotic tissue or other attachments of the wound, and the wound condition is relatively complex; there are obvious breaks and irregular tissue morphologies at the wound in endoscopic image C, and the color also shows some dullness, indicating the degree of wound injury; there are several metal clips at the wound in endoscopic image D, which are used to close the wound, and the red closed line marks the unclosed area.

[0047] Specifically, the user can click on the above images to perform operations such as zooming and rotating.

[0048] Furthermore, there is also a dynamic broadcast window in the endoscopic image display area for real-time display of endoscopic images.

[0049] As Figure 3 shown, the wound treatment interface also includes a photo-taking control and a recording control. Clicking on the photo-taking control can take a screenshot of the current wound treatment interface. Clicking on the recording control for the first time can record a video of the current wound treatment interface, and clicking on the recording control for the second time stops the recording.

[0050] S230. Determine a first closing operation plan for the unclosed wound according to the wound information.

[0051] Specifically, the first closing operation plan includes the quantity information of the closing clips required for the unclosed wound and the position information of each closing clip in the unclosed wound, and the closing clips are used to close the wound.

[0052] Specifically, the determination of the first closing operation plan for the unclosed wound according to the wound information includes: determining the quantity information according to the wound size and the wound morphology; determining the position information according to the wound size, the wound position, the wound morphology, and the quantity information; and obtaining the first closing operation plan according to the quantity information and the position information.

[0053] Among them, the closing clips are metal clips. The quantity of the closing clips can be initially determined according to the wound size. The current wound can be determined as a small wound, a medium-sized wound, or a large wound according to the length and width of the wound, or according to the area of the wound. Then, referring to the quantity of the closing clips under each wound grade, the initially determined quantity of the closing clips is obtained. Exemplarily, for a relatively small wound with a length and width both within 1 cm, the initially determined quantity of the closing clips is 1.

[0054] Further, it is possible to determine whether additional closing clips are needed according to the wound surface morphology. Specifically, if the wound surface is irregular in shape, such as strip-shaped or polygonal, the number of closing clips needs to be appropriately increased according to the protruding parts and turning points to ensure effective closure of all parts.

[0055] Among them, after determining the number of closing clips, the positions of the closing clips are determined according to the principles of uniform distribution of the closing clips, complete coverage of the wound surface, and avoidance of important structures. First, referring to the wound surface morphology, the placement rules of the closing clips are determined according to the wound surface morphology. For example, for a linear wound surface, starting from one end of the wound surface, a closing clip is placed at a certain distance along the edge of the wound surface; for an irregular wound surface, the key parts of the wound surface are first determined, such as the corners, protruding parts, or wider parts of the wound surface, and closing clips are placed at these positions first to fix the basic shape of the wound surface. According to the above placement rules, the length and width of the wound surface, and the position information of the unclosed wound surface, the position information of each closing clip is determined, and the position information of the closing clip is the specific coordinate information that the closing clip needs to hold in the endoscopic image.

[0056] Among them, the position where the metal clip closes is on the normal tissue mucosa close to the wound surface, and the contact position between the metal clip and the normal tissue mucosa around the wound surface will be marked on the wound surface image. Exemplarily, please refer to Figure 4 , Figure 4 which is a closing clip position marking diagram provided by an embodiment of the present application. As shown in Figure 4 , the unclosed wound surface is oval, and 3 closing clips are needed. Each closing clip has two landing points, which are represented by the same color in Figure 4 . Specifically, two purple marks are used to represent the closing position of the first closing clip, two gray marks are used to represent the closing position of the second closing clip, and two green marks are used to represent the closing position of the third closing clip.

[0057] It can be seen that in the embodiment of the present application, an accurate closing operation can be provided for the user, thereby effectively closing the wound surface and achieving stable closure.

[0058] S240, output a first prompt message according to the first closing operation plan.

[0059] Among them, a plurality of closing steps are determined according to the number and positions of the closing clips. Among them, the plurality of closing steps are arranged in sequence, and auxiliary information is carried in each closing step. The auxiliary information is used to explain the reason for a single closing step in the current sequence. Further, only the order of important closing steps may be determined in the plurality of closing steps, and other closing steps are for the user to determine independently.

[0060] Specifically, the endoscope system further includes a display screen for displaying a wound treatment interface. The wound treatment interface includes an endoscope image display area and a prompt information display area. After outputting a first prompt information according to the first closing operation plan, the method further includes: displaying a first endoscope image of the unclosed wound that has been marked in the endoscope image display area; and displaying the first prompt information in the prompt information display area.

[0061] Among them, after outputting the prompt information of the current wound, a first endoscope image of the unclosed wound that has been marked is displayed in the endoscope image display area in the wound treatment interface, similar to Figure 3 Image D in, except that the currently displayed endoscope image has not yet performed the closing operation and there is no metal clip. Further, please refer to Figure 5 , Figure 5 is a schematic diagram of the first prompt information display area provided by an embodiment of the present application. As shown in Figure 5 , the prompt information display area includes a first closing step area, a second closing step area, a third closing step area..., and each closing step area may include at least one closing step. The priority of the closing steps in the first closing step area is higher than that of the closing steps in the second closing step area, and the priority of the closing steps in the second closing step area is higher than that of the closing steps in the third closing step area. Exemplarily, multiple closing steps in the first prompt information can be displayed in the prompt information display area in order of priority.

[0062] It can be seen that in the embodiment of the present application, through the automatic recognition and analysis of the unclosed wound in the postoperative endoscope image, the accurate assessment of the current wound treatment situation is realized, and then a prompt is given to the wound treatment situation, improving the treatment efficiency of the wound.

[0063] In a possible embodiment, after outputting the first prompt information according to the first closing operation plan, the method further includes: obtaining a second endoscope image after the user performs a second closing step; determining a second closing operation plan according to the second endoscope image; and outputting a second prompt information according to the second closing operation plan.

[0064] Among them, every time the doctor finishes performing a closing operation, an endoscope image is obtained, and then the unclosed wound in the newly obtained endoscope image is recognized to obtain information such as the size, location, and shape of the unclosed wound, and a new closing operation plan is determined according to the above information, and then a new prompt information is output.

[0065] It can be seen that in the embodiment of the present application, by real-time recognizing the unclosed wound, real-time prompt for wound treatment is realized, thereby improving the treatment efficiency of the wound.

[0066] Specifically, the first prompt information includes a plurality of first closing steps. After obtaining the second endoscope image after the user performs the second closing step and before determining the second closing operation plan according to the second endoscope image, the method further includes: determining whether the user refers to the first prompt information according to the plurality of first closing steps and the second closing step; if the user does not refer to the first prompt information, determining the number of times the user fails to refer; and detecting that the number of times of non-reference is less than or equal to a preset number of times.

[0067] Among them, after the doctor performs a closing operation of closing the wound surface with a metal clip, it is determined whether the closing operation is related to the plurality of closing operations in the first prompt information. Specifically, it can be determined whether there is a corresponding closing step in the plurality of first closing steps according to the position of the metal clip in the second closing operation. If so, it is determined that the doctor refers to the first prompt information; if not, it is determined that the doctor does not refer to the first prompt information.

[0068] Among them, each time it is determined that the doctor does not refer to the prompt information, the number of times the doctor fails to refer is updated, that is, the number of times of non-reference is increased by one. When the current number of times of non-reference is less than or equal to the preset number of times, the closing operation plan is continuously generated and new prompt information is output.

[0069] Specifically, when it is detected that the number of times of non-reference is greater than the preset number of times, a confirmation message is output, and the confirmation message is used for the user to confirm whether to turn off the prompt function, and the prompt function is used to display a single prompt message.

[0070] Specifically, the endoscope system further includes a display screen for displaying a wound surface treatment interface. After outputting the confirmation message when it is detected that the number of times of non-reference is greater than the preset number of times, the method further includes: popping up a confirmation window on the wound surface treatment interface, where the confirmation window includes the confirmation message, an OK control and a Cancel control. The OK control is used to determine to turn off the prompt function, and the Cancel control is used to determine not to turn off the prompt function.

[0071] Among them, when the current number of times of non-reference is greater than the preset number of times, an operation to turn off the function is triggered, and a confirmation interface window pops up in the wound surface treatment interface to determine whether to turn off the function, that is, whether to continue to output prompt information. Please refer to Figure 6 , Figure 6 is a schematic diagram of the second wound surface treatment interface provided by the embodiment of the present application. As Figure 6 shown, a confirmation interface window pops up in the wound surface treatment interface. The confirmation interface window includes a confirmation message for whether to turn off the prompt function, a Yes control and a No control. The Yes control corresponds to the OK control, and the No control corresponds to the Cancel control.

[0072] Among them, it is determined whether to close the prompt function by the user clicking on the control. If the user does not click on the confirmation control or the negative control within the preset time, the confirmation interface window is automatically closed, and the prompt information is continuously output and displayed in real time.

[0073] In a possible embodiment, after obtaining the second endoscope image after the user performs the second closing step, the method further includes: determining whether the second closing step needs to be corrected according to the second endoscope image; if it is necessary to correct the second closing step, guiding information is output, and the guiding information includes the deviation position and / or deviation angle of the closing clip used in the second closing step.

[0074] Among them, after the doctor performs a closing step, it is determined whether the closing step is appropriate according to the endoscope image after completion. Specifically, it is determined whether the current placement direction and position of the current metal clip are appropriate, and further, it is determined whether the closing operation needs to be corrected.

[0075] Among them, if it is necessary to correct the second closing step, then it is further determined whether the doctor performed the closing operation with reference to the prompt information. If the doctor performed the closing operation with reference to the prompt information, it is determined whether the metal clip is perpendicular to the wound surface, and whether the metal clip corresponds to the marked position. If the metal clip is perpendicular to the wound surface and the metal clip exactly falls into the marked position, it is determined that the second closing step does not need to be corrected. If the metal clip is not perpendicular to the wound surface, or the metal clip does not correspond to the marked position, it is determined that the second closing step needs to be corrected.

[0076] Among them, if the doctor did not perform the closing operation with reference to the prompt information, it is determined whether there is a gap or tissue exposure. If there is, it is determined that correction is needed. If not, it is determined that correction is not needed.

[0077] Further, guiding information is output according to the position of the metal clip and the clamping direction of the metal clip displayed on the endoscope image, and the guiding information includes the correct clamping position and correct clamping angle of the metal clip.

[0078] Specifically, the endoscope system further includes a display screen for displaying a wound treatment interface. The wound treatment interface includes a prompt information display area. After outputting the guiding information if it is necessary to correct the second closing step, the method further includes: displaying the guiding information in the prompt information display area.

[0079] Among them, please refer to Figure 7 , Figure 7 is a schematic diagram of the second prompt information display area provided by the embodiment of the present application, as Figure 7As shown, the prompt information display area further includes a guiding information display area for displaying guiding information, and the prompt information is displayed below the guiding information display area.

[0080] In a possible embodiment, after obtaining the second endoscope image after the user performs the second closing step and before determining the second closing operation plan according to the second endoscope image, the method further includes: outputting a closing result of the second closing step according to the second endoscope image, where the closing result includes a closing operation type, the number of closing clips used, the closing state of the unclosed wound surface, and / or marking information of the uncompletely closed area in the second endoscope image; detecting that the closing state of the unclosed wound surface is that the unclosed wound surface is not completely closed.

[0081] Among them, for each closing step executed, a closing result is output, and it is automatically recognized whether there is still an unclosed area. If so, the edge of the unclosed wound area is automatically outlined and the unclosed area is displayed. If the closing state of the unclosed wound surface is completely closed, the closing result includes the closing operation type, the number of closing clips used, and the completely closed state, and at the same time, no more prompt information is output; if the closing state of the unclosed wound surface is not completely closed, the closing result includes the closing operation type, the number of closing clips used, the not completely closed state, and the marking information of the uncompletely closed area in the obtained real-time endoscope image, and at the same time, new prompt information is continuously output.

[0082] Specifically, the endoscope system further includes a display screen for displaying a wound surface treatment interface. The wound surface treatment interface includes an endoscope image display area and a prompt information display area. After outputting the second prompt information according to the second closing operation plan, the method further includes: displaying the second endoscope image with the uncompletely closed area marked in the endoscope image display area; and displaying the second prompt information and the closing result in the prompt information display area.

[0083] Among them, if the closing state of the unclosed wound surface is completely closed, the second endoscope image is displayed in the endoscope image display area, and the closing result including the closing operation type, the number of closing clips used, and the completely closed state is displayed in the prompt information display area. If the closing state of the unclosed wound surface is not completely closed, the second endoscope image with the uncompletely closed area marked is displayed in the endoscope image display area, for example Figure 3 endoscope image D in; and the closing result including the closing operation type, the number of closing clips used, the not completely closed state, and the marking information of the uncompletely closed area in the obtained real-time endoscope image, and the second prompt information are displayed in the prompt information display area.

[0084] See also Figure 8 , Figure 8 is a schematic diagram of a third prompt information display area provided in an embodiment of the present application, such as Figure 8 As shown, the prompt information display area also includes a closing result display area for displaying the closing result, and a second prompt information is displayed at the prompt information. Figure 3 The closing result obtained from the endoscopic image D in FIG1 is as follows: the closing operation type is metal clip closing, the number of closing clips used is 4, the closing state is incompletely closed, and further processing is required. The identification information of the incompletely closed area is the red mark in the endoscopic image D.

[0085] In a possible embodiment, the wound closure method also includes nylon rope purse-string suture, which is suitable for ESD wounds with larger areas. The wound can be effectively closed by nylon rope purse-string suture. Specifically, the nylon rope is first sent into the stomach through the endoscopic biopsy channel; then the nylon rope is placed around the wound, and the nylon rope ring and the mucosal tissue at the edge of the wound are fixed with a titanium clip; finally, the nylon rope is retracted under direct endoscopy to complete the purse-string suture.

[0086] Among them, the wound closure method also includes stapler-assisted operation, such as Over-The-Scope Clip (OTSC), which is suitable for larger wounds. First, select a suitable stapler, then send the stapler to the wound position and accurately align it; finally, stimulate the stapler to make it fit tightly to the wound to complete the closure. Among them, the wound closure method also includes thermosensitive gel, which can be directly applied through the catheter.

[0087] If the wound cannot be closed by the metal clip closure method, other closure methods can be selected to perform the closure operation, display the closure progress in real time, and generate corresponding prompt information. For example, in the nylon purse string closure operation mode, prompts or guidance information on the suture position are output, and in the stapler-assisted closure operation mode, prompts or guidance on determining the angle between the stapler and the wound surface are output.

[0088] For the above embodiments, please refer to Figure 9 , Figure 9 is a functional unit block diagram of an endoscope-based wound treatment device provided in an embodiment of the present application, such as Figure 9As shown in the figure, the endoscope-based wound treatment device 90 includes: an acquisition unit 91 for acquiring a first endoscope image after endoscopic submucosal dissection; an identification unit 92 for identifying the unclosed wound in the first endoscope image to obtain wound information, where the wound information at least includes: wound size, wound location, and wound shape. The wound size is used to indicate the length and width of the unclosed wound, the wound location is used to indicate the coordinates and range of the unclosed wound in the first endoscope image, and the wound shape is used to indicate the edge shape of the unclosed wound; a determination unit 93 for determining a first closing operation plan for the unclosed wound according to the wound information; and an output unit 94 for outputting a first prompt message according to the first closing operation plan.

[0089] In a possible embodiment, the first closing operation plan includes the quantity information of the closing clips required for the unclosed wound and the position information of each closing clip in the unclosed wound, and the closing clips are used to close the wound.

[0090] In a possible embodiment, in terms of determining the first closing operation plan for the unclosed wound according to the wound information, the determination unit 93 is specifically configured to: determine the quantity information according to the wound size and the wound shape; determine the position information according to the wound size, the wound location, the wound shape, and the quantity information; and obtain the first closing operation plan according to the quantity information and the position information.

[0091] In a possible embodiment, after outputting the first prompt message according to the first closing operation plan, the endoscope-based wound treatment device 90 is specifically further configured to: acquire a second endoscope image after the user performs a second closing step; determine a second closing operation plan according to the second endoscope image; and output a second prompt message according to the second closing operation plan.

[0092] In a possible embodiment, the first prompt message includes a plurality of first closing steps. After acquiring the second endoscope image after the user performs the second closing step and before determining the second closing operation plan according to the second endoscope image, the endoscope-based wound treatment device 90 is specifically further configured to: determine whether the user refers to the first prompt message according to the plurality of first closing steps and the second closing step; if the user does not refer to the first prompt message, determine the number of times the user fails to refer; and detect that the number of times of non-reference is less than or equal to a preset number of times.

[0093] In a possible embodiment, the endoscope-based wound treatment device 90 is further specifically configured to: when it detects that the number of non-reference times is greater than the preset number, output a confirmation message for the user to confirm whether to turn off the prompt function, where the prompt function is used to display a single prompt message.

[0094] In a possible embodiment, after obtaining the second endoscope image after the user performs the second closing step, the endoscope-based wound treatment device 90 is further specifically configured to: determine whether the second closing step needs to be corrected according to the second endoscope image; if the second closing step needs to be corrected, output guiding information, where the guiding information includes the deviation position and / or deviation angle of the closing clip used in the second closing step.

[0095] In a possible embodiment, after obtaining the second endoscope image after the user performs the second closing step and before determining the second closing operation plan according to the second endoscope image, the endoscope-based wound treatment device 90 is further specifically configured to: output the closing result of the second closing step according to the second endoscope image, where the closing result includes the closing operation type, the number of used closing clips, the closing state of the unclosed wound, and / or the marking information of the incompletely closed area in the second endoscope image; and detect that the closing state of the unclosed wound is that the unclosed wound is not completely closed.

[0096] In a possible embodiment, the endoscope system further includes a display screen for displaying a wound treatment interface, where the wound treatment interface includes an endoscope image display area and a prompt message display area. After outputting the first prompt message according to the first closing operation plan, the endoscope-based wound treatment device 90 is further specifically configured to: display the first endoscope image with the unclosed wound marked in the endoscope image display area; and display the first prompt message in the prompt message display area.

[0097] In a possible embodiment, the endoscope system further includes a display screen for displaying a wound treatment interface. After detecting that the number of non-reference times is greater than the preset number and outputting the confirmation message, the endoscope-based wound treatment device 90 is further specifically configured to: pop up a confirmation window on the wound treatment interface, where the confirmation window includes the confirmation message, an OK control, and a Cancel control, and the OK control is used to confirm turning off the prompt function, and the Cancel control is used to confirm not turning off the prompt function.

[0098] In a possible embodiment, the endoscope system further includes a display screen for displaying a wound treatment interface. The wound treatment interface includes a prompt information display area. After outputting the guiding information if the second closing step needs to be corrected, the wound treatment device 90 based on the endoscope is specifically further configured to: display the guiding information in the prompt information display area.

[0099] In a possible embodiment, the endoscope system further includes a display screen for displaying a wound treatment interface. The wound treatment interface includes an endoscope image display area and a prompt information display area. After outputting a second prompt information according to the second closing operation plan, the wound treatment device 90 based on the endoscope is specifically further configured to: display a second endoscope image in which the uncompletely closed area has been marked in the endoscope image display area; and display the second prompt information and the closing result in the prompt information display area.

[0100] It can be understood that since the method embodiment and the device embodiment are different presentation forms of the same technical concept, the content of the method embodiment part in this application should be synchronously adapted to the device embodiment part, and will not be elaborated here.

[0101] In the case of adopting an integrated unit, please refer to Figure 10 , Figure 10 is a functional unit composition block diagram of another wound treatment device based on an endoscope provided by an embodiment of the present application. As Figure 10 shown, the wound treatment device 90 based on the endoscope includes: a processing module 902 and a communication module 901. The processing module 902 is used to control and manage the actions of the wound treatment device 90 based on the endoscope. For example, it executes the steps of the acquisition unit 91, the recognition unit 92, the determination unit 93, and the output unit 94, and / or is used to execute other processes of the technologies described herein. The communication module 901 is used for the interaction between the wound treatment device 90 based on the endoscope and other devices. As Figure 10 shown, the wound treatment device 90 based on the endoscope may further include a storage module 903, and the storage module 903 is used to store the program code and data of the wound treatment device 90 based on the endoscope.

[0102] Among them, the processing module 902 can be a processor or a controller. For example, it can be a Central Processing Unit (CPU), a general-purpose processor, a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field Programmable Gate Array (FPGA), or other programmable logic devices, transistor logic devices, hardware components, or any combination thereof. It can implement or execute various exemplary logic blocks, modules, and circuits described in connection with the disclosure of this application. The processor can also be a combination that implements computing functions, such as a combination of one or more microprocessors, a combination of a DSP and a microprocessor, and so on. The communication module 901 can be a transceiver, an RF circuit, or a communication interface, etc. The storage module 903 can be a memory.

[0103] Among them, all relevant contents of each scenario involved in the above method embodiment can be cited in the function description of the corresponding functional module, and will not be elaborated here. The above endoscopic-based wound treatment device 90 can execute the above Figure 2 shown endoscopic-based wound treatment method.

[0104] Please refer to Figure 11 , Figure 11 which is a schematic structural diagram of an electronic device proposed in an embodiment of this application. As Figure 11 shown, the electronic device 1100 includes a processor 1110, a memory 1120, a communication interface 1130, and one or more programs 1121. The above one or more programs 1121 are stored in the above memory and are configured to be executed by the above processor. When the program is executed, it includes some or all of the steps of any of the process configuration methods described in the above method embodiment. The processor, the memory, and the communication interface are interconnected and complete the communication work between them.

[0105] Among them, the memory can be a volatile memory such as a Dynamic Random Access Memory (DRAM), or a non-volatile memory such as a mechanical hard disk. The above memory is used to store a set of executable program codes. The above processor is used to call the executable program codes stored in the memory and can execute some or all of the steps of any endoscopic-based wound treatment method described in the above endoscopic-based wound treatment method embodiment.

[0106] It can be seen that for the electronic device 1100 described in the embodiments of the present application, first, a first endoscopic image after an endoscopic submucosal dissection operation is acquired; then, an unclosed wound surface in the first endoscopic image is recognized to obtain wound surface information, where the wound surface information at least includes: the size of the wound surface, the position of the wound surface, and the shape of the wound surface. The size of the wound surface is used to indicate the length and width of the unclosed wound surface, the position of the wound surface is used to indicate the coordinates and range of the unclosed wound surface in the first endoscopic image, and the shape of the wound surface is used to indicate the edge shape of the unclosed wound surface; then, a first closing operation plan for the unclosed wound surface is determined according to the wound surface information; finally, a first prompt message is output according to the first closing operation plan. Through the automatic recognition and analysis of the unclosed wound surface in the postoperative endoscopic image, the present application realizes an accurate assessment of the current wound surface treatment situation, and then gives a prompt for the wound surface treatment situation, improving the treatment efficiency of the wound surface.

[0107] The embodiments of the present application further provide a computer storage medium, where the computer storage medium stores a computer program for electronic data exchange, and the computer program enables a computer to execute some or all of the steps of any of the methods described in the above method embodiments. The above computer includes an electronic device.

[0108] The embodiments of the present application further provide a computer program product. The above computer program product includes a non-transitory computer-readable storage medium storing a computer program, and the above computer program is operable to enable a computer to execute some or all of the steps of any of the methods described in the above method embodiments. The computer program product can be a software installation package, and the above computer includes an electronic device.

[0109] It should be noted that for the foregoing method embodiments, for the sake of simple description, they are all expressed as a series of action combinations. However, those skilled in the art should know that the present application is not limited by the described action sequence, because according to the present application, some steps can be performed in other sequences or simultaneously. Secondly, those skilled in the art should also know that the embodiments described in the specification are all optional embodiments, and the actions and modules involved are not necessarily essential to the present application.

[0110] In the above embodiments, the descriptions of the various embodiments have their own emphases. For the parts not detailed in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0111] In several embodiments provided by the present application, it should be understood that the disclosed device can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For example, the division of the units is only a logical function division. In actual implementation, there may be other division methods. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the displayed or discussed coupling or direct coupling or communication connection between each other can be through some interfaces. The indirect coupling or communication connection of the device or unit can be in electrical or other forms.

[0112] The units described as separate components may or may not be physically separated. The components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed to multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0113] In addition, in each embodiment of the present application, the functional units can be integrated in a processing unit, or each unit can exist physically alone, or two or more units can be integrated in one unit. The above-mentioned integrated units can be implemented in the form of hardware or in the form of software program modules.

[0114] If the above-mentioned integrated unit is implemented in the form of a software program module and sold or used as an independent product, it can be stored in a computer-readable memory. Based on such an understanding, the technical solution of the present application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a memory and includes several instructions for causing a computer device (which can be a personal computer, a server or a network device, etc.) to execute all or part of the steps of the methods described in each embodiment of the present application. And the aforementioned memory includes: USB flash drives, read-only memories (ROMs), random access memories (RAMs), mobile hard disks, magnetic disks or optical discs and other media that can store program codes.

[0115] Those of ordinary skill in the art can understand that all or part of the steps in the various methods of the above embodiments can be completed by instructing relevant hardware through a program. The program can be stored in a computer-readable memory. The memory can include: flash drives, read-only memories, random access memories, magnetic disks or optical discs, etc.

[0116] The above has introduced the embodiments of the present application in detail. Specific examples are used herein to illustrate the principle and embodiments of the present application. The description of the above embodiments is only for helping to understand the method and its core idea of the present application; at the same time, for those of ordinary skill in the art, according to the idea of the present application, there will be changes in the specific embodiments and application scopes. In summary, the content of this specification should not be construed as a limitation to the present application.

Claims

1. A wound treatment method based on endoscope, characterized in that: A processor applied to an endoscope system, the method comprising: Obtaining the first endoscopic image after the endoscopic submucosal dissection procedure; Identify an unclosed wound in the first endoscopic image and obtain wound information, wherein the wound information includes at least: wound size, wound position and wound morphology, wherein the wound size is used to indicate the length and width of the unclosed wound, the wound position is used to indicate the coordinates and range of the unclosed wound in the first endoscopic image, and the wound morphology is used to indicate the edge shape of the unclosed wound; Determining a first closing operation plan for the unclosed wound surface according to the wound surface information; Outputting first prompt information according to the first closing operation scheme.

2. The method according to claim 1, characterized in that The first closing operation scheme includes information on the number of closing clips required for the unclosed wound surface and information on the position of each closing clip in the unclosed wound surface. The closing clips are used to close the wound surface.

3. The method according to claim 2, characterized in that The determining a first closing operation scheme for the unclosed wound surface according to the wound surface information comprises: Determining the quantity information according to the wound size and the wound morphology; Determine the position information according to the wound size, the wound position, the wound morphology and the quantity information; The first closing operation plan is obtained according to the quantity information and the position information.

4. The method according to claim 1, characterized in that: After outputting the first prompt information according to the first closing operation scheme, the method further includes: acquiring a second endoscopic image after the user performs a second closing step; determining a second closing operation plan according to the second endoscopic image; Output second prompt information according to the second closing operation scheme.

5. The method according to claim 4, characterized in that The first prompt information includes a plurality of first closing steps, and after acquiring a second endoscopic image after the user performs a second closing step, and before determining a second closing operation scheme according to the second endoscopic image, the method further includes: determining, according to the plurality of first closing steps and the second closing step, whether the user refers to the first prompt information; If the user does not refer to the first prompt information, determining the number of times the user does not refer to the first prompt information; It is detected that the number of non-reference times is less than or equal to a preset number.

6. The method according to claim 5, characterized in that The method further comprises: If it is detected that the number of non-references is greater than the preset number, a confirmation message is output, where the confirmation message is used for the user to confirm whether to turn off the prompt function, where the prompt function is used to display a single prompt message.

7. The method according to claim 4, characterized in that After obtaining the second endoscopic image after the user performs the second closing step, the method further includes: determining whether the second closing step needs to be modified according to the second endoscopic image; If the second closing step needs to be corrected, guidance information is output, wherein the guidance information includes a deviation position and / or deviation angle of the closing clamp used in the second closing step.

8. The method according to claim 4, characterized in that After acquiring the second endoscopic image after the user performs the second closing step and before determining the second closing operation scheme according to the second endoscopic image, the method further includes: Outputting a closing result of the second closing step according to the second endoscopic image, the closing result including a closing operation type, a number of used closing clips, a closing state of the unclosed wound surface, and / or marking information of an incompletely closed area in the second endoscopic image; The closed state of the unclosed wound surface is detected as the unclosed wound surface is not completely closed.

9. The method according to any one of claims 1 to 8, characterized in that: The endoscope system further includes a display screen, the display screen is used to display a wound surface treatment interface, the wound surface treatment interface includes an endoscope image display area and a prompt information display area, after outputting the first prompt information according to the first closing operation scheme, the method further includes: The first endoscopic image with the unclosed wound surface marked is displayed in the endoscopic image display area; and the first prompt information is displayed in the prompt information display area.

10. The method according to claim 6, characterized in that The endoscope system further includes a display screen, and the display screen is used to display a wound surface treatment interface. After detecting that the number of non-reference times is greater than the preset number, outputting a confirmation message, the method further includes: A confirmation window pops up on the wound treatment interface, the confirmation window including the confirmation information, a confirmation control and a negation control, the confirmation control is used to determine whether to close the prompt function, and the negation control is used to determine whether to close the prompt function.

11. The method according to claim 7, characterized in that The endoscope system further includes a display screen, the display screen is used to display a wound surface treatment interface, the wound surface treatment interface includes a prompt information display area, if the second closing step needs to be corrected, after the guide information is output, the method further includes: The guide information is displayed in the prompt information display area.

12. The method according to claim 8, characterized in that The endoscope system further includes a display screen, the display screen is used to display a wound surface treatment interface, the wound surface treatment interface includes an endoscope image display area and a prompt information display area, after outputting the second prompt information according to the second closing operation scheme, the method further includes: The second endoscopic image in which the incompletely closed area is marked is displayed in the endoscopic image display area; and the second prompt information and the closing result are displayed in the prompt information display area.

13. An endoscope system, characterized in that: The endoscope system comprises: a processor, a display, and an endoscope; The endoscope is configured to execute the endoscopic image acquisition instructions of the processor; the display is used to display the endoscopic image; and the processor is configured to implement the method according to any one of claims 1-12.

Images