An endoscopic submucosal dissection auxiliary system based on deep learning

Abstract

The present invention relates to the technical field of endoscopic submucosal dissection, and discloses an endoscopic submucosal dissection auxiliary system based on deep learning, including: a variance calculation module, which is used to continuously obtain a plurality of frames of lesion observation images under endoscopy; for each frame of lesion observation image, the observation state characteristics of the lesion area are calculated; when the number of frames reaches a certain number, the variance is calculated, and if the variance is greater than the judgment threshold, a fine-tuning optimization module is entered; the fine-tuning optimization module is used to calculate the observation state characteristics of the new lesion area, and after calculating the nearest plurality of lesion observation images to the new lesion observation image, the points on the boundary of the new lesion area are sampled, and key points are matched with the nearest plurality of lesion observation images, the boundary of the new lesion area is optimized, and the position of the electrocoagulation mark is obtained and displayed. The problem of decreased attention of endoscopists as fatigue increases during operation is solved, and the accuracy of endoscopic submucosal dissection is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of endoscopic submucosal dissection, and in particular to an endoscopic submucosal dissection auxiliary system based on deep learning. Background Art

[0002] The statements in this section merely mention background art related to the present invention and do not necessarily constitute prior art.

[0003] Malignant tumors of the digestive tract seriously threaten people's life and health. Discovering and removing early digestive tract tumors through digestive endoscopy (gastroscopy or colonoscopy) can increase the 5-year survival rate to more than 90%. Endoscopic submucosal dissection (ESD) has the advantages of less trauma and high complete resection rate. It has become the preferred surgical method for early digestive tract cancer. ESD is a process of using special surgical instruments such as high-frequency electric knives in the digestive tract cavity to remove lesions confined to the mucosal layer or submucosal layer in one piece. The surgeon is required to accurately identify the cutting site and complete the surgical operation. However, the following problems still exist:

[0004] (1) Faced with the increasing number of surgical procedures, endoscopists’ attention will decrease as they become more tired during the operation, which may lead to inaccurate identification of the incision site;

[0005] (2) ESD surgery takes a long time. If the procedure is performed by an inexperienced endoscopist, the risk of intestinal bleeding and perforation during the operation may increase. This puts forward efficiency requirements, especially for the real-time performance of the navigation system. This is a test for the navigation model algorithm. The real-time performance of the good recognition system is poor, and the real-time performance of the good recognition system is difficult to meet the requirements.

[0006] (3) In ESD training for beginners, experienced endoscopists are needed to guide them in identifying the cutting site, which consumes a lot of manpower and hinders the promotion and application of ESD worldwide;

[0007] (4) Since ESD is a dynamic surgical process, one step follows another, and each step must have an accurate target. The identification results of these steps together constitute a landmark reference for navigation. If there is a slight error in one step, it will affect the operation. Summary of the invention

[0008] In order to address the deficiencies of the prior art, the present invention provides an endoscopic submucosal dissection auxiliary system based on deep learning, which ensures sufficient observation of the lesion area by calculating the variance of observation state features, and optimizes the lesion area through the fully observed images, thereby solving the problem of decreased attention of endoscopists as fatigue increases during operation, improving the accuracy of endoscopic submucosal dissection and reducing the burden on doctors.

[0009] In a first aspect, the present invention provides an endoscopic submucosal dissection auxiliary system based on deep learning;

[0010] An endoscopic submucosal dissection auxiliary system based on deep learning, comprising:

[0011] A variance calculation module is used to continuously acquire several frames of lesion observation images under endoscopy; for each frame of lesion observation image, a lesion area is obtained through a lesion segmentation model, and the observation state characteristics of the lesion area are calculated; when the number of frames of lesion observation images reaches a certain number, variance calculation is performed based on the observation state characteristics, and if the variance is greater than the judgment threshold, the fine-tuning optimization module is entered; otherwise, the lesion observation image is continuously acquired and the variance is recalculated;

[0012] The fine-tuning optimization module is used to obtain endoscopic new lesion observation images, obtain the new lesion area through the lesion segmentation model, and calculate the observation state characteristics of the new lesion area. After calculating several lesion observation images closest to the new lesion observation image in combination with the observation state characteristics of the lesion area, points on the boundary of the new lesion area are sampled, and key point matching is performed with the several lesion observation images closest to the new lesion observation image. Based on the matched key points, the boundary of the new lesion area is optimized, and the optimized boundary of the new lesion area is displayed as the position of the electrocoagulation mark.

[0013] Furthermore, the observation status feature includes: the distance between the centers of the maximum inscribed circle of the lesion area and the minimum circumscribed circle of the lesion area.

[0014] Furthermore, the observation state feature includes: a radius ratio of a maximum inscribed circle of the lesion area to a minimum circumscribed circle of the lesion area.

[0015] Furthermore, the observation status feature includes: the relative position of the center of the largest inscribed circle of the lesion area and the center point of the lesion observation image.

[0016] Furthermore, the relative position includes: a difference in abscissa and a difference in ordinate between the center of the largest inscribed circle of the lesion area and the center point of the lesion observation image.

[0017] Furthermore, it also includes a judgment threshold calculation module;

[0018] The judgment threshold calculation module is used to obtain the lesion area for each frame image in the lesion observation video clip through the lesion segmentation model, calculate the observation state characteristics of each frame image in the lesion observation video clip, and use the clustering algorithm to perform clustering, calculate the variance of the center points of all clusters, and select the judgment threshold based on the variance of all lesion observation video clips.

[0019] Furthermore, the variance is calculated by: after calculating the variance value of each dimension in the observation state feature, weighted averaging all the dimensional variance values.

[0020] Furthermore, if the lesion segmentation model obtains multiple lesion regions, the lesion region with the largest area is retained.

[0021] In a second aspect, the present invention further provides an electronic device, comprising:

[0022] a memory for non-transitory storage of computer-readable instructions; and

[0023] a processor for executing the computer readable instructions,

[0024] When the computer readable instructions are executed by the processor, the following steps are performed:

[0025] Continuously acquiring a number of frames of lesion observation images under endoscopy; for each frame of lesion observation image, obtaining the lesion area through the lesion segmentation model, and calculating the observation state characteristics of the lesion area; when the number of frames of lesion observation images reaches a certain number, performing variance calculation based on the observation state characteristics, if the variance is greater than the judgment threshold, acquiring a new lesion observation image under endoscopy; otherwise, continuing to acquire lesion observation images and recalculating the variance;

[0026] For new lesion observation images, the new lesion area is obtained through the lesion segmentation model, and the observation state characteristics of the new lesion area are calculated. After calculating several lesion observation images closest to the new lesion observation image in combination with the observation state characteristics of the lesion area, points on the boundary of the new lesion area are sampled, and key point matching is performed with the several lesion observation images closest to the new lesion observation image. Based on the matched key points, the boundary of the new lesion area is optimized, and the optimized boundary of the new lesion area is displayed as the position of the electrocoagulation mark.

[0027] In a third aspect, the present invention further provides a storage medium that non-temporarily stores computer-readable instructions, wherein when the non-temporary computer-readable instructions are executed by a computer, the following steps are performed:

[0028] Continuously acquiring a number of frames of lesion observation images under endoscopy; for each frame of lesion observation image, obtaining the lesion area through the lesion segmentation model, and calculating the observation state characteristics of the lesion area; when the number of frames of lesion observation images reaches a certain number, performing variance calculation based on the observation state characteristics, if the variance is greater than the judgment threshold, acquiring a new lesion observation image under endoscopy; otherwise, continuing to acquire lesion observation images and recalculating the variance;

[0029] For new lesion observation images, the new lesion area is obtained through the lesion segmentation model, and the observation state characteristics of the new lesion area are calculated. After calculating several lesion observation images closest to the new lesion observation image in combination with the observation state characteristics of the lesion area, points on the boundary of the new lesion area are sampled, and key point matching is performed with the several lesion observation images closest to the new lesion observation image. Based on the matched key points, the boundary of the new lesion area is optimized, and the optimized boundary of the new lesion area is displayed as the position of the electrocoagulation mark.

[0030] Compared with the prior art, the present invention has the following beneficial effects:

[0031] The present invention ensures sufficient observation of the lesion area by calculating the variance of the observation state characteristics, and optimizes the lesion area through the fully observed images, thereby solving the problem of decreased attention of endoscopists as they become more tired during the operation, improving the accuracy of endoscopic submucosal dissection and reducing the burden on doctors.

[0032] Advantages of additional aspects of the present invention will be partially given in the following description or learned through practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] The accompanying drawings in the specification, which constitute a part of the present invention, are used to provide a further understanding of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute improper limitations on the present invention.

[0034] Figure 1 This is a data flow diagram of an endoscopic submucosal dissection auxiliary system based on deep learning in the first embodiment. DETAILED DESCRIPTION

[0035] It should be noted that the following detailed descriptions are exemplary and are intended to provide further explanation of the present invention. Unless otherwise specified, all technical and scientific terms used herein have the same meanings as those commonly understood by those skilled in the art to which the present invention belongs.

[0036] In the absence of conflict, the embodiments of the present invention and the features of the embodiments may be combined with each other.

[0037] In this embodiment, all data is obtained in compliance with laws and regulations and based on the user's consent, and is used legally.

[0038] Embodiment 1

[0039] This embodiment provides an endoscopic submucosal dissection auxiliary system based on deep learning.

[0040] With the gradual development of digestive endoscopic surgical robots, the foundation and conditions for robotic automated surgery have been provided, and this method can be transferred to the field of automation at no cost.

[0041] Although some attempts have been made in the field of traditional image processing, they are limited to partial recognition of certain specific scenes, with weak generalization and inability to achieve accurate coverage. In recent years, artificial intelligence technology has developed rapidly, and deep learning has been applied to medical image analysis and has achieved great success. It can solve problems that were difficult to handle with traditional image processing. Deep learning has achieved very significant results in the diagnosis of gastrointestinal diseases and quality control of digestive endoscopy. However, deep learning has not been fully applied in endoscopic minimally invasive surgery, such as ESD surgery assisted navigation.

[0042] The present embodiment provides an endoscopic submucosal dissection auxiliary system based on deep learning, which solves the AI-assisted navigation of gastrointestinal diseases ESD under electronic endoscopy, recognizes and predicts surgical images collected under a single frame of electronic gastroscopy (or colonoscopy), can accurately identify the boundaries of early cancer, and display the cutting position in the image; it can provide a real-time position navigation method during endoscopic surgery, which can help doctors to make accurate cuts, reduce the risk of complications, assist in the training of beginners, reduce training costs, and promote the promotion and application of ESD.

[0043] A deep learning-based endoscopic submucosal dissection auxiliary system, such as Figure 1 As shown, including:

[0044] (1) A data collection module, which is configured to collect data required for model training and inspection analysis.

[0045] The data collection module is used to collect video clips of full observation of lesions during endoscopic examination or preoperative electrocoagulation marking, and to collect and deduplicate the collected video clips frame by frame and perform manual segmentation and labeling of lesion areas.

[0046] The data collection module is used to collect an equal number of lesion-free endoscopic images based on the number of lesion images that have been annotated.

[0047] (2) A model training module, which is configured to: train a lesion segmentation model.

[0048] Based on the lesion annotated images and lesion-free images collected by the data collection module, a deep learning segmentation algorithm is used for training to obtain a lesion segmentation model A.

[0049] (3) A video acquisition module, which is configured to acquire a video of a full observation of the lesion.

[0050] (4) An observation status statistics module, which is configured to perform lesion observation status statistics for a single video clip.

[0051] The observation state statistics module is used to use the lesion segmentation model A to identify the video clips collected by the video acquisition module frame by frame, retain the segmentation results with a lesion segmentation probability (i.e., the probability that the segmented area is judged to be a lesion) higher than a certain threshold, and calculate the minimum circumscribed circle and the maximum inscribed circle for the irregular lesion area of ​​each frame image. If there are multiple lesion areas in the same frame image, only the largest area is retained. If the lesion area is not identified, the frame image is directly discarded. The minimum circumscribed circle and the maximum inscribed circle are defined as follows:

[0052] Minimum circumscribed circle: the circle with the smallest area among all circles that can include the lesion area.

[0053] Largest inscribed circle: the circle with the largest area among all circles contained in the lesion area.

[0054] The observation status statistics module is used to roughly estimate the observation status of the lesion area.

[0055] Each sample is a feature vector composed of multiple dimensional features. The distance between the centers of the largest inscribed circle and the smallest circumscribed circle (d) is used to describe the distance of the lens. The radius ratio of the largest inscribed circle to the smallest circumscribed circle (r~=r 内 / r 外 ) is used to describe the morphological changes of the lesion; the relative position of the center of the maximum inscribed circle and the center of the image is used to describe the observation angle state (l horizontal, l 纵 ).

[0056] Among them l 横 , l 纵 are the horizontal and vertical deviation distances of the center of the maximum inscribed circle of the lesion area relative to the center of the image. Specifically, if the abscissa of the center of the maximum inscribed circle is greater than the abscissa of the center of the image, then l 横 The sign is positive, and the magnitude is the difference in the horizontal coordinates, and vice versa. 纵 Calculate in the same way.

[0057] The observation state statistics module is used to standardize different observation states and form a feature vector v for the lesion observation state of the image.

[0058] The lens distance state d is normalized and calculated based on the image size to obtain The range is [0,1]. No processing is required, the range is [0,1]. Observation angle state l 横 , l 纵 Based on the image size, the normalization process is The range is [-1,1].

[0059] The lesion observation state feature vector v of a single frame image can be expressed as

[0060] The judgment threshold calculation module is configured to: perform observation sufficiency statistics on the video clips to obtain a sufficient observation judgment threshold γ.

[0061] The judgment threshold calculation module is used to repeat the observation state statistics module to observe the state feature vector v of each frame image in the video clip (lesion observation video clip) n calculate.

[0062] The judgment threshold calculation module is used to cluster the lesion observation feature vectors of the video clip based on the k-means clustering algorithm to obtain K vector clusters and record the center point vectors of the K clusters.

[0063] The judgment threshold calculation module is used to calculate the variance of the K center point vectors to obtain the variance σ 2 , this step requires weighted calculation of different dimensions, because the importance and variation range of different dimensions are different, and the specific weights can be set separately according to the actual situation. Variance σ 2 It can represent the degree of adequacy of lesion observation in the video clip. The larger the variance, the more observation states there are.

[0064] The judgment threshold calculation module is used to perform the above variance calculation on all prepared video clips and calculate the obtained variance σ 2 Sort by size, taking 80% of the prepared videos as fully observed videos by default, and assigning their corresponding variance σ 2 As the sufficient observation judgment threshold γ.

[0065] Among them, for all preoperative preparation videos collected in the video acquisition module, 80% of the videos are assumed to be sufficiently observed. For example, if a total of 100 video clips are prepared, 80 of them are qualified and 20 are unqualified by default. The weighted variance of the multi-dimensional feature vectors of the 100 videos is calculated and sorted, and the 80th weighted variance from high to low is the sufficient observation judgment threshold γ.

[0066] An auxiliary module is configured to perform real-time navigation of focal submucosal dissection operation.

[0067] The auxiliary module is divided into two sub-modules. The variance calculation module collects the fully observed lesion features. At this time, the red dotted line displays the preliminary segmentation results in real time. After the variance calculation module collects enough features to be observed, it fine-tunes the subsequent lesion display. The green dotted line is used to display the optimized segmentation results. This is the fine-tuning optimization module.

[0068] Variance calculation module, used for lesion segmentation display and full observation and judgment (shown by red dotted line).

[0069] The variance calculation module first calls the observation state statistics module. During the inspection process, the acquired images are segmented by the lesion segmentation model A at 10 frames per second. The segmentation results are displayed in real time through the red dotted line, and the corresponding lesion observation state feature vector v is collected. Then, when the lesion observation state feature vector v (or lesion observation image) reaches a certain number, the variance calculation is performed. If it is greater than the judgment threshold γ, it means that sufficient observation has been completed and enters the fine-tuning optimization module. Otherwise, continue to collect and judge the observation situation.

[0070] The fine-tuning and optimization module is used for fine-tuning and optimization of the lesion segmentation area and final display (shown by the green dotted line).

[0071] The fine-tuning optimization module is used to perform inference segmentation on the newly acquired image (new lesion observation image) using the lesion segmentation model A to obtain the new lesion area, and calculate the observation state feature vector v of the new lesion area, and perform Euclidean distance calculation on all feature vectors obtained by full observation of the variance calculation module to find t images with similar distances.

[0072] The fine-tuning optimization module is used to sample points on the boundary of the newly acquired image lesion segmentation, describe them using SIFT, match key points with t images of similar distance, and determine whether the matching points of the t images corresponding to the sampling points on the boundary are located inside or outside the segmentation area. The current boundary position is optimized according to the proportion of the number of images inside and outside, and the optimized result is displayed as a green dotted line to prompt the doctor to mark the location for electrocoagulation.

[0073] Among them, the current boundary position is optimized according to the ratio of the number of images inside and outside. Specifically, the points on the lesion segmentation boundary of the current image are described using SIFT and matched with the points in t similar images collected previously. There is one matching point in each image, and t matching points in t images. Each of these t images has a lesion segmentation area. The number of matching points in each of the t images that are inside and outside the segmentation area is counted. If there are more points inside the segmentation area, it means that the boundary point of the current image needs to be extended outward, and vice versa. The extension distance takes the average value of the distance from the matching point to the boundary.

[0074] The variance is calculated for each sample in four dimensions, and four variance values ​​are obtained. The formula is as follows:

[0075]

[0076] where x ij is the value of the i-th sample on the j-th feature, represents the mean of the jth feature, and n represents the number of samples.

[0077] The four variance values ​​need to be weighted averaged to obtain the final weighted variance σ 2 .

[0078] Variance σ 2 It can represent the degree of sufficiency of the lesion observation in the video clip. The larger the variance, the more observed states. However, the importance of each dimensional feature on the state change is different, so it is necessary to add a weight to each dimension, and the specific size of the weight is set according to the actual situation.

[0079] The deep learning-based endoscopic submucosal dissection auxiliary system provided in this embodiment has been clinically tested to effectively improve the ESD operation and training effects, reduce the burden on doctors, and save medical resources.

[0080] This embodiment provides an endoscopic submucosal dissection auxiliary system based on deep learning, which solves the problem of decreased attention of endoscopists as they become more tired during the operation, improves cutting accuracy, and reduces the burden on doctors.

[0081] This embodiment provides an endoscopic submucosal dissection auxiliary system based on deep learning, which saves labor costs and improves training efficiency in ESD training for beginners.

[0082] The deep learning-based endoscopic submucosal dissection auxiliary system provided in this embodiment provides the basis and conditions for robotic automated surgery as digestive endoscopic surgical robots are gradually developing.

[0083] Embodiment 2

[0084] This embodiment further provides an electronic device, including: one or more processors, one or more memories, and one or more computer programs; wherein the processor is connected to the memory, the one or more computer programs are stored in the memory, and when the electronic device is running, the processor executes the one or more computer programs stored in the memory, so that the electronic device performs the following steps:

[0085] Continuously acquiring a number of frames of lesion observation images under endoscopy; for each frame of lesion observation image, obtaining the lesion area through the lesion segmentation model, and calculating the observation state characteristics of the lesion area; when the number of frames of lesion observation images reaches a certain number, performing variance calculation based on the observation state characteristics, if the variance is greater than the judgment threshold, acquiring a new lesion observation image under endoscopy; otherwise, continuing to acquire lesion observation images and recalculating the variance;

[0086] For new lesion observation images, the new lesion area is obtained through the lesion segmentation model, and the observation state characteristics of the new lesion area are calculated. After calculating several lesion observation images closest to the new lesion observation image in combination with the observation state characteristics of the lesion area, points on the boundary of the new lesion area are sampled, and key point matching is performed with the several lesion observation images closest to the new lesion observation image. Based on the matched key points, the boundary of the new lesion area is optimized, and the optimized boundary of the new lesion area is displayed as the position of the electrocoagulation mark.

[0087] It should be understood that in this embodiment, the processor may be a central processing unit CPU, and the processor may also be other general-purpose processors, digital signal processors DSP, application-specific integrated circuits ASIC, off-the-shelf programmable gate arrays FPGA or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.

[0088] The memory may include a read-only memory and a random access memory, and provide instructions and data to the processor. A portion of the memory may also include a non-volatile random access memory. For example, the memory may also store information about the device type.

[0089] In the implementation process, each step of the above method can be completed by an integrated logic circuit of hardware in a processor or an instruction in the form of software.

[0090] The method in the first embodiment can be directly embodied as a hardware processor, or a combination of hardware and software modules in the processor. The software module can be located in a mature storage medium in the field such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory, or an electrically erasable programmable memory, a register, etc. The storage medium is located in the memory, and the processor reads the information in the memory and completes the steps of the above method in combination with its hardware. To avoid repetition, it will not be described in detail here.

[0091] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with this embodiment can be implemented in electronic hardware or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.

[0092] Embodiment 3

[0093] This embodiment further provides a computer-readable storage medium for storing computer instructions. When the computer instructions are executed by a processor, the following steps are performed:

[0094] Continuously acquiring a number of frames of lesion observation images under endoscopy; for each frame of lesion observation image, obtaining the lesion area through the lesion segmentation model, and calculating the observation state characteristics of the lesion area; when the number of frames of lesion observation images reaches a certain number, performing variance calculation based on the observation state characteristics, if the variance is greater than the judgment threshold, acquiring a new lesion observation image under endoscopy; otherwise, continuing to acquire lesion observation images and recalculating the variance;

[0095] For new lesion observation images, the new lesion area is obtained through the lesion segmentation model, and the observation state characteristics of the new lesion area are calculated. After calculating several lesion observation images closest to the new lesion observation image in combination with the observation state characteristics of the lesion area, points on the boundary of the new lesion area are sampled, and key point matching is performed with the several lesion observation images closest to the new lesion observation image. Based on the matched key points, the boundary of the new lesion area is optimized, and the optimized boundary of the new lesion area is displayed as the position of the electrocoagulation mark.

[0096] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. An endoscopic submucosal dissection auxiliary system based on deep learning, characterized in that: include: A variance calculation module is used to continuously obtain a number of frames of lesion observation images under endoscopy; For each frame of lesion observation image, a lesion region is obtained through a lesion segmentation model, and observation state features of the lesion region are calculated; When the number of frames of the lesion observation image reaches a certain number, variance calculation is performed based on the observation state characteristics. If the variance is greater than the judgment threshold, the fine-tuning optimization module is entered; otherwise, the lesion observation image is continuously acquired and the variance is recalculated; The variance represents the adequacy of the observation of the lesion in the video clip, and the larger the variance, the more states are observed; The observation state features include: the distance between the centers of the maximum inscribed circle of the lesion area and the minimum circumscribed circle of the lesion area; The minimum circumscribed circle and the maximum inscribed circle are calculated for the irregular lesion area of ​​each frame image. If there are multiple lesion areas in the same frame image, only the largest area is retained. If no lesion area is identified, the frame image is directly discarded. The fine-tuning optimization module is used to obtain endoscopic observation images of new lesions, obtain the new lesion area through the lesion segmentation model, calculate the observation state characteristics of the new lesion area, and calculate t lesion observation images with a similar distance to the new lesion observation image in combination with the observation state characteristics of the lesion area. It is then used to sample points on the lesion segmentation boundary of the newly acquired image, describe them using SIFT, match key points with t images with similar distances, and determine whether the t image matching points corresponding to the sampling points on the boundary are located inside or outside the segmentation area; optimize the current boundary position according to the proportion of the number of images inside and outside, and display the boundary of the optimized new lesion area as the position of the electrocoagulation mark.

2. The deep learning-based endoscopic submucosal dissection auxiliary system according to claim 1, characterized in that: The observation state feature includes: a radius ratio of a maximum inscribed circle of the lesion area to a minimum circumscribed circle of the lesion area.

3. The deep learning-based endoscopic submucosal dissection auxiliary system according to claim 1, characterized in that: The observation state feature includes: the relative position of the center of the largest inscribed circle of the lesion area and the center point of the lesion observation image.

4. The deep learning-based endoscopic submucosal dissection auxiliary system according to claim 3, characterized in that: The relative position includes: the difference between the horizontal coordinate and the vertical coordinate of the center point of the lesion observation image and the center point of the lesion area.

5. The deep learning-based endoscopic submucosal dissection auxiliary system according to claim 1, characterized in that: It also includes a judgment threshold calculation module; The judgment threshold calculation module is used to obtain the lesion area for each frame image in the lesion observation video clip through the lesion segmentation model, calculate the observation state characteristics of each frame image in the lesion observation video clip, and use the clustering algorithm to perform clustering, calculate the variance of the center points of all clusters, and select the judgment threshold based on the variance of all lesion observation video clips.

6. The deep learning-based endoscopic submucosal dissection auxiliary system according to claim 1, characterized in that: The variance is calculated by: after calculating the variance value of each dimension in the observed state feature, weighted average of all the dimensional variance values ​​is performed.

7. An electronic device, comprising: a memory for non-transitory storage of computer readable instructions; as well as a processor for executing the computer readable instructions, When the computer readable instructions are executed by the processor, the following steps are performed: Continuously acquiring a number of frames of lesion observation images under endoscopy; for each frame of lesion observation image, obtaining the lesion area through the lesion segmentation model, and calculating the observation state characteristics of the lesion area; when the number of frames of lesion observation images reaches a certain number, performing variance calculation based on the observation state characteristics, if the variance is greater than the judgment threshold, acquiring a new lesion observation image under endoscopy; otherwise, continuing to acquire lesion observation images and recalculating the variance; The variance represents the adequacy of the observation of the lesion in the video clip, and the larger the variance, the more states are observed; The observation state features include: the distance between the centers of the maximum inscribed circle of the lesion area and the minimum circumscribed circle of the lesion area; The minimum circumscribed circle and the maximum inscribed circle are calculated for the irregular lesion area of ​​each frame image. If there are multiple lesion areas in the same frame image, only the largest area is retained. If no lesion area is identified, the frame image is directly discarded. For new lesion observation images, a new lesion area is obtained through a lesion segmentation model, and the observation state characteristics of the new lesion area are calculated. After t lesion observation images with a similar distance to the new lesion observation image are calculated in combination with the observation state characteristics of the lesion area, they are used to sample points on the lesion segmentation boundary of the newly acquired image, describe them using SIFT, perform key point matching with t images with a similar distance, and determine whether the t image matching points corresponding to the sampling points on the boundary are located inside or outside the segmentation area; optimize the current boundary position according to the proportion of the number of images inside and outside, and display the boundary of the optimized new lesion area as the position of the electrocoagulation mark.

8. A storage medium characterized by being non-temporary The computer readable instructions are stored in a memory, wherein when the computer readable instructions are executed by a computer, the following steps are performed: Continuously acquiring a number of frames of lesion observation images under endoscopy; for each frame of lesion observation image, obtaining the lesion area through the lesion segmentation model, and calculating the observation state characteristics of the lesion area; when the number of frames of lesion observation images reaches a certain number, performing variance calculation based on the observation state characteristics, if the variance is greater than the judgment threshold, acquiring a new lesion observation image under endoscopy; otherwise, continuing to acquire lesion observation images and recalculating the variance; The variance represents the adequacy of the observation of the lesion in the video clip, and the larger the variance, the more states are observed; The observation state features include: the distance between the centers of the maximum inscribed circle of the lesion area and the minimum circumscribed circle of the lesion area; The minimum circumscribed circle and the maximum inscribed circle are calculated for the irregular lesion area of ​​each frame image. If there are multiple lesion areas in the same frame image, only the largest area is retained. If no lesion area is identified, the frame image is directly discarded. For new lesion observation images, a new lesion area is obtained through a lesion segmentation model, and the observation state characteristics of the new lesion area are calculated. After t lesion observation images with a similar distance to the new lesion observation image are calculated in combination with the observation state characteristics of the lesion area, they are used to sample points on the lesion segmentation boundary of the newly acquired image, describe them using SIFT, perform key point matching with t images with a similar distance, and determine whether the t image matching points corresponding to the sampling points on the boundary are located inside or outside the segmentation area; optimize the current boundary position according to the proportion of the number of images inside and outside, and display the boundary of the optimized new lesion area as the position of the electrocoagulation mark.

Images