Endoscopic procedure monitoring method and apparatus

Abstract

This application discloses an endoscopic operation monitoring method and apparatus, comprising: acquiring the current image category of the site to be captured; determining the target predicted site category corresponding to the current endoscopic image based on the current endoscopic image; determining whether the target predicted site category corresponding to the current endoscopic image is the same as the current image category of the site to be captured; if they are the same, issuing a first prompt message; acquiring the image set of the target endoscope; when the total image capture time of multiple images captured by the target endoscope is greater than a preset time, determining the next image category of the target site in the preset image category set as the current image category, thereby obtaining the image set of each image category of the target site in the preset image category set; and determining endoscopic operation monitoring parameters based on the image sets of each image category of the target site in the preset image category set. This application can improve the accuracy and efficiency of endoscopic operations.

Description

Technical Field

[0001] This application relates to the field of image processing technology, specifically to an endoscopy operation monitoring method and device. Background Technology

[0002] A complete gastroscopy involves multiple areas. It requires the physician to observe all areas and take real-time images during the procedure, promptly conducting further detailed examinations when suspicious lesions are found. Physicians often need extensive experience to successfully complete a full gastroscopy, observe all areas, and produce an accurate diagnostic report. For physicians lacking clinical experience, missed diagnoses and misdiagnoses are common, leading to lower accuracy and efficiency in endoscopic procedures. Summary of the Invention

[0003] This application provides an endoscopic operation monitoring method and device, which can improve the accuracy and efficiency of endoscopic operations.

[0004] Firstly, the endoscopic procedure monitoring method provided in this application includes:

[0005] Obtain the category of the current image part to be retained, wherein the current image part category is one of the categories of image parts to be retained in a preset set of image part categories to be retained, and the categories of image parts to be retained in the preset set of image part categories to be retained are arranged in a preset order;

[0006] Acquire the current endoscopic image captured by the target endoscope;

[0007] Based on the current endoscopic image, determine the target predicted site category corresponding to the current endoscopic image;

[0008] Determine whether the target predicted region category corresponding to the current endoscopic image is the same as the region category of the current image to be retained;

[0009] If the target predicted part category corresponding to the current endoscopic image is the same as the part category to be saved in the current image, a first prompt message is issued. The first prompt message is used to prompt the user to observe for a preset time and save the image.

[0010] Multiple images captured by the target endoscope are acquired and used as the image set for the current image retention site category;

[0011] When the total image retention time of the target endoscope acquiring multiple images exceeds a preset time, the next image retention category in the preset image retention category set is determined as the current image retention category, thus obtaining the image retention image set for each image retention category in the preset image retention category set.

[0012] Endoscopic operation monitoring parameters are determined based on the image sets of each image category in the preset image category set.

[0013] Optionally, determining the target predicted site category corresponding to the current endoscopic image based on the current endoscopic image includes:

[0014] The current endoscopic image and multiple endoscopic images preceding the current endoscopic image are defined as multiple time-series images, wherein the multiple time-series images are arranged sequentially according to the order of their capture time;

[0015] When multiple time-series images are input into a preset convolutional neural network model, multiple first predicted part categories corresponding to the multiple time-series images are obtained;

[0016] Multiple time-series images and corresponding multiple first predicted part categories are input into a preset long short-term memory model to obtain the target predicted part category.

[0017] Optionally, determining the endoscopic operation monitoring parameters based on the image set of each image site category in the preset image site category set includes:

[0018] Obtain the target predicted part category of each image in the image set of the image to be retained;

[0019] The image that is the same as the target predicted part category as the part category to be retained is determined as the correctly recognized retained image, and the correct recognition percentage of the correctly recognized retained images in the image set is obtained, and the correct recognition percentage of each image set is obtained.

[0020] The category of the part to be retained corresponding to the image set whose correct recognition rate is greater than a preset rate is determined as the category of successfully recognized part and added to the set of successfully recognized part categories.

[0021] The category of the part to be retained corresponding to the image set whose correctly identified proportion is not greater than a preset proportion is determined as the unidentified part category and placed into the unidentified part category set.

[0022] Endoscopic operation monitoring parameters are determined based on the information from the set of successfully identified site categories and the information from the set of unidentified site categories.

[0023] Optionally, the endoscopic operation monitoring method includes:

[0024] If the target predicted part category corresponding to the current endoscopic image is different from the part category of the current image to be retained, a second prompt message is issued, which is used to prompt the user to move the target endoscope;

[0025] When movement of the target endoscope is detected, the first endoscopic image re-acquired by the target endoscope is acquired and the number of acquisitions is recorded;

[0026] If the target predicted part category corresponding to the first endoscopic image is different from the part category of the current image to be retained, then it is determined whether the number of times the image has been collected has reached the preset number.

[0027] If the number of times the images have been collected reaches a preset number, the current image part category to be retained is determined as an unidentified part category and placed into the unidentified part category set. The next image part category to be retained in the preset image part category set is determined as the current image part category to be retained, thus obtaining the image set of the next image part category to be retained.

[0028] Optionally, the endoscopic operation monitoring method includes:

[0029] If the target predicted region category corresponding to the first endoscopic image is the same as the region category of the current image to be retained, a first prompt message is issued.

[0030] Optionally, determining the endoscopic operation monitoring parameters based on the information of the successfully identified site category set and the information of the unidentified site category set includes:

[0031] Obtain the number of successfully identified body part categories in the set of successfully identified body part categories;

[0032] Obtain the number of unidentified part categories in the set of unidentified part categories;

[0033] The endoscopic operation monitoring parameters are determined based on the number of successfully identified site categories and the number of unidentified site categories.

[0034] Optionally, determining the endoscopic operation monitoring parameters based on the number of successfully identified site categories and the number of unidentified site categories includes:

[0035] Obtain the total endoscopic operation time of the target endoscope;

[0036] The time difference between two adjacent image part categories in the preset image part category set being determined as the current image part category is determined as the part dwell time of the image part category, thus obtaining the average part dwell time of each image part category in the preset image part category set;

[0037] The endoscopic operation monitoring parameters are determined based on the total duration of the gastroscopy operation, the average time spent at each site, the number of successfully identified site categories, and the number of unidentified site categories.

[0038] Secondly, the endoscopic operation monitoring device provided in this application includes:

[0039] The first acquisition module is used to acquire the category of the current image part to be retained, wherein the current image part to be retained category is one of the image part categories in the preset set of image part categories to be retained, and the image part categories to be retained in the preset set of image part categories to be retained are arranged in a preset order;

[0040] The second acquisition module is used to acquire the current endoscopic image collected by the target endoscope;

[0041] The first determining module is used to determine the target predicted site category corresponding to the current endoscopic image based on the current endoscopic image;

[0042] The judgment module is used to determine whether the target predicted part category corresponding to the current endoscopic image is the same as the part category of the current image to be retained;

[0043] The issuing module is used to issue a first prompt message if the target predicted part category corresponding to the current endoscopic image is the same as the part category to be retained in the current image. The first prompt message is used to prompt the user to observe for a preset time and retain the image.

[0044] The third acquisition module is used to acquire multiple images captured by the target endoscope as the image set of the current image-to-record site category;

[0045] The second determining module is used to determine the next image retention category in the preset image retention category set as the current image retention category when the total image retention time of the target endoscope acquiring multiple image retention images is greater than a preset time, thereby obtaining the image retention image set of each image retention category in the preset image retention category set.

[0046] The third determining module is used to determine the endoscopy operation monitoring parameters based on the image set of each image category in the preset image category set.

[0047] Optionally, determining the target predicted site category corresponding to the current endoscopic image based on the current endoscopic image includes:

[0048] The current endoscopic image and multiple endoscopic images preceding the current endoscopic image are defined as multiple time-series images, wherein the multiple time-series images are arranged sequentially according to the order of their capture time;

[0049] When multiple time-series images are input into a preset convolutional neural network model, multiple first predicted part categories corresponding to the multiple time-series images are obtained;

[0050] Multiple time-series images and corresponding multiple first predicted part categories are input into a preset long short-term memory model to obtain the target predicted part category.

[0051] Optionally, determining the endoscopic operation monitoring parameters based on the image set of each image site category in the preset image site category set includes:

[0052] Obtain the target predicted part category of each image in the image set of the image to be retained;

[0053] The image that is the same as the target predicted part category as the part category to be retained is determined as the correctly recognized retained image, and the correct recognition percentage of the correctly recognized retained images in the image set is obtained, and the correct recognition percentage of each image set is obtained.

[0054] The category of the part to be retained corresponding to the image set whose correct recognition rate is greater than a preset rate is determined as the category of successfully recognized part and added to the set of successfully recognized part categories.

[0055] The category of the part to be retained corresponding to the image set whose correctly identified proportion is not greater than a preset proportion is determined as the unidentified part category and placed into the unidentified part category set.

[0056] Endoscopic operation monitoring parameters are determined based on the information from the set of successfully identified site categories and the information from the set of unidentified site categories.

[0057] Optionally, the endoscopic operation monitoring method includes:

[0058] If the target predicted part category corresponding to the current endoscopic image is different from the part category of the current image to be retained, a second prompt message is issued, which is used to prompt the user to move the target endoscope;

[0059] When movement of the target endoscope is detected, the first endoscopic image re-acquired by the target endoscope is acquired and the number of acquisitions is recorded;

[0060] If the target predicted part category corresponding to the first endoscopic image is different from the part category of the current image to be retained, then it is determined whether the number of times the image has been collected has reached the preset number.

[0061] If the number of times the images have been collected reaches a preset number, the current image part category to be retained is determined as an unidentified part category and placed into the unidentified part category set. The next image part category to be retained in the preset image part category set is determined as the current image part category to be retained, thus obtaining the image set of the next image part category to be retained.

[0062] Optionally, the endoscopic operation monitoring method includes:

[0063] If the target predicted region category corresponding to the first endoscopic image is the same as the region category of the current image to be retained, a first prompt message is issued.

[0064] Optionally, determining the endoscopic operation monitoring parameters based on the information of the successfully identified site category set and the information of the unidentified site category set includes:

[0065] Obtain the number of successfully identified body part categories in the set of successfully identified body part categories;

[0066] Obtain the number of unidentified part categories in the set of unidentified part categories;

[0067] The endoscopic operation monitoring parameters are determined based on the number of successfully identified site categories and the number of unidentified site categories.

[0068] Optionally, determining the endoscopic operation monitoring parameters based on the number of successfully identified site categories and the number of unidentified site categories includes:

[0069] Obtain the total endoscopic operation time of the target endoscope;

[0070] The time difference between two adjacent image part categories in the preset image part category set being determined as the current image part category is determined as the part dwell time of the image part category, thus obtaining the average part dwell time of each image part category in the preset image part category set;

[0071] The endoscopic operation monitoring parameters are determined based on the total duration of the gastroscopy operation, the average time spent at each site, the number of successfully identified site categories, and the number of unidentified site categories.

[0072] Thirdly, the electronic device provided in this application includes a memory and a processor. The memory stores a computer program, and the processor is used to run the computer program in the memory to implement the steps in the endoscopy operation monitoring method provided in this application.

[0073] Fourthly, the computer-readable storage medium provided in this application stores multiple instructions that are adapted for loading by a processor to implement the steps in the endoscopy operation monitoring method provided in this application.

[0074] Fifthly, the computer program product provided in this application includes a computer program or instructions that, when executed by a processor, implement the steps in the endoscopic operation monitoring method provided in this application.

[0075] In this application, compared to related technologies, the method involves: obtaining the category of the current image part to be retained, wherein the current image part category is one of the categories in a preset set of image part categories to be retained, and the categories in the preset set are arranged in a preset order; acquiring the current endoscopic image acquired by the target endoscope; determining the target predicted part category corresponding to the current endoscopic image based on the current endoscopic image; determining whether the target predicted part category corresponding to the current endoscopic image is the same as the current image part category; and if the target predicted part category corresponding to the current endoscopic image is the same as the current image part category. The system then issues a first prompt message to the user, prompting them to observe for a preset time and capture images. Multiple images captured by the target endoscope are acquired as the image set for the current image-capturing site category. When the total image capture time exceeds the preset time, the next image-capturing site category in the preset image-capturing site category set is determined as the current image-capturing site category, resulting in image sets for each image-capturing site category in the preset image-capturing site category set. Endoscopic operation monitoring parameters are determined based on the image sets for each image-capturing site category in the preset image-capturing site category set. During endoscopic operation, this application guides the user to capture images at each image-capturing site according to a pre-set order. When an image matches the image-capturing site, the user is prompted to perform the operation. Furthermore, the system can determine endoscopic operation monitoring parameters based on the captured image information to accurately assess the operation, thereby improving the accuracy and efficiency of endoscopic operations. Attached Figure Description

[0076] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0077] Figure 1 This is a schematic diagram of a scenario for the endoscopy operation monitoring system provided in an embodiment of this application;

[0078] Figure 2 This is a flowchart illustrating one embodiment of the endoscopic operation monitoring method provided in this application.

[0079] Figure 3 and Figure 4 This is a schematic diagram of the images captured for 26 different locations in one embodiment of the endoscopic operation monitoring method provided in this application.

[0080] Figure 5 This is a schematic diagram of an LSTM model in one embodiment of the endoscopic operation monitoring method provided in this application;

[0081] Figure 6 This is a schematic diagram of the endoscopy operation monitoring device provided in the embodiments of this application;

[0082] Figure 7 This is a schematic diagram of the structure of the electronic device provided in the embodiments of this application. Detailed Implementation

[0083] It should be noted that the principles of this application are illustrated by example in a suitable computing environment. The following description is based on the specific embodiments of this application that are illustrated, and should not be regarded as limiting other specific embodiments not detailed herein.

[0084] In the following description of this application, "some embodiments" are referred to, which describe a subset of all possible embodiments. However, it is understood that "some embodiments" may be the same subset or different subset of all possible embodiments, and may be combined with each other without conflict.

[0085] In the following description of this application, the terms "first, second, third" are used merely to distinguish similar objects and do not represent a specific ordering of objects. It is understood that "first, second, third" may be interchanged in a specific order or sequence where permitted, so that the embodiments of this application described herein can be implemented in an order other than that illustrated or described herein.

[0086] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein is for the purpose of describing embodiments of this application only and is not intended to limit this application.

[0087] To improve the effectiveness of endoscopic procedure monitoring, this application provides an endoscopic procedure monitoring method, an endoscopic procedure monitoring device, an electronic device, a computer-readable storage medium, and a computer program product. The endoscopic procedure monitoring method can be executed by the endoscopic procedure monitoring device or by an electronic device integrating the endoscopic procedure monitoring device.

[0088] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0089] Please refer to Figure 1 This application also provides an endoscopic operation monitoring system, such as Figure 1As shown, the endoscopic operation monitoring system includes an electronic device 100 and a target endoscope 200. The electronic device 100 integrates the endoscopic operation monitoring device provided in this application. The electronic device 100 and the target endoscope 200 are connected. The target endoscope 200 can be a gastroscope, colonoscope, etc.

[0090] Among them, electronic device 100 can be any device equipped with a processor and having processing capabilities, such as mobile electronic devices with processors such as smartphones, tablets, PDAs, laptops, and smart speakers, or fixed electronic devices with processors such as desktop computers, televisions, servers, and industrial equipment.

[0091] In addition, such as Figure 1 As shown, the endoscopic operation monitoring system may also include a memory for storing raw data, intermediate data, and result data.

[0092] In this embodiment of the application, the storage device can be a cloud storage device. Cloud storage is a new concept that is extended and developed from the concept of cloud computing. A distributed cloud storage system (hereinafter referred to as a storage system) refers to a storage system that uses cluster applications, grid technology and distributed storage file system functions to bring together a large number of storage devices of various types in the network (storage devices are also called storage nodes) through application software or application interfaces to work together to provide data storage and business access functions to the outside world.

[0093] Currently, the storage method in storage systems is as follows: Logical volumes are created, and during creation, physical storage space is allocated to each logical volume. This physical storage space may consist of a single storage device or the disks of several storage devices. Clients store data on a logical volume, which means storing the data on the file system. The file system divides the data into many parts, each part being an object. Each object contains not only the data but also additional information such as a data identifier (ID, ID entity). The file system writes each object to the physical storage space of that logical volume and records the storage location information of each object. Therefore, when a client requests access to data, the file system can allow the client to access the data based on the storage location information of each object.

[0094] The process by which a storage system allocates physical storage space to a logical volume is as follows: the physical storage space is pre-divided into strips according to the capacity estimate of the objects stored in the logical volume (this estimate often has a large margin relative to the actual capacity of the objects to be stored) and the grouping of Redundant Array of Independent Disks (RAID). A logical volume can be understood as a strip, thus allocating physical storage space to the logical volume.

[0095] It should be noted that, Figure 1 The schematic diagram of the endoscopy operation monitoring system shown is merely an example. The endoscopy operation monitoring system and scenario described in this application are for the purpose of more clearly illustrating the technical solutions of this application, and do not constitute a limitation on the technical solutions provided in this application. As those skilled in the art will know, with the evolution of endoscopy operation monitoring systems and the emergence of new business scenarios, the technical solutions provided in this application are also applicable to similar technical problems.

[0096] The following sections provide detailed descriptions of each example. It should be noted that the sequence numbers of the following embodiments are not intended to limit the preferred order of the embodiments.

[0097] Please refer to Figure 2 , Figure 2 This is a flowchart illustrating one embodiment of the endoscopic operation monitoring method provided in this application, as shown below. Figure 2 As shown, the procedure for monitoring endoscopic procedures provided in this application is as follows:

[0098] 201. Obtain the category of the part of the image to be retained.

[0099] A gastroscopy is a medical examination method, and also refers to the instrument used in this examination. It uses a thin, flexible tube inserted into the stomach, allowing doctors to directly observe lesions in the esophagus, stomach, and duodenum, especially tiny lesions, under direct vision.

[0100] Artificial intelligence (AI) is the study of using computers to simulate certain human thought processes and intelligent behaviors (such as learning, reasoning, thinking, and planning). It primarily includes understanding the principles of computer intelligence, creating computers that resemble human brain intelligence, and enabling computers to achieve higher-level applications. AI involves disciplines such as computer science, psychology, philosophy, and linguistics. It encompasses almost all disciplines in the natural and social sciences, extending far beyond the scope of computer science. The relationship between AI and cognitive science is one of practice and theory; AI is at the technological application level of cognitive science, a branch of its application. From a cognitive perspective, AI is not limited to logical thinking; it must also consider visual and intuitive thinking to promote breakthroughs. Mathematics is often considered the foundation of many disciplines and has permeated areas such as language and thought. AI must also utilize mathematical tools; mathematics plays a role not only in standard logic and fuzzy mathematics but also mutually promotes and accelerates AI's development.

[0101] Deep learning (DL) is a new research direction in the field of machine learning (ML). It was introduced into machine learning to bring it closer to its original goal—artificial intelligence (AI). Deep learning learns the inherent patterns and hierarchical representations of sample data. The information gained during this learning process greatly aids in the interpretation of data such as text, images, and sound. Its ultimate goal is to enable machines to possess analytical and learning capabilities like humans, capable of recognizing data such as text, images, and sound. Deep learning is a complex machine learning algorithm that has achieved far greater results than previous related technologies in speech and image recognition. Deep learning has also made significant progress in search technology, data mining, machine learning, machine translation, natural language processing, multimedia learning, speech recognition, recommendation and personalization technologies, and other related fields. Deep learning enables machines to mimic human activities such as sight, hearing, and thought, solving many complex pattern recognition problems and leading to significant advancements in artificial intelligence-related technologies.

[0102] The current category of the image to be retained is one of the categories of the image to be retained from the preset set of categories of image to be retained. The categories of the image to be retained in the preset set of categories of image to be retained are arranged in a preset order.

[0103] In this embodiment, the endoscopic device is started. This endoscopic device can be a gastroscope, colonoscope, etc., depending on the specific situation. The endoscopic host is turned on, and the cleaned target endoscope is connected to the endoscopic host. The image is ensured to be normal, and the endoscopic host parameters are adjusted.

[0104] For example, the endoscopic device is a gastroscope. Turn on the gastroscope, turn on the electronic system, connect the cleaned gastroscope to the electronic system, ensure the image is normal, and adjust the electronic system parameters. (This process is repeated three times in the original text.)

[0105] For example, the endoscope is a colonoscope. Turn on the colonoscope, turn on the electronic system, connect the cleaned colonoscope to the electronic system, ensure the image is normal, and adjust the electronic system parameters. Turn on the electronic system, connect the cleaned colonoscope to the electronic system, ensure the image is normal, and adjust the electronic system parameters.

[0106] Start the endoscopy operation monitoring system. Connect the input terminal of the electronic device to the signal output terminal of the target endoscope and confirm that the signal connection is successful. Once the electronic device and the target endoscope are successfully connected, the system will prompt the physician that the endoscopic operation can begin.

[0107] Among them, the categories of the parts to be retained in the preset set of parts to be retained are arranged in a preset order. For example, the preset set of image retention categories includes 26 image retention categories, and the preset order of the 26 image retention categories is: esophagus, cardia, greater curvature of the gastric antrum, posterior wall of the gastric antrum, anterior wall of the gastric antrum, lesser curvature of the gastric antrum, duodenal bulb, descending duodenum, lower greater curvature of the gastric body (positive view), lower posterior wall of the gastric body (positive view), lower anterior wall of the gastric body (positive view), lower lesser curvature of the gastric body (positive view), upper-middle greater curvature of the gastric body (positive view), upper-middle posterior wall of the gastric body (positive view), upper-middle anterior wall of the gastric body (positive view), upper-middle lesser curvature of the gastric body (positive view), greater curvature of the gastric fundus (reverse view), posterior wall of the gastric fundus (reverse view), anterior wall of the gastric fundus (reverse view), lesser curvature of the gastric fundus (reverse view), upper-middle posterior wall of the gastric body (reverse view), upper-middle anterior wall of the gastric body (reverse view), upper-middle lesser curvature of the gastric body (reverse view), posterior wall of the gastric angle (reverse view), anterior wall of the gastric angle (reverse view), lesser curvature of the gastric angle (reverse view).

[0108] like Figure 3 and Figure 4 As shown, Figure 3 and Figure 4 The image captures are shown for 26 different regions to be captured. Figure 3 The images captured are as follows: esophagus, cardia, greater curvature of the gastric antrum, posterior wall of the gastric antrum, anterior wall of the gastric antrum, lesser curvature of the gastric antrum, duodenal bulb, descending part of the duodenum, greater curvature of the lower part of the gastric body (frontal view), posterior wall of the lower part of the gastric body (frontal view), anterior wall of the lower part of the gastric body (frontal view), and lesser curvature of the lower part of the gastric body (frontal view). Figure 4 The images captured are as follows: the greater curvature of the upper part of the stomach body in frontal view, the posterior wall of the upper part of the stomach body in frontal view, the anterior wall of the upper part of the stomach body in frontal view, the lesser curvature of the upper part of the stomach body in frontal view, the greater curvature of the stomach fundus in reverse view, the posterior wall of the stomach fundus in reverse view, the anterior wall of the stomach fundus in reverse view, the lesser curvature of the stomach fundus in reverse view, the posterior wall of the upper part of the stomach body in reverse view, the anterior wall of the upper part of the stomach body in reverse view, the lesser curvature of the upper part of the stomach body in reverse view, the posterior wall of the stomach angle in reverse view, the anterior wall of the stomach angle in reverse view, and the lesser curvature of the stomach angle in reverse view.

[0109] For example, the current category of the part to be captured is the esophagus.

[0110] 202. Obtain the current endoscopic image captured by the target endoscope.

[0111] In this embodiment, the target endoscope automatically acquires endoscopic images at a preset cycle after user operation, thus obtaining the current endoscopic image acquired by the target endoscope. The preset cycle can be 1 second, 2 seconds, etc., and can be set according to specific circumstances.

[0112] 203. Determine the target prediction site category corresponding to the current endoscopic image based on the current endoscopic image.

[0113] In this embodiment of the application, determining the target prediction site category corresponding to the current endoscopic image based on the current endoscopic image includes:

[0114] (1) The current endoscopic image and the multiple endoscopic images preceding the current endoscopic image are determined as multiple time-series images, wherein the multiple time-series images are arranged in order of their capture time.

[0115] For example, obtain N time series images.

[0116] (2) When multiple time series images are input into a preset convolutional neural network model, multiple first predicted part categories corresponding to multiple time series images are obtained.

[0117] In this embodiment, the pre-defined convolutional neural network model is a CNN model. The CNN model identifies the corresponding typical parts in a single input time-series image based on the weights corresponding to 26 typical parts of the stomach, thereby classifying the parts of the single time-series image to obtain the first predicted part category of the time-series image, resulting in N time-series images and their corresponding first predicted part categories.

[0118] (3) Input multiple time series images and corresponding multiple first predicted part categories into a preset long short memory model to obtain the target predicted part category.

[0119] The default Long Short-Term Memory (LSTM) model is used. The LSTM model outputs the body part category of the last image in a series of N consecutive images; that is, it outputs the predicted target body part category of the current endoscopic image. The network structure of the LSTM model is as follows: Figure 5 As shown.

[0120] Furthermore, the first predicted region category of the current endoscopic image is obtained from the output of a preset convolutional neural network model, and the second predicted region category of the current endoscopic image is obtained from the output of a preset long short-term memory model. If the first predicted region category and the second predicted region category of the current endoscopic image are the same, then the first predicted region category of the current endoscopic image is determined as the target predicted region category of the current endoscopic image; if the first predicted region category and the second predicted region category of the current endoscopic image are different, then the unrecognizable category is determined as the target predicted region category of the current endoscopic image.

[0121] 204. Determine whether the target prediction region category corresponding to the current endoscopic image is the same as the region category of the current image to be retained.

[0122] 205. If the target predicted part category corresponding to the current endoscopic image is the same as the part category to be retained in the current image, a first prompt message is issued. The first prompt message is used to remind the user to observe for the preset time and retain the image.

[0123] The preset duration can be 5 seconds, 6 seconds, etc., which can be determined according to the specific situation.

[0124] In this embodiment, if the target predicted region category corresponding to the current endoscopic image is the same as the region category to be captured, a first prompt message is issued. The first prompt message prompts the user to observe for a preset time and capture the image. After receiving the first prompt message, the user operates the target endoscope to capture the image, obtaining multiple captured images.

[0125] Further methods for monitoring endoscopic procedures include:

[0126] (1) If the target predicted part category corresponding to the current endoscope image is different from the current part category to be retained, a second prompt message is issued. The second prompt message is used to prompt the user to move the target endoscope.

[0127] (2) When the target endoscope is detected to move, the first endoscope image re-acquired by the target endoscope is acquired and the number of acquisitions is recorded.

[0128] (3) If the target predicted part category corresponding to the first endoscopic image is different from the part category of the current image to be retained, then determine whether the number of times the images have been collected has reached the preset number.

[0129] The preset number of attempts is set according to the specific circumstances.

[0130] (4) If the number of collections reaches the preset number, the current image part category to be retained is determined as the unidentified part category and placed into the unidentified part category set. The next image part category to be retained in the preset image part category set is determined as the current image part category to be retained, and the image set of the next image part category to be retained is obtained.

[0131] If the preset number of image captures has been reached, it indicates that there may be a user error at this point, and a matching image cannot be obtained. In this case, the image of the next part will be captured.

[0132] Furthermore, if the target predicted region category corresponding to the first endoscopic image is the same as the region category of the current image to be retained, a first prompt message is issued.

[0133] 206. Acquire multiple images captured by the target endoscope as the image set for the current image retention site category.

[0134] In this embodiment of the application, multiple images are obtained by the user operating the target endoscope and capturing the images.

[0135] 207. When the total image retention time of multiple images acquired by the target endoscope exceeds the preset time, the next image retention category in the preset image retention category set is determined as the current image retention category, and the image retention sets of each image retention category in the preset image retention category set are obtained.

[0136] In this embodiment of the application, when the total image retention time of the target endoscope for acquiring multiple images is greater than 5 seconds, it indicates that the user continues to operate at the current site and remains at the current site for more than 5 seconds, prompting the user that the current site examination has been completed and proceeding to the next site in sequence.

[0137] In this embodiment of the application, each image part category in the preset set of image part categories to be retained is determined as the current image part category to be retained, thereby obtaining the image set of each image part category to be retained in the preset set of image part categories to be retained.

[0138] 208. Determine the endoscopy operation monitoring parameters based on the image set of each image part category in the preset image part category set.

[0139] In this embodiment of the application, after obtaining the image set of each image part category in the preset image part category set, the endoscopy operation monitoring parameters are determined based on the image set of each image part category in the preset image part category set.

[0140] In this embodiment of the application, the endoscopic operation monitoring parameters are determined based on the image sets of each image category in the preset set of image categories to be retained, including:

[0141] (1) Obtain the target predicted part category of each image in the image set of the image to be retained.

[0142] (2) The images whose target predicted part category is the same as the part category to be retained are identified as correctly retained images. The correct recognition percentage of the correctly retained images in the image set is obtained, and the correct recognition percentage of each image set is obtained.

[0143] In this embodiment of the application, the target predicted part category of the retained image is the same as the part category to be retained, indicating that the observation is performed in the correct part, and the retained image is determined to be the correctly identified retained image.

[0144] (3) The category of the part to be retained corresponding to the image set with the correct recognition rate greater than the preset rate is determined as the category of successfully recognized part and put into the set of successfully recognized part categories.

[0145] The preset percentage can be 80%, 90%, etc., depending on the specific setting, and this application does not limit it.

[0146] (4) The category of the part to be retained corresponding to the image set with the correct recognition rate not greater than the preset rate is determined as the category of unrecognized part and put into the unrecognized part category set.

[0147] (5) Determine the endoscopic operation monitoring parameters based on the information of the successfully identified site category set and the information of the unidentified site category set.

[0148] In this embodiment of the application, endoscopic operation monitoring parameters are determined based on information from the successfully identified site category set and information from the unidentified site category set, including:

[0149] (1) Obtain the number of successfully identified part categories in the set of successfully identified part categories.

[0150] (2) Obtain the number of unidentified part categories in the set of unidentified part categories.

[0151] (3) Determine the endoscopy operation monitoring parameters based on the number of successfully identified site categories and the number of unidentified site categories.

[0152] In one specific embodiment, the number of successfully identified site categories and the number of unidentified site categories are weighted and summed based on preset weights to obtain the endoscopic operation monitoring parameters.

[0153] In another specific embodiment, endoscopic procedure monitoring parameters are determined based on the number of successfully identified site categories and the number of unidentified site categories, including:

[0154] (1) The total time of gastroscopy operation to obtain the target endoscope.

[0155] In this embodiment of the application, timing begins when the target endoscope is detected to enter the human body and stops when the target endoscope is detected to leave the human body. The timing time at this point is determined as the total time of the gastroscopy operation of the target endoscope.

[0156] (2) The time difference between two adjacent categories of the map to be retained in the preset set of map to be retained is determined as the current category of the map to be retained. The average duration of the map to be retained is obtained.

[0157] The time difference between two adjacent regions being identified as the current region category can represent the duration of the region category remaining in the previous region category.

[0158] (3) Endoscopic operation monitoring parameters are determined based on the total duration of gastroscopy, the average time spent at each site, the number of successfully identified site categories, and the number of unidentified site categories.

[0159] In one specific embodiment, endoscopic operation monitoring parameters are determined based on preset weighting coefficients for the total duration of gastroscopy, average site dwell time, number of successfully identified site categories, and number of unidentified site categories. Specifically, the longer the total duration of gastroscopy, the lower the endoscopic operation monitoring parameters; the longer the average site dwell time, the lower the endoscopic operation monitoring parameters; the smaller the number of successfully identified site categories, the lower the endoscopic operation monitoring parameters; and the larger the number of unidentified site categories, the lower the endoscopic operation monitoring parameters.

[0160] Furthermore, after determining the monitoring parameters for the endoscopic procedure, the system will indicate the end of the gastroscopy once the endoscope is removed from the body. Based on the user's actual experience during the procedure, the system will provide an AI analysis report, which includes the following information:

[0161]

[0162] Furthermore, to improve the accuracy of endoscopic operation monitoring parameters, these parameters are determined based on the total duration of the gastroscopy operation, the average time spent at each site, the number of successfully identified site categories, and the number of unidentified site categories. This includes: inputting each captured image from the image set into a preset sharpness detection model to obtain the sharpness of each captured image; determining the average sharpness of the captured images in the image set as the average sharpness of the captured image set; and identifying the site categories to be captured corresponding to captured images with an average sharpness greater than a preset sharpness as clear captured site categories, thus obtaining the number of clear captured site categories. The endoscopic operation monitoring parameters are determined based on the total duration of the gastroscopy operation, the average time spent at each site, the number of successfully identified site categories, the number of clear captured site categories, and the number of unidentified site categories.

[0163] Specifically, endoscopic operation monitoring parameters are obtained by weighting and summing the total gastroscopy operation time, average site dwell time, number of successfully identified site categories, number of site categories with clear images, and number of unidentified site categories based on preset weighting coefficients. Among these, the longer the total gastroscopy operation time, the lower the endoscopic operation monitoring parameters; the longer the average site dwell time, the lower the endoscopic operation monitoring parameters; the smaller the number of successfully identified site categories, the lower the endoscopic operation monitoring parameters; the larger the number of unidentified site categories, the lower the endoscopic operation monitoring parameters; and the smaller the number of site categories with clear images, the lower the endoscopic operation monitoring parameters.

[0164] For example, suppose the weighting coefficients for the total duration of the gastroscopy procedure, the average time spent at each site, the number of successfully identified site categories, the number of site categories with clear images, and the number of unidentified site categories are set to -0.3, -0.2, 0.2, 0.15, and 0.15, respectively. In a certain gastroscopy procedure, the total duration of the procedure is 30 minutes (corresponding to a weighting coefficient of -0.3, calculated as -30 × 0.3 = -9), the average time spent at each site is 2 minutes (corresponding to a weighting coefficient of -0.2, calculated as -2 × 0.2 = -0.4), the number of successfully identified site categories is 5 (corresponding to a weighting coefficient of 0.2, calculated as 5 × 0.2 = 1), the number of site categories with clear images is 4 (corresponding to a weighting coefficient of 0.15, calculated as 4 × 0.15 = 0.6), and the number of unidentified site categories is 2 (corresponding to a weighting coefficient of 0.15, calculated as 2 × 0.15 = 0.3). Adding these results together, we get -9 + (-0.4) + 1 + 0.6 + 0.3 = -7.5, which gives us the monitoring parameter for this endoscopic procedure as -7.5.

[0165] Furthermore, to improve the accuracy of endoscopic operation monitoring parameters, endoscopic operation monitoring parameters are determined based on the total duration of gastroscopy, average site dwell time, number of successfully identified site categories, number of site categories with clear images, and number of unidentified site categories. These parameters include: inputting each image from the image set into a preset shooting angle detection model to obtain the shooting angle of each image; calculating the variance of the shooting angles of the images in the image set to obtain the angle variance corresponding to each image set; identifying the site categories to be captured corresponding to image sets with angle variances greater than a preset variance value as angle-qualified image-capturing site categories, and obtaining the number of angle-qualified image-capturing site categories.

[0166] Specifically, endoscopic operation monitoring parameters are obtained by weighting and summing the total gastroscopy operation time, average site dwell time, number of successfully identified site categories, number of site categories with acceptable angles, number of site categories with clear images, and number of unidentified site categories based on preset weighting coefficients. Among these, the longer the total gastroscopy operation time, the lower the endoscopic operation monitoring parameters; the longer the average site dwell time, the lower the endoscopic operation monitoring parameters; the smaller the number of successfully identified site categories, the lower the endoscopic operation monitoring parameters; the larger the number of unidentified site categories, the lower the endoscopic operation monitoring parameters; the smaller the number of site categories with clear images, the lower the endoscopic operation monitoring parameters; and the smaller the number of site categories with acceptable angles, the lower the endoscopic operation monitoring parameters.

[0167] Assume the weighting coefficients for the total endoscopy duration, average site dwell time, number of successfully identified site categories, number of site categories with acceptable angles for image capture, number of site categories with clear images, and number of unidentified site categories are -0.2, -0.15, -0.15, -0.1, 0.15, and -0.2, respectively. In a particular endoscopy procedure, the total duration was 25 minutes (calculated as 25 × -0.2 = -5), the average site dwell time was 1.8 minutes (calculated as 1.8 × -0.15 = -0.27), the number of successfully identified site categories was 6 (calculated as 6 × 0.15 = 0.9), the number of site categories with acceptable angles for image capture was 3 (calculated as 3 × 0.1 = 0.3), the number of site categories with clear images was 4 (calculated as 4 × 0.15 = 0.6), and the number of unidentified site categories was 3 (calculated as 3 × 0.2 = 0.6). Adding these results together, we get -5 + (-0.27) + 0.9 + 0.3 + 0.6 + (-0.6) = -4.07, which gives us the monitoring parameter for this endoscopic procedure as -4.07.

[0168] Furthermore, when the endoscopy operation monitoring parameters fall below the preset parameter values, a third prompt message is issued to inform the user that the operation score is low. The preset value can be -4, which can be set according to the specific time.

[0169] This application identifies the location of a patient within a real-time image captured from a gastroscopy sequence. It utilizes a convolutional neural network (CNN) model trained with backpropagation and a long short-term memory (LSTM) network model. The CNN model classifies the preprocessed image by location and lesion features, while the LSTM model performs sequence fitting on the classification results to obtain the identification outcome. This guides the physician to carefully observe the current location before moving on to the next, up to a total of 26 locations. This effectively assists endoscopists, improving their gastroscopy skills, reducing the rate of missed diagnoses of gastric diseases, and lowering the risk of cancer in patients. This invention performs image quality recognition, location identification, and feature recognition on the acquired images, displaying the results on a client-side interface. This provides operators with more reliable reference data, improving the accuracy and effectiveness of detection. It is simple and easy to use, avoiding secondary patient discomfort and additional medical expenses due to incomplete initial examinations.

[0170] This application addresses the problems of complex gastroscopy procedures, high skill requirements for physicians, and the potential for image blind spots and missed lesion diagnoses. It provides physicians with accurate and reliable references, improves the accuracy and effectiveness of detection, is simple to use, and has significant social and economic value.

[0171] This application can improve the standardization of doctors' operations. For beginners in gastroscopy, it is unsafe and unethical if they do not have the guidance of senior doctors. However, the teaching of each clinician is different. Adding a gastroscopy-assisted teaching system can standardize both the instructors and learners. Under the premise of standardization, through repeated training, doctors will go further and patients will also benefit.

[0172] To facilitate better implementation of the endoscopic operation monitoring method provided in this application, this application also provides an endoscopic operation monitoring device based on the above-described endoscopic operation monitoring method. The meanings of the terms used are the same as in the above-described endoscopic operation monitoring method; for specific implementation details, please refer to the descriptions in the above method embodiments.

[0173] Please refer to Figure 6 , Figure 6 This is a schematic diagram of the structure of the endoscopy operation monitoring device provided in the embodiments of this application. The endoscopy operation monitoring device may include:

[0174] The first acquisition module 701 is used to acquire the current image part category to be retained, wherein the current image part category to be retained is one of the image part categories to be retained in the preset image part category set, and the image part categories to be retained in the preset image part category set are arranged in a preset order;

[0175] The second acquisition module 702 is used to acquire the current endoscopic image collected by the target endoscope;

[0176] The first determining module 703 is used to determine the target prediction site category corresponding to the current endoscope image based on the current endoscope image;

[0177] The judgment module 704 is used to determine whether the target predicted part category corresponding to the current endoscopic image is the same as the part category of the current image to be retained.

[0178] The issuing module 705 is used to issue a first prompt message if the target predicted part category corresponding to the current endoscopic image is the same as the part category to be retained in the current image. The first prompt message is used to prompt the user to observe for a preset time and retain the image.

[0179] The third acquisition module 706 is used to acquire multiple images captured by the target endoscope as the image set of the current image-to-image site category;

[0180] The second determining module 707 is used to determine the next image retention category in the preset image retention category set as the current image retention category when the total image retention time of the target endoscope acquiring multiple image retention images is greater than the preset time, thereby obtaining the image retention image set of each image retention category in the preset image retention category set.

[0181] The third determining module 708 is used to determine the endoscopy operation monitoring parameters based on the image set of each image category in the preset image category set.

[0182] Optionally, the target prediction site category corresponding to the current endoscopic image is determined based on the current endoscopic image, including:

[0183] The current endoscopic image and multiple endoscopic images preceding the current endoscopic image are defined as multiple time-series images, which are arranged sequentially according to the order of their capture time.

[0184] When multiple time-series images are input into a preset convolutional neural network model, multiple first predicted part categories corresponding to the multiple time-series images are obtained;

[0185] Multiple time-series images and their corresponding multiple first predicted part categories are input into a preset long short-term memory model to obtain the target predicted part category.

[0186] Optionally, endoscopic operation monitoring parameters are determined based on the image sets of each image site category in a preset set of image site categories, including:

[0187] Obtain the target predicted part category of each image in the image set of images to be retained;

[0188] Images whose target predicted part category is the same as the part category to be retained are identified as correctly retained images. The percentage of correctly recognized images in the retained image set is obtained, and the percentage of correctly recognized images in each retained image set is obtained.

[0189] The category of the part to be retained corresponding to the image set with a correct recognition rate greater than the preset rate is determined as the category of successfully recognized part and added to the set of successfully recognized part categories.

[0190] The category of the part to be retained corresponding to the image set whose correctly identified proportion is not greater than the preset proportion is determined as the unidentified part category and added to the unidentified part category set.

[0191] Endoscopic procedure monitoring parameters are determined based on information from the successfully identified site category set and the unidentified site category set.

[0192] Optionally, endoscopic procedure monitoring methods include:

[0193] If the target predicted part category corresponding to the current endoscopic image is different from the part category of the current image to be retained, a second prompt message is issued to prompt the user to move the target endoscope.

[0194] When movement of the target endoscope is detected, acquire the first endoscope image re-acquired by the target endoscope and record the number of acquisitions;

[0195] If the target predicted part category corresponding to the first endoscopic image is different from the part category of the current image to be retained, then determine whether the number of times the images have been collected has reached the preset number.

[0196] If the number of data collections reaches the preset number, the current image part category to be retained is determined as an unidentified part category and placed into the unidentified part category set. The next image part category to be retained in the preset image part category set is determined as the current image part category, thus obtaining the image set of the next image part category to be retained.

[0197] Optionally, endoscopic procedure monitoring methods include:

[0198] If the target predicted region category corresponding to the first endoscopic image is the same as the region category of the current image to be retained, then a first prompt message is issued.

[0199] Optionally, endoscopic procedure monitoring parameters are determined based on information from the successfully identified site category set and information from the unidentified site category set, including:

[0200] Get the number of successfully identified body part categories in the set of successfully identified body part categories;

[0201] Get the number of unidentified body part categories in the set of unidentified body part categories;

[0202] Endoscopic procedure monitoring parameters are determined based on the number of successfully identified site categories and the number of unidentified site categories.

[0203] Optionally, endoscopic procedure monitoring parameters are determined based on the number of successfully identified site categories and the number of unidentified site categories, including:

[0204] Total endoscopic procedure time for obtaining the target endoscope;

[0205] The time difference between two adjacent categories of images to be retained in the preset set of categories of images to be retained is determined as the current category of images to be retained. This determines the duration of image retention for each category of images to be retained, thus obtaining the average duration of image retention for each category of images to be retained in the preset set of categories of images to be retained.

[0206] Endoscopic procedure monitoring parameters are determined based on the total duration of the gastroscopy procedure, the average time spent at each site, the number of successfully identified site categories, and the number of unidentified site categories.

[0207] For details on the implementation of each of the above modules, please refer to the previous examples, which will not be repeated here.

[0208] This application also provides an electronic device, including a memory and a processor, wherein the processor executes the steps in the endoscopic operation monitoring method provided in this embodiment by calling a computer program stored in the memory.

[0209] Please refer to Figure 7 , Figure 7 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application.

[0210] The electronic device may include components such as a processor 101 with one or more processing cores, a memory 102 with one or more computer-readable storage media, a power supply 103, and an input unit 104. Those skilled in the art will understand that the electronic device structure shown in the figures does not constitute a limitation on the electronic device, and may include more or fewer components than shown, or combine certain components, or have different component arrangements. Wherein:

[0211] The processor 101 is the control center of the electronic device, connecting various parts of the device via various interfaces and lines. It executes software programs and / or modules stored in the memory 102, and calls data stored in the memory 102, to perform various functions and process data. Optionally, the processor 101 may include one or more processing cores; alternatively, the processor 101 may integrate an application processor and a modem processor, wherein the application processor mainly handles the operating system, user interface, and applications, while the modem processor mainly handles wireless communication. It is understood that the modem processor may not be integrated into the processor 101.

[0212] The memory 102 can be used to store software programs and modules. The processor 101 executes various functional applications and data processing by running the software programs and modules stored in the memory 102. The memory 102 may mainly include a program storage area and a data storage area. The program storage area may store the operating system, application programs required for at least one function (such as sound playback function, image playback function, etc.), etc.; the data storage area may store data created according to the use of the electronic device, etc. In addition, the memory 102 may include high-speed random access memory, and may also include non-volatile memory, such as at least one disk storage device, flash memory device, or other volatile solid-state storage device. Accordingly, the memory 102 may also include a memory controller to provide the processor 101 with access to the memory 102.

[0213] The electronic device also includes a power supply 103 that supplies power to the various components. Optionally, the power supply 103 can be logically connected to the processor 101 through a power management system, thereby enabling functions such as charging, discharging, and power consumption management through the power management system. The power supply 103 may also include one or more DC or AC power supplies, recharging systems, power fault detection circuits, power converters or inverters, power status indicators, and other arbitrary components.

[0214] The electronic device may also include an input unit 104, which can be used to receive input digital or character information and generate keyboard, mouse, joystick, optical or trackball signal inputs related to user settings and function control.

[0215] Although not shown, the electronic device may also include a display unit, an image acquisition component, etc., which will not be described in detail here. Specifically, in this embodiment, the processor 101 in the electronic device loads one or more executable codes corresponding to computer programs into the memory 102 according to the following instructions, and the processor 101 executes the steps in the endoscopy operation monitoring method provided in this application, such as:

[0216] Obtain the category of the current image region to be retained, where the current image region category is one of the categories in a preset set of image region categories to be retained, and the categories in the preset set are arranged in a preset order; acquire the current endoscopic image acquired by the target endoscope; determine the target predicted region category corresponding to the current endoscopic image based on the current endoscopic image; determine whether the target predicted region category corresponding to the current endoscopic image is the same as the current image region category; if the target predicted region category corresponding to the current endoscopic image is the same as the current image region category, then issue the first request. The system displays information, including a first prompt to the user to observe for a preset duration and capture images; multiple images captured by the target endoscope are acquired as the image set for the current image-capturing site category; when the total image capture time for the multiple images captured by the target endoscope exceeds the preset duration, the next image-capturing site category in the preset image-capturing site category set is determined as the current image-capturing site category, resulting in image sets for each image-capturing site category in the preset image-capturing site category set; and endoscopy operation monitoring parameters are determined based on the image sets for each image-capturing site category in the preset image-capturing site category set.

[0217] It should be noted that the electronic device provided in this application embodiment and the endoscopy operation monitoring method in the above embodiment belong to the same concept. The specific implementation process can be found in the above related embodiments, and will not be repeated here.

[0218] This application also provides a computer-readable storage medium storing a computer program thereon. When the computer program stored thereon is executed on the processor of the electronic device provided in the embodiments of this application, the processor of the electronic device performs the steps in the endoscopy operation monitoring method provided in this application. The storage medium may be a magnetic disk, optical disk, read-only memory (ROM), or random access memory (RAM), etc.

[0219] This application also provides a computer program product or computer program that includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform various optional implementations of the above-described endoscopic operation monitoring method.

[0220] The above provides a detailed description of the endoscopic operation monitoring method and device provided in this application. Specific examples have been used to illustrate the principle and implementation of this application. The description of the above embodiments is only for the purpose of helping to understand the method and core idea of ​​this application. At the same time, for those skilled in the art, there will be changes in the specific implementation and application scope based on the idea of ​​this application. Therefore, the content of this specification should not be construed as a limitation of this application.

[0221] It should be noted that when the above embodiments of this application are applied to specific products or technologies, and user-related data is involved, user permission or consent is required, and the collection, use and processing of related data must comply with the relevant laws, regulations and standards of the relevant countries and regions.

Claims

1. A method for monitoring endoscopic procedures, characterized in that, include: Obtain the category of the current image part to be retained, wherein the current image part category is one of the categories of image parts to be retained in a preset set of image part categories to be retained, and the categories of image parts to be retained in the preset set of image part categories to be retained are arranged in a preset order; Acquire the current endoscopic image captured by the target endoscope; Based on the current endoscopic image, determine the target predicted site category corresponding to the current endoscopic image; Determine whether the target predicted region category corresponding to the current endoscopic image is the same as the region category of the current image to be retained; If the target predicted part category corresponding to the current endoscopic image is the same as the part category to be saved in the current image, a first prompt message is issued. The first prompt message is used to prompt the user to observe for a preset time and save the image. Multiple images captured by the target endoscope are acquired and used as the image set for the current image retention site category; When the total image retention time of the target endoscope acquiring multiple images exceeds a preset time, the next image retention category in the preset image retention category set is determined as the current image retention category, thus obtaining the image retention image set for each image retention category in the preset image retention category set. Endoscopic operation monitoring parameters are determined based on the image sets of each image category in the preset image set to be retained. Specifically, the target predicted region category of each image in the image set of the image category to be retained is obtained; images whose target predicted region category matches the image category to be retained are identified as correctly retained images, and the percentage of correctly recognized images in the image set is obtained, thus obtaining the percentage of correctly recognized images in each image set; image sets with a percentage of correctly recognized images greater than a preset percentage are identified as successfully recognized region categories and placed into a successfully recognized region category set; image sets with a percentage of correctly recognized images less than a preset percentage are identified as unrecognized region categories and placed into an unrecognized region category set; and endoscopic operation monitoring parameters are determined based on the information from the successfully recognized region category set and the unrecognized region category set.

2. The endoscopic operation monitoring method according to claim 1, characterized in that, The step of determining the target predicted site category corresponding to the current endoscopic image based on the current endoscopic image includes: The current endoscopic image and multiple endoscopic images preceding the current endoscopic image are defined as multiple time-series images, wherein the multiple time-series images are arranged sequentially according to the order of their capture time; When multiple time-series images are input into a preset convolutional neural network model, multiple first predicted part categories corresponding to the multiple time-series images are obtained; Multiple time-series images and corresponding multiple first predicted part categories are input into a preset long short-term memory model to obtain the target predicted part category.

3. The endoscopic operation monitoring method according to claim 2, characterized in that, The endoscopic procedure monitoring method includes: If the target predicted part category corresponding to the current endoscopic image is different from the part category of the current image to be retained, a second prompt message is issued, which is used to prompt the user to move the target endoscope; When movement of the target endoscope is detected, the first endoscopic image re-acquired by the target endoscope is acquired and the number of acquisitions is recorded; If the target predicted part category corresponding to the first endoscopic image is different from the part category of the current image to be retained, then it is determined whether the number of times the image has been collected has reached the preset number. If the number of times the images have been collected reaches a preset number, the current image part category to be retained is determined as an unidentified part category and placed into the unidentified part category set. The next image part category to be retained in the preset image part category set is determined as the current image part category to be retained, thus obtaining the image set of the next image part category to be retained.

4. The endoscopic operation monitoring method according to claim 3, characterized in that, The endoscopic procedure monitoring method includes: If the target predicted region category corresponding to the first endoscopic image is the same as the region category of the current image to be retained, a first prompt message is issued.

5. The endoscopic operation monitoring method according to claim 4, characterized in that, The determination of endoscopic operation monitoring parameters based on the information of the successfully identified site category set and the information of the unidentified site category set includes: Obtain the number of successfully identified body part categories in the set of successfully identified body part categories; Obtain the number of unidentified part categories in the set of unidentified part categories; The endoscopic operation monitoring parameters are determined based on the number of successfully identified site categories and the number of unidentified site categories.

6. The endoscopic operation monitoring method according to claim 5, characterized in that, The determination of the endoscopic operation monitoring parameters based on the number of successfully identified site categories and the number of unidentified site categories includes: Obtain the total endoscopic operation time of the target endoscope; The time difference between two adjacent image part categories in the preset image part category set being determined as the current image part category is determined as the part dwell time of the image part category, thus obtaining the average part dwell time of each image part category in the preset image part category set; The endoscopic operation monitoring parameters are determined based on the total duration of the gastroscopy operation, the average time spent at each site, the number of successfully identified site categories, and the number of unidentified site categories.

7. An endoscopic operation monitoring device, characterized in that, include: The first acquisition module is used to acquire the category of the current image part to be retained, wherein the current image part to be retained category is one of the image part categories in the preset set of image part categories to be retained, and the image part categories to be retained in the preset set of image part categories to be retained are arranged in a preset order; The second acquisition module is used to acquire the current endoscopic image collected by the target endoscope; The first determining module is used to determine the target predicted site category corresponding to the current endoscopic image based on the current endoscopic image; The judgment module is used to determine whether the target predicted part category corresponding to the current endoscopic image is the same as the part category of the current image to be retained; The issuing module is used to issue a first prompt message if the target predicted part category corresponding to the current endoscopic image is the same as the part category to be retained in the current image. The first prompt message is used to prompt the user to observe for a preset time and retain the image. The third acquisition module is used to acquire multiple images captured by the target endoscope as the image set of the current image-to-record site category; The second determining module is used to determine the next image retention category in the preset image retention category set as the current image retention category when the total image retention time of the target endoscope acquiring multiple image retention images is greater than a preset time, thereby obtaining the image retention image set of each image retention category in the preset image retention category set. The third determining module is used to determine endoscopic operation monitoring parameters based on the image sets of each image category in the preset image category set. Specifically, it acquires the target predicted region category of each image in the image set of each image category; determines images whose target predicted region category matches the image category of the image to be retained as correctly identified images, obtains the correct recognition percentage of correctly identified images in the image set, and obtains the correct recognition percentage of each image set; determines the image category corresponding to the image set with a correct recognition percentage greater than a preset percentage as a successfully identified region category and places it into a successfully identified region category set; determines the image category corresponding to the image set with a correct recognition percentage not greater than a preset percentage as an unidentified region category and places it into an unidentified region category set; and determines the endoscopic operation monitoring parameters based on the information of the successfully identified region category set and the information of the unidentified region category set.

8. An electronic device, characterized in that, The method includes a memory and a processor, the memory storing a computer program and the processor running the computer program in the memory to perform the steps of the endoscopic operation monitoring method according to any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a plurality of instructions adapted for loading by a processor to perform the steps of the endoscopic operation monitoring method according to any one of claims 1 to 6.

Images