Endoscope operation monitoring method and device

Abstract

The invention discloses an endoscope operation monitoring method and device, and the method comprises the steps: obtaining the type of a current part to be subjected to image reservation; determining a target prediction part category corresponding to the current endoscope image based on the current endoscope image; judging whether the target predicted part category corresponding to the current endoscopic image is the same as the current to-be-reserved image part category or not; if yes, first prompt information is sent out; acquiring a left image set acquired by the target endoscope; when the total image retention time of the target endoscope for collecting the plurality of image retention images is greater than a preset duration, determining the next to-be-retained part category in the preset to-be-retained part category set as the current to-be-retained part category, and obtaining an image retention image set of each to-be-retained part category in the preset to-be-retained part category set; and determining endoscope operation monitoring parameters based on the reserved image set of each to-be-reserved part category in the preset to-be-reserved part category set. The endoscope operation accuracy and efficiency can be improved.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to a method and device for monitoring endoscopic operation. Background Art

[0002] A complete gastroscopy covers a wide range of areas. Doctors are required to observe all areas and take real-time images during the examination. If suspicious lesions are found, further detailed examinations should be performed promptly. Doctors often need a long period of experience to smoothly complete a complete gastroscopy, observe all areas, and produce an accurate diagnosis. For doctors lacking clinical experience, missed examinations and misdiagnoses are common, resulting in low accuracy and efficiency in endoscopic procedures. Summary of the Invention

[0003] The embodiments of the present application provide a method and device for monitoring endoscopic operation, which can improve the accuracy and efficiency of endoscopic operation.

[0004] In a first aspect, the present application provides a method for monitoring endoscopic operation, comprising: Obtaining a current part category to be retained, wherein the current part category to be retained is a part category to be retained in a preset part category set, and the part categories to be retained in the preset part category set are arranged in a preset order; Acquire the current endoscopic image acquired by the target endoscope; determining, based on the current endoscopic image, a target prediction part category corresponding to the current endoscopic image; Determining whether the target predicted part category corresponding to the current endoscopic image is the same as the part category of the current image to be retained; If the target predicted part category corresponding to the current endoscopic image is the same as the current part category to be retained, a first prompt message is issued, wherein the first prompt message is used to prompt the user to observe for a preset time and retain the image; Acquire multiple retained images collected by the target endoscope as a retained image set of the current part category to be imaged; When the total retention time of the target endoscope acquiring the plurality of images is greater than a preset time length, the next part category to be imaged in the preset part category set to be imaged is determined as the current part category to be imaged, and a set of retained images of each part category to be imaged in the preset part category set to be imaged is obtained; Endoscopic operation monitoring parameters are determined based on the image sets of each part category to be retained in the preset part category set to be retained.

[0005] Optionally, determining a target prediction part category corresponding to the current endoscopic image based on the current endoscopic image includes: determining the current endoscopic image and a plurality of endoscopic images preceding the current endoscopic image as a plurality of time-sequence images, wherein the plurality of time-sequence images are arranged in sequence according to the order of shooting time; When a plurality of time-series images are respectively input into a preset convolutional neural network model, a plurality of first predicted part categories corresponding to the plurality of time-series images are obtained; The plurality of time series images and the corresponding plurality of the first predicted part categories are input into a preset long short-term memory model to obtain the target predicted part category.

[0006] Optionally, determining the endoscopic operation monitoring parameters based on the image sets of each part category to be imaged in the preset set of part categories to be imaged includes: Respectively obtaining the target predicted part category of each of the retained images in the retained image set of the part category to be retained; Determine the retained image whose target prediction part category is the same as the part category to be retained as the correctly identified retained image, obtain the correct recognition ratio of the correctly identified retained images in the retained image set, and obtain the correct recognition ratio of each retained image set; Determine the part category to be retained corresponding to the retained image set whose correct recognition ratio is greater than a preset ratio as a successfully recognized part category and put it into a successfully recognized part category set; Determine the part category to be retained corresponding to the retained image set whose correct recognition ratio is not greater than a preset ratio as an unrecognized part category and put it into an unrecognized part category set; Endoscopic operation monitoring parameters are determined based on the information of the successfully identified part category set and the information of the unidentified part category set.

[0007] Optionally, the endoscopic operation monitoring method includes: If the target predicted part category corresponding to the current endoscopic image is different from the part category of the current image to be retained, a second prompt message is issued, wherein the second prompt message is used to prompt the user to move the target endoscope; When movement of the target endoscope is detected, acquiring a first endoscopic image recaptured by the target endoscope and recording the number of acquisitions; If the target predicted part category corresponding to the first endoscopic image is different from the part category of the current image to be retained, determining whether the number of acquisitions has reached a preset number; If the number of acquisitions reaches the preset number, the current part category to be retained is determined as an unidentified part category and placed in an unidentified part category set, and the next part category to be retained in the preset part category set to be retained is determined as the current part category to be retained, to obtain a retained image set of the next part category to be retained.

[0008] Optionally, the endoscopic operation monitoring method includes: If the target predicted part category corresponding to the first endoscopic image is the same as the part category of the current image to be retained, a first prompt message is issued.

[0009] Optionally, determining the endoscopic operation monitoring parameter based on the information of the successfully identified part category set and the information of the unidentified part category set includes: Obtaining the number of successfully identified part categories in the successfully identified part category set; Obtaining the number of unidentified part categories in the unidentified part category set; The endoscopic operation monitoring parameter is determined based on the number of successfully identified site categories and the number of unidentified site categories.

[0010] Optionally, determining the endoscopic operation monitoring parameter based on the number of successfully identified part categories and the number of unidentified part categories includes: Obtaining the total duration of the gastroscopic operation of the target endoscope; Determine the time difference between two adjacent parts of the preset part category set to be retained and determined as the current part category to be retained as the part stay time of the part category to be retained, and obtain the average part stay time of each part category to be retained in the preset part category set; The endoscopic operation monitoring parameters are determined based on the total duration of the gastroscopic operation, the average site residence time, the number of successfully identified site categories, and the number of unidentified site categories.

[0011] In a second aspect, the present application provides an endoscope operation monitoring device, comprising: A first acquisition module is configured to acquire a current part category to be retained, wherein the current part category to be retained is a part category to be retained in a preset part category set, and the part categories to be retained in the preset part category set are arranged in a preset order; A second acquisition module is used to acquire the current endoscopic image acquired by the target endoscope; A first determination module is configured to determine, based on the current endoscopic image, a target prediction part category corresponding to the current endoscopic image; A judgment module, configured to judge whether the target predicted part category corresponding to the current endoscopic image is the same as the part category of the current image to be retained; an issuing module, configured to issue a first prompt message if the target predicted part category corresponding to the current endoscopic image is the same as the current part category to be retained, wherein the first prompt message is used to prompt the user to observe for a preset time period and retain the image; A third acquisition module is used to acquire a plurality of retained images collected by the target endoscope as a retained image set of the current part category to be imaged; A second determining module is configured to, when the total retention time of the plurality of images acquired by the target endoscope is greater than a preset time length, determine the next part category to be imaged in the preset part category set as the current part category to be imaged, and obtain a set of retained images of each part category to be imaged in the preset part category set; The third determining module is configured to determine endoscopic operation monitoring parameters based on the image sets of each part category to be imaged in the preset set of part categories to be imaged.

[0012] Optionally, determining a target prediction part category corresponding to the current endoscopic image based on the current endoscopic image includes: determining the current endoscopic image and a plurality of endoscopic images preceding the current endoscopic image as a plurality of time-sequence images, wherein the plurality of time-sequence images are arranged in sequence according to the order of shooting time; When a plurality of time-series images are respectively input into a preset convolutional neural network model, a plurality of first predicted part categories corresponding to the plurality of time-series images are obtained; The plurality of time series images and the corresponding plurality of the first predicted part categories are input into a preset long short-term memory model to obtain the target predicted part category.

[0013] Optionally, determining the endoscopic operation monitoring parameters based on the image sets of each part category to be imaged in the preset set of part categories to be imaged includes: Respectively obtaining the target predicted part category of each of the retained images in the retained image set of the part category to be retained; Determine the retained image whose target prediction part category is the same as the part category to be retained as the correctly identified retained image, obtain the correct recognition ratio of the correctly identified retained images in the retained image set, and obtain the correct recognition ratio of each retained image set; Determine the part category to be retained corresponding to the retained image set whose correct recognition ratio is greater than a preset ratio as a successfully recognized part category and put it into a successfully recognized part category set; Determine the part category to be retained corresponding to the retained image set whose correct recognition ratio is not greater than a preset ratio as an unrecognized part category and put it into an unrecognized part category set; Endoscopic operation monitoring parameters are determined based on the information of the successfully identified part category set and the information of the unidentified part category set.

[0014] Optionally, the endoscopic operation monitoring method includes: If the target predicted part category corresponding to the current endoscopic image is different from the part category of the current image to be retained, a second prompt message is issued, wherein the second prompt message is used to prompt the user to move the target endoscope; When movement of the target endoscope is detected, acquiring a first endoscopic image recaptured by the target endoscope and recording the number of acquisitions; If the target predicted part category corresponding to the first endoscopic image is different from the part category of the current image to be retained, determining whether the number of acquisitions has reached a preset number; If the number of acquisitions reaches the preset number, the current part category to be retained is determined as an unidentified part category and placed in an unidentified part category set, and the next part category to be retained in the preset part category set to be retained is determined as the current part category to be retained, to obtain a retained image set of the next part category to be retained.

[0015] Optionally, the endoscopic operation monitoring method includes: If the target predicted part category corresponding to the first endoscopic image is the same as the part category of the current image to be retained, a first prompt message is issued.

[0016] Optionally, determining the endoscopic operation monitoring parameter based on the information of the successfully identified part category set and the information of the unidentified part category set includes: Obtaining the number of successfully identified part categories in the successfully identified part category set; Obtaining the number of unidentified part categories in the unidentified part category set; The endoscopic operation monitoring parameter is determined based on the number of successfully identified site categories and the number of unidentified site categories.

[0017] Optionally, determining the endoscopic operation monitoring parameter based on the number of successfully identified part categories and the number of unidentified part categories includes: Obtaining the total duration of the gastroscopic operation of the target endoscope; Determine the time difference between two adjacent parts of the preset part category set to be retained and determined as the current part category to be retained as the part stay time of the part category to be retained, and obtain the average part stay time of each part category to be retained in the preset part category set; The endoscopic operation monitoring parameters are determined based on the total duration of the gastroscopic operation, the average site residence time, the number of successfully identified site categories, and the number of unidentified site categories.

[0018] On the third aspect, the electronic device provided in the present application includes a memory and a processor, the memory stores a computer program, and the processor is used to run the computer program in the memory to implement the steps in the endoscopic operation monitoring method provided in the present application.

[0019] In a fourth aspect, the computer-readable storage medium provided in the present application stores a plurality of instructions, which are suitable for loading by a processor to implement the steps in the endoscopic operation monitoring method provided in the present application.

[0020] In a fifth aspect, the computer program product provided in the present application includes a computer program or instructions, which, when executed by a processor, implement the steps in the endoscopic operation monitoring method provided in the present application.

[0021] In the present application, compared with the related art, the current part category to be retained is obtained, wherein the current part category to be retained is a part category to be retained in a preset part category set, and each part category to be retained in the preset part category set is arranged in a preset order; the current endoscopic image acquired by the target endoscope is obtained; based on the current endoscopic image, the target predicted part category corresponding to the current endoscopic image is determined; it is judged whether the target predicted part category corresponding to the current endoscopic image is the same as the current part category to be retained; if the target predicted part category corresponding to the current endoscopic image is the same as the current part category to be retained , a first prompt message is issued, and the first prompt message is used to prompt the user to observe for a preset time and leave an image; obtain multiple images collected by the target endoscope as the image set of the current part category to be left; when the total image retention time of the target endoscope collecting multiple images is greater than the preset time, the next part category to be left in the preset part category set is determined as the current part category to be left, and the image set of each part category to be left in the preset part category set is obtained; the endoscope operation monitoring parameters are determined based on the image set of each part category to be left in the preset part category set. During the endoscopic operation, the present application guides the user to leave an image at the part to be left one by one according to the operation part in a preset order, and prompts the user to perform the operation when the monitored image is consistent with the part to be left, and can determine the endoscopic operation monitoring parameters based on the image information collected by the user to make an accurate evaluation of the operation, which can improve the accuracy and efficiency of the endoscopic operation. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For those skilled in the art, other drawings can be obtained based on these drawings without creative work.

[0023] Figure 1 is a schematic diagram of a scenario of an endoscope operation monitoring system provided in an embodiment of the present application; Figure 2 This is a flow chart of an embodiment of the method for monitoring endoscopic operation provided by the embodiment of the present application; Figure 3 and Figure 4 Schematic diagram of 26 image retention categories of parts to be imaged in one embodiment of the endoscopic operation monitoring method provided in an embodiment of the present application; Figure 5 Schematic diagram of an LSTM model in an embodiment of the endoscopic operation monitoring method provided in an embodiment of the present application; Figure 6 Schematic diagram of the structure of the endoscope operation monitoring device provided in an embodiment of the present application; Figure 7 It is a structural diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0024] It should be noted that the principles of this application are illustrated by implementing them in an appropriate computing environment. The following description is based on the illustrated specific embodiments of this application and should not be considered as limiting other specific embodiments not described in detail herein.

[0025] In the following description of this application, reference is made to “some embodiments”, which describe a subset of all possible embodiments. However, it can be understood that “some embodiments” may be the same subset or different subsets of all possible embodiments, and may be combined with each other without conflict.

[0026] In the following description of this application, the terms "first\second\third" involved are merely used to distinguish similar objects and do not represent a specific ordering of the objects. It can be understood that "first\second\third" can be interchanged with a specific order or sequence where permitted, so that the embodiments of the present application described herein can be implemented in an order other than that illustrated or described herein.

[0027] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by those skilled in the art to which this application pertains. The terms used herein are for the purpose of describing the embodiments of this application only and are not intended to limit this application.

[0028] To improve the effectiveness of endoscopic operation monitoring, embodiments of the present application provide an endoscopic operation monitoring method, an endoscopic operation monitoring device, an electronic device, a computer-readable storage medium, and a computer program product. The endoscopic operation monitoring method can be executed by the endoscopic operation monitoring device, or by an electronic device incorporating the endoscopic operation monitoring device.

[0029] The following will be combined with the drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are only part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without making creative efforts are within the scope of protection of this application.

[0030] Please refer to Figure 1 , this application also provides an endoscope operation monitoring system, such as Figure 1 As shown, the endoscope operation monitoring system includes an electronic device 100 and a target endoscope 200. The electronic device 100 is integrated with the endoscope operation monitoring device provided by the present application. The electronic device 100 is connected to the target endoscope 200. The target endoscope 200 can be a gastroscope, colonoscope, etc.

[0031] Among them, the electronic device 100 can be any device equipped with a processor and having processing capabilities, such as mobile electronic devices with processors such as smart phones, tablet computers, PDAs, laptops, smart speakers, or fixed electronic devices with processors such as desktop computers, televisions, servers, industrial equipment, etc.

[0032] In addition, if Figure 1 As shown, the endoscope operation monitoring system may further include a memory for storing raw data, intermediate data, and result data.

[0033] In the embodiments of the present application, the memory may be a cloud memory. Cloud storage is a new concept extended and developed from the concept of cloud computing. A distributed cloud storage system (hereinafter referred to as the storage system) refers to a storage system that uses cluster applications, grid technology, and distributed storage file systems to bring together a large number of different types of storage devices (storage devices are also called storage nodes) in the network through application software or application interfaces to work together and provide external data storage and business access functions.

[0034] Currently, storage systems utilize a storage method that creates logical volumes. During the creation of a logical volume, physical storage space is allocated for each logical volume. This physical storage space may consist of disks on a storage device or several storage devices. When a client stores data on a logical volume, it stores the data on a file system. The file system divides the data into multiple parts, each of which is an object. An object contains not only the data but also additional information such as the data identifier (ID). The file system writes each object to the physical storage space of the logical volume and records the storage location of each object. Therefore, when a client requests data access, the file system can provide access based on the storage location of each object.

[0035] The storage system allocates physical storage space to logical volumes by pre-dividing the physical storage space into stripes based on the estimated capacity of the objects to be stored in the logical volume (this estimate often has a large margin relative to the actual capacity of the objects to be stored) and the Redundant Array of Independent Disks (RAID) groupings. A logical volume can be understood as a stripe, thereby allocating physical storage space to the logical volume.

[0036] It should be noted that Figure 1 The scenario diagram of the endoscopic operation monitoring system shown is merely an example. The endoscopic operation monitoring system and scenario described in the embodiment of the present application are intended to more clearly illustrate the technical solution of the embodiment of the present application, and do not constitute a limitation on the technical solution provided in the embodiment of the present application. A person of ordinary skill in the art will appreciate that, with the evolution of the endoscopic operation monitoring system and the emergence of new business scenarios, the technical solution provided in the embodiment of the present application is equally applicable to similar technical problems.

[0037] It should be noted that the serial numbers of the following embodiments are not intended to limit the preferred order of the embodiments.

[0038] Please refer to Figure 2 , Figure 2 FIG. 1 is a flow chart of an embodiment of the method for monitoring endoscopic operation provided in the embodiment of the present application. Figure 2 As shown, the process of the endoscopic operation monitoring method provided in this application is as follows: 201. Obtain the category of the part of the current image to be retained.

[0039] Gastroscopy is a medical examination method and also refers to the instrument used for this examination. It uses a thin, flexible tube to be inserted into the stomach, allowing the doctor to directly observe lesions, especially tiny lesions, in the esophagus, stomach and duodenum under direct vision.

[0040] Artificial intelligence (AI) is the study of using computers to simulate certain human thought processes and intelligent behaviors (such as learning, reasoning, thinking, and planning). It primarily encompasses the principles of computer intelligence, the creation of computers with intelligence similar to that of the human brain, and the realization of higher-level computer applications. AI involves disciplines such as computer science, psychology, philosophy, and linguistics. It encompasses virtually all disciplines in the natural and social sciences, extending far beyond the confines of computer science. The relationship between AI and the science of thinking is a practical one, as AI lies at the technological application level of the science of thinking and is an applied branch of it. From a cognitive perspective, AI is not limited to logical thinking; it must also consider visual and inspirational thinking to promote breakthroughs in AI. Mathematics is often considered a foundational science across multiple disciplines, permeating areas such as language and thinking. AI, therefore, must also draw upon mathematical tools. Mathematics plays a role not only in standard logic and fuzzy mathematics, but also in mutually reinforcing and accelerating AI development.

[0041] Deep learning (DL) is a new research direction within the field of machine learning (ML). It was introduced to bring ML closer to its original goal: artificial intelligence (AI). Deep learning involves learning the inherent patterns and representational hierarchies of sample data. The information gained during this learning process significantly aids in interpreting data such as text, images, and sound. Its ultimate goal is to enable machines to possess human-like analytical and learning capabilities, enabling them to recognize data such as text, images, and sound. Deep learning is a complex machine learning algorithm that has achieved results in speech and image recognition that far surpass previous technologies. Deep learning has achieved significant success in search technology, data mining, machine learning, machine translation, natural language processing, multimedia learning, speech recognition, recommendation and personalization technologies, and other related fields. Deep learning enables machines to mimic human activities such as seeing, hearing, and thinking, solving many complex pattern recognition challenges and significantly advancing AI-related technologies.

[0042] The current part category to be retained is a part category to be retained in a preset part category set, and the part categories to be retained in the preset part category set are arranged in a preset order.

[0043] In the embodiment of the present application, the endoscope device is started, wherein the endoscope device can be a gastroscope device, a colonoscope device, etc., which can be set according to the specific situation. The endoscope host is turned on, the cleaned target endoscope is connected to the endoscope host, the image is ensured to be normal, and the parameters of the endoscope host are debugged.

[0044] For example, if the endoscope is a gastroscope, start the gastroscope, turn on the electronic device, connect the cleaned gastroscope to the electronic device, ensure that the image is normal, and complete the electronic device parameter debugging. Turn on the electronic device, connect the cleaned gastroscope to the electronic device, ensure that the image is normal, and complete the electronic device parameter debugging.

[0045] For example, the endoscope device is a colonoscope. The colonoscope device is started, the electronic device is turned on, and the cleaned colonoscope is connected to the electronic device. Ensure that the display is normal and the electronic device parameters are debugged. The electronic device is turned on, and the cleaned colonoscope is connected to the electronic device. Ensure that the display is normal and the electronic device parameters are debugged.

[0046] Start the endoscope operation monitoring system. Connect the electronic device input terminal to the target endoscope signal output terminal and confirm the signal connection is successful. When the electronic device and the target endoscope are successfully connected, the system prompts the doctor to begin the endoscope operation.

[0047] The parts categories to be retained in the preset set of parts categories to be retained are arranged in a preset order. For example, the preset set of categories of parts to be retained in the images includes 26 categories of parts to be retained in the images, and the preset order of the 26 categories of parts to be retained in the images is: esophagus, cardia, greater curvature of gastric antrum, posterior wall of gastric antrum, anterior wall of gastric antrum, lesser curvature of gastric antrum, duodenal bulb, descending duodenum, greater curvature of lower part of gastric body, posterior wall of lower part of gastric body, anterior wall of lower part of gastric body, lesser curvature of lower part of gastric body, greater curvature of upper and middle part of gastric body, posterior wall of upper and middle part of gastric body, anterior wall of upper and middle part of gastric body, lesser curvature of upper and middle part of gastric body, greater curvature of inverted gastric fund, posterior wall of inverted gastric fund, anterior wall of inverted gastric fund, lesser curvature of inverted gastric fund, posterior wall of upper and middle part of gastric body, anterior wall of upper and middle part of gastric body, lesser curvature of upper and middle part of gastric body, posterior wall of inverted gastric angle, anterior wall of inverted gastric angle, lesser curvature of inverted gastric angle.

[0048] like Figure 3 and Figure 4 As shown, Figure 3 and Figure 4 The images of 26 categories of parts to be retained are shown. Figure 3 The images retained in the image are: esophagus, cardia, greater curvature of gastric antrum, posterior wall of gastric antrum, anterior wall of gastric antrum, lesser curvature of gastric antrum, duodenal bulb, descending duodenum, greater curvature of lower gastric body, posterior wall of lower gastric body, anterior wall of lower gastric body, lesser curvature of lower gastric body. Figure 4 The images retained in the image are: the greater curvature of the middle and upper part of the gastric body in the positive mirror, the posterior wall of the middle and upper part of the gastric body in the positive mirror, the anterior wall of the middle and upper part of the gastric body in the positive mirror, the lesser curvature of the middle and upper part of the gastric body in the positive mirror, the greater curvature of the gastric fundus in the inverted mirror, the posterior wall of the gastric fundus in the inverted mirror, the anterior wall of the gastric fundus in the inverted mirror, the lesser curvature of the gastric fundus in the inverted mirror, the posterior wall of the middle and upper part of the gastric body in the inverted mirror, the anterior wall of the middle and upper part of the gastric body in the inverted mirror, the lesser curvature of the middle and upper part of the gastric body in the inverted mirror, the posterior wall of the gastric angle in the inverted mirror, the anterior wall of the gastric angle in the inverted mirror, and the lesser curvature of the gastric angle in the inverted mirror.

[0049] For example, the current category of the part to be retained is esophagus.

[0050] 202. Acquire a current endoscopic image acquired by a target endoscope.

[0051] In the embodiment of the present application, the target endoscope automatically captures endoscopic images according to a preset period after the user's operation to obtain the current endoscopic image captured by the target endoscope. The preset period can be 1s, 2s, etc., which can be set according to the specific situation.

[0052] 203. Determine a target prediction part category corresponding to the current endoscopic image based on the current endoscopic image.

[0053] In the embodiment of the present application, determining the target prediction part category corresponding to the current endoscopic image based on the current endoscopic image includes: (1) A current endoscopic image and a plurality of endoscopic images preceding the current endoscopic image are determined as a plurality of time-series images, wherein the plurality of time-series images are arranged in sequence according to the chronological order of shooting time.

[0054] For example, N time-series images are acquired.

[0055] (2) When multiple time series images are respectively input into the preset convolutional neural network model, multiple first predicted part categories corresponding to the multiple time series images are obtained.

[0056] In the embodiment of the present application, the preset convolutional neural network model is a CNN model. The CNN model identifies the corresponding typical parts in the input single time-series image based on the weights corresponding to the 26 typical parts of the stomach, thereby performing part classification on the single time-series image to obtain a first predicted part category of the time-series image, and obtains N time-series images and the corresponding first predicted part categories.

[0057] (3) Inputting a plurality of time series images and a corresponding plurality of first predicted part categories into a preset long-short memory model to obtain a target predicted part category.

[0058] The preset long short-term memory model is the LSTM model. The LSTM model outputs the part category of the last image in N consecutive images, that is, the LSTM model outputs the target predicted part category of the current endoscopic image. The network structure of the LSTM model is as follows: Figure 5 shown.

[0059] Furthermore, the first predicted part category of the current endoscopic image output by the preset convolutional neural network model is obtained, and the second predicted part category of the current endoscopic image output by the preset long-short memory model is obtained. If the first predicted part category of the current endoscopic image and the second predicted part category of the current endoscopic image are the same, the first predicted part category of the current endoscopic image is determined as the target predicted part category of the current endoscopic image; if the first predicted part category of the current endoscopic image and the second predicted part category of the current endoscopic image are not the same, the unrecognizable category is determined as the target predicted part category of the current endoscopic image.

[0060] 204. Determine whether the target predicted part category corresponding to the current endoscopic image is the same as the part category of the current image to be retained.

[0061] 205. If the target predicted part category corresponding to the current endoscopic image is the same as the current part category to be retained, a first prompt message is issued, where the first prompt message is used to prompt the user to observe for a preset time and retain the image.

[0062] The preset duration may be 5 seconds, 6 seconds, etc., and may be determined according to the specific situation.

[0063] In this embodiment of the present application, if the target predicted part category corresponding to the current endoscopic image is the same as the current part category to be imaged, a first prompt message is issued, prompting the user to observe for a preset time period and retain the image. After receiving the first prompt message, the user operates the target endoscope to retain the image, obtaining multiple retained images.

[0064] Furthermore, the endoscopic operation monitoring method includes: (1) If the target predicted part category corresponding to the current endoscopic image is different from the part category of the current image to be retained, a second prompt message is issued, and the second prompt message is used to prompt the user to move the target endoscope.

[0065] (2) When the target endoscope is detected to be moving, the first endoscopic image recaptured by the target endoscope is acquired and the number of acquisitions is recorded.

[0066] (3) If the target predicted part category corresponding to the first endoscopic image is different from the part category of the current image to be retained, it is determined whether the number of acquisitions has reached the preset number.

[0067] Among them, the preset number of times is set according to specific circumstances.

[0068] (4) If the number of acquisitions reaches the preset number, the current part category to be retained is determined as an unidentified part category and placed in the unidentified part category set, and the next part category to be retained in the preset part category set is determined as the current part category to be retained, thereby obtaining the image set of the next part category to be retained.

[0069] If the number of acquisitions reaches the preset number, it indicates that there may be a user error and a matching image cannot be obtained, so the image of the next part is acquired.

[0070] Furthermore, if the target predicted part category corresponding to the first endoscopic image is the same as the part category of the current image to be retained, a first prompt message is issued.

[0071] 206. Acquire multiple retained images collected by the target endoscope as a retained image set for the current category of the part to be imaged.

[0072] In the embodiment of the present application, multiple retained images are obtained by the user operating the target endoscope and retaining the images.

[0073] 207. When the total retention time of multiple images collected by the target endoscope is greater than a preset time length, the next part category to be imaged in the preset part category set to be imaged is determined as the current part category to be imaged, and a set of retained images of each part category to be imaged in the preset part category set to be imaged is obtained.

[0074] In an embodiment of the present application, when the total retention time of multiple images collected by the target endoscope is greater than 5s, it indicates that the user continues to operate at the current part and remains at the current part for more than 5 seconds, prompting the user that the inspection of the current part is completed and to proceed to the next part in sequence.

[0075] In an embodiment of the present application, each part category to be retained in the preset part category set is determined as the current part category to be retained, and a set of retained images of each part category to be retained in the preset part category set is obtained.

[0076] 208. Determine endoscopic operation monitoring parameters based on the image sets of each part category to be imaged in the preset set of part categories to be imaged.

[0077] In an embodiment of the present application, after obtaining the image sets of each part category to be imaged in the preset part category set, the endoscopic operation monitoring parameters are determined based on the image sets of each part category to be imaged in the preset part category set.

[0078] In an embodiment of the present application, determining the endoscopic operation monitoring parameters based on the image sets of each part category to be retained in the preset part category set includes: (1) Obtain the target predicted part category of each retained image in the retained image set of the part category to be retained.

[0079] (2) The retained images whose target predicted part category is the same as the part category to be retained are determined as correctly identified retained images, and the correct recognition ratio of correctly identified retained images in the retained image set is obtained, and the correct recognition ratio of each retained image set is obtained.

[0080] In the embodiment of the present application, the target predicted part category of the retained image is the same as the part category to be retained, indicating that the correct part is observed and the retained image is determined to be the identified correct retained image.

[0081] (3) The part categories to be retained corresponding to the retained image set whose correct recognition ratio is greater than the preset ratio are determined as the successful recognition part categories and put into the successful recognition part category set.

[0082] Among them, the preset proportion can be 80%, 90%, etc., depending on the specific settings, and this application does not limit this.

[0083] (4) The part categories to be retained corresponding to the retained image set whose correct recognition ratio is not greater than the preset ratio are determined as unrecognized part categories and placed in the unrecognized part category set.

[0084] (5) Determine endoscopic operation monitoring parameters based on the information of the successfully identified part category set and the information of the unidentified part category set.

[0085] In the embodiment of the present application, the endoscopic operation monitoring parameters are determined based on the information of the successfully identified part category set and the information of the unidentified part category set, including: (1) Obtain the number of successfully identified part categories in the successfully identified part category set.

[0086] (2) Obtain the number of unidentified part categories in the unidentified part category set.

[0087] (3) Determine the endoscopic operation monitoring parameters based on the number of successfully identified part categories and the number of unidentified part categories.

[0088] In a specific embodiment, the number of successfully identified part categories and the number of unidentified part categories are weighted and summed based on preset weights to obtain the endoscope operation monitoring parameter.

[0089] In another specific embodiment, determining an endoscopic operation monitoring parameter based on the number of successfully identified part categories and the number of unidentified part categories includes: (1) Obtain the total duration of the gastroscopic operation of the target endoscope.

[0090] In an embodiment of the present application, timing starts when it is detected that the target endoscope enters the human body, and stops when it is detected that the target endoscope leaves the human body. The timing time at this time is determined as the total duration of the gastroscopic operation of the target endoscope.

[0091] (2) The time difference between two adjacent part categories to be retained in the preset part category set is determined as the current part category to be retained as the part stay time of the part category to be retained, and the average part stay time of each part category to be retained in the preset part category set is obtained.

[0092] The time difference between two adjacent part categories to be retained and determined as the current part category to be retained can represent the stay time of the part of the previous part category to be retained.

[0093] (3) Endoscopic operation monitoring parameters were determined based on the total duration of the gastroscopic operation, the average site dwell time, the number of successfully identified site categories, and the number of unidentified site categories.

[0094] In a specific embodiment, the endoscopic operation monitoring parameter is determined based on the total duration of the gastroscopic operation, the average site dwell time, the number of successfully identified site categories, and the number of unidentified site categories based on preset weight coefficients. The longer the total duration of the gastroscopic operation, the lower the endoscopic operation monitoring parameter; the longer the average site dwell time, the lower the endoscopic operation monitoring parameter; the fewer the number of successfully identified site categories, the lower the endoscopic operation monitoring parameter; and the greater the number of unidentified site categories, the lower the endoscopic operation monitoring parameter.

[0095] Furthermore, after determining the endoscopic operation monitoring parameters, when the gastroscope device leaves the body, the system will prompt the end of the gastroscope and provide an artificial intelligence analysis report based on the user's actual situation during the gastroscope operation. The artificial intelligence analysis report includes the following information:

[0096] Furthermore, to improve the accuracy of endoscopic operation monitoring parameters, endoscopic operation monitoring parameters are determined based on the total duration of the gastroscopic operation, the average site dwell time, the number of successfully identified site categories, and the number of unidentified site categories. This includes: inputting each retained image in the retained image set into a preset clarity detection model to obtain the clarity of each retained image; determining the average clarity of the retained images in the retained image set as the average clarity of the retained image set; determining the to-be-imaged site categories corresponding to the retained image set whose clarity average is greater than the preset clarity as clear retained site categories, and obtaining the number of clear retained site categories. The endoscopic operation monitoring parameters are determined based on the total duration of the gastroscopic operation, the average site dwell time, the number of successfully identified site categories, the number of clear retained site categories, and the number of unidentified site categories.

[0097] Specifically, the endoscopic operation monitoring parameter is obtained by weighting the total duration of the gastroscopic operation, the average site dwell time, the number of successfully identified site categories, the number of clearly imaged site categories, and the number of unidentified site categories based on preset weight coefficients. The longer the total duration of the gastroscopic operation, the lower the endoscopic operation monitoring parameter; the longer the average site dwell time, the lower the endoscopic operation monitoring parameter; the smaller the number of successfully identified site categories, the lower the endoscopic operation monitoring parameter; the greater the number of unidentified site categories, the lower the endoscopic operation monitoring parameter; and the smaller the number of clearly imaged site categories, the lower the endoscopic operation monitoring parameter.

[0098] For example, suppose the weight coefficients for the total gastroscopic procedure duration, average site dwell time, number of successfully identified site categories, number of clearly imaged site categories, and number of unidentified site categories are set to -0.3, -0.2, 0.2, 0.15, and 0.15, respectively. In a given gastroscopic procedure, the total procedure duration is 30 minutes (corresponding to a weight coefficient of -0.3, calculated as -30 × 0.3 = -9), the average site dwell time is 2 minutes (corresponding to a weight coefficient of -0.2, calculated as -2 × 0.2 = -0.4), the number of successfully identified site categories is 5 (corresponding to a weight coefficient of 0.2, calculated as 5 × 0.2 = 1), the number of clearly imaged site categories is 4 (corresponding to a weight coefficient of 0.15, calculated as 4 × 0.15 = 0.6), and the number of unidentified site categories is 2 (corresponding to a weight coefficient of 0.15, calculated as 2 × 0.15 = 0.3). Adding these results, i.e., -9 + (-0.4) +1 + 0.6+ 0.3 = -7.5, the monitoring parameter for this endoscopic operation is obtained as -7.5.

[0099] Furthermore, in order to improve the accuracy of endoscopic operation monitoring parameters, endoscopic operation monitoring parameters are determined based on the total duration of gastroscopic operation, average site residence time, the number of successfully identified site categories, the number of clear site categories, and the number of unidentified site categories, including: inputting each retained image in the retained image set into a preset shooting angle detection model to obtain the shooting angle of each retained image; calculating the variance of the shooting angle of the retained image in the retained image set to obtain the angle variance corresponding to each retained image set, determining the to-be-imaged site category corresponding to the retained image set whose angle variance is greater than the preset variance value as the angle-qualified retained site category, and obtaining the number of angle-qualified retained site categories.

[0100] Specifically, the endoscopic operation monitoring parameter is obtained by weighting and summing the total duration of the gastroscopic operation, the average site dwell time, the number of successfully identified site categories, the number of site categories with qualified angles, the number of clear site categories with clear images, and the number of unidentified site categories based on preset weight coefficients. The longer the total duration of the gastroscopic operation, the lower the endoscopic operation monitoring parameter; the longer the average site dwell time, the lower the endoscopic operation monitoring parameter; the smaller the number of successfully identified site categories, the lower the endoscopic operation monitoring parameter; the greater the number of unidentified site categories, the lower the endoscopic operation monitoring parameter; the smaller the number of clear site categories with clear images, the lower the endoscopic operation monitoring parameter; and the smaller the number of site categories with qualified angles with clear images, the lower the endoscopic operation monitoring parameter.

[0101] Assume that the weight coefficients for the total gastroscopic procedure duration, average site dwell time, number of successfully identified site categories, number of site categories with acceptable angles, number of clear site categories, and number of unidentified site categories are -0.2, -0.15, -0.15, -0.1, 0.15, and -0.2, respectively. In a given gastroscopic procedure, the total procedure duration is 25 minutes (calculated as 25 × -0.2 = -5), the average site dwell time is 1.8 minutes (calculated as 1.8 × -0.15 = -0.27), the number of successfully identified site categories is 6 (calculated as 6 × 0.15 = 0.9), the number of site categories with acceptable angles is 3 (calculated as 3 × 0.1 = 0.3), the number of clear site categories is 4 (calculated as 4 × 0.15 = 0.6), and the number of unidentified site categories is 3 (calculated as 3 × 0.2 = 0.6). Adding these results, -5 + (-0.27) +0.9 + 0.3 + 0.6 + (- 0.6) = -4.07, we obtain the monitoring parameter of this endoscopic operation as -4.07.

[0102] Furthermore, when the endoscope operation monitoring parameter is lower than the preset parameter value, a third prompt message is issued to remind the user that the operation score is low. The preset value can be -4, which can be set according to the specific time.

[0103] This application identifies the current screen part by intercepting the gastroscope screen picture played in real time in the gastroscope image. It includes a convolutional neural network model and a long-short time memory network model trained by the backpropagation algorithm. The convolutional neural network model is used to classify the parts and lesion features of the pre-processed image. The long-short time memory network model performs sequence fitting on the classification results to obtain the recognition result, and guides the doctor to observe the current part carefully before entering the next part for observation until the observation of 26 parts is completed, thereby effectively assisting the endoscopist, improving his gastroscope operation level, reducing the missed diagnosis rate of gastric diseases and the patient's risk of cancer. The present invention performs image quality recognition, part recognition and part feature recognition on the collected images and displays them on the client, providing the operator with a more reliable reference basis, improving the accuracy and effectiveness of the detection, being simple and easy to use, and avoiding the patient's secondary pain and increasing additional medical expenses due to an inadequate examination.

[0104] This application solves the problems of complex gastroscopy procedures, high requirements for physician skills, and the proneness to image blind spots and missed lesion diagnosis. It provides physicians with accurate and reliable reference, improves the accuracy and effectiveness of detection, is simple and easy to use, and has significant social and economic value.

[0105] This application can improve the standardization of doctors' operations. If beginners perform gastroscopy without the guidance of senior doctors, it is unsafe and unethical. However, the teaching of each clinical physician is individual. The addition of a gastroscopy-assisted teaching system can standardize both teachers and learners. Only through repeated training under the premise of standardization can doctors go a long way and patients will also benefit.

[0106] To facilitate better implementation of the endoscopic operation monitoring method provided in the embodiments of the present application, the embodiments of the present application also provide an endoscopic operation monitoring device based on the above endoscopic operation monitoring method. The meanings of the terms herein are the same as those in the above endoscopic operation monitoring method. For specific implementation details, please refer to the description in the above method embodiments.

[0107] Please refer to Figure 6 , Figure 6 This is a schematic diagram of the structure of an endoscope operation monitoring device provided in an embodiment of the present application. The endoscope operation monitoring device may include: The first acquisition module 701 is used to obtain a current part category to be retained, wherein the current part category to be retained is a part category to be retained in a preset part category set, and the part categories to be retained in the preset part category set are arranged in a preset order; The second acquisition module 702 is used to acquire the current endoscopic image acquired by the target endoscope; A first determination module 703 is configured to determine, based on the current endoscopic image, a target prediction part category corresponding to the current endoscopic image; A judgment module 704 is used to judge whether the target predicted part category corresponding to the current endoscopic image is the same as the part category of the current image to be retained; The issuing module 705 is configured to issue a first prompt message if the target predicted part category corresponding to the current endoscopic image is the same as the part category of the current part to be retained. The first prompt message is used to prompt the user to observe for a preset time and retain the image; The third acquisition module 706 is used to acquire multiple images captured by the target endoscope as the image set of the current part category to be imaged; The second determining module 707 is configured to, when the total retention time of the multiple images acquired by the target endoscope is greater than a preset retention time, determine the next part category to be imaged in the preset part category set as the current part category to be imaged, and obtain a set of retained images for each part category to be imaged in the preset part category set; The third determining module 708 is configured to determine endoscopic operation monitoring parameters based on the image sets of each part category to be imaged in the preset part category set to be imaged.

[0108] Optionally, determining a target prediction part category corresponding to the current endoscopic image based on the current endoscopic image includes: determining a current endoscopic image and a plurality of endoscopic images preceding the current endoscopic image as a plurality of time-sequence images, wherein the plurality of time-sequence images are arranged in sequence according to the order of shooting time; When the multiple time series images are respectively input into the preset convolutional neural network model, a plurality of first predicted part categories corresponding to the multiple time series images are obtained; The plurality of time series images and the corresponding plurality of first predicted part categories are input into a preset long short-term memory model to obtain a target predicted part category.

[0109] Optionally, determining the endoscopic operation monitoring parameters based on the image sets of each part category to be imaged in the preset set of part categories to be imaged includes: Respectively obtain the target predicted part category of each retained image in the retained image set of the part category to be retained; The retained images whose target predicted part category is the same as the part category to be retained are determined as correctly identified retained images, and the correct recognition ratio of the correctly identified retained images in the retained image set is obtained, and the correct recognition ratio of each retained image set is obtained; The part categories to be retained corresponding to the retained image set whose correct recognition ratio is greater than the preset ratio are determined as the successfully recognized part categories and put into the successfully recognized part category set; The part categories to be retained corresponding to the retained image set whose correct recognition ratio is not greater than the preset ratio are determined as unrecognized part categories and placed into the unrecognized part category set; Endoscopic operation monitoring parameters are determined based on information of the successfully identified part category set and information of the unidentified part category set.

[0110] Optionally, the endoscopic operation monitoring method comprises: If the target predicted part category corresponding to the current endoscopic image is different from the part category of the current image to be retained, a second prompt message is issued, and the second prompt message is used to prompt the user to move the target endoscope; When the target endoscope is detected to be moving, a first endoscopic image recaptured by the target endoscope is acquired and the number of acquisitions is recorded; If the target predicted part category corresponding to the first endoscopic image is different from the part category of the current image to be retained, determining whether the number of acquisitions has reached a preset number; If the number of acquisitions reaches the preset number, the current part category to be retained is determined as an unidentified part category and placed in the unidentified part category set, and the next part category to be retained in the preset part category set is determined as the current part category to be retained, to obtain the image set of the next part category to be retained.

[0111] Optionally, the endoscopic operation monitoring method comprises: If the target predicted part category corresponding to the first endoscopic image is the same as the part category of the current image to be retained, a first prompt message is issued.

[0112] Optionally, determining the endoscopic operation monitoring parameters based on the information of the successfully identified part category set and the information of the unidentified part category set includes: Obtain the number of successfully identified part categories in the successfully identified part category set; Get the number of unidentified part categories in the unidentified part category set; Endoscopic operation monitoring parameters are determined based on the number of successfully identified site categories and the number of unidentified site categories.

[0113] Optionally, determining an endoscopic operation monitoring parameter based on the number of successfully identified part categories and the number of unidentified part categories includes: The total duration of the gastroscopic procedure to obtain the target endoscope; The time difference between two adjacent part categories to be retained in the preset part category set to be retained and being determined as the current part category to be retained is determined as the part stay time of the part category to be retained, and the average part stay time of each part category to be retained in the preset part category set to be retained is obtained; Endoscopic operation monitoring parameters were determined based on the total duration of gastroscopic operation, average site dwell time, number of successfully identified site categories, and number of unidentified site categories.

[0114] The specific implementation of each of the above modules can be found in the previous embodiments and will not be described again here.

[0115] An embodiment of the present application also provides an electronic device, comprising a memory and a processor, wherein the processor is configured to execute the steps of the endoscopic operation monitoring method provided in this embodiment by calling a computer program stored in the memory.

[0116] Please refer to Figure 7 , Figure 7 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application.

[0117] The electronic device may include components such as a processor 101 with one or more processing cores, a memory 102 with one or more computer-readable storage media, a power supply 103, and an input unit 104. Those skilled in the art will appreciate that the electronic device structure shown in the figure does not limit the electronic device and may include more or fewer components than shown, or combine certain components, or arrange the components differently. Among them: Processor 101 is the control center of the electronic device. It connects all parts of the electronic device using various interfaces and circuits. It executes software programs and / or modules stored in memory 102 and accesses data stored in memory 102 to perform various functions of the electronic device and process data. Optionally, processor 101 may include one or more processing cores. Alternatively, processor 101 may integrate an application processor and a modem processor. The application processor primarily handles the operating system, user interface, and application programs, while the modem processor primarily handles wireless communications. It is understood that the modem processor may not be integrated into processor 101.

[0118] Memory 102 can be used to store software programs and modules. Processor 101 executes various functional applications and data processing by running the software programs and modules stored in memory 102. Memory 102 may primarily include a program storage area and a data storage area. The program storage area may store an operating system and at least one application required for a function (such as sound playback or image playback); the data storage area may store data generated based on the use of the electronic device. Furthermore, memory 102 may include high-speed random access memory and non-volatile memory, such as at least one disk storage device, flash memory device, or other volatile solid-state storage device. Accordingly, memory 102 may also include a memory controller to provide processor 101 with access to memory 102.

[0119] The electronic device also includes a power supply 103 for supplying power to various components. Optionally, the power supply 103 can be logically connected to the processor 101 via a power management system, thereby enabling the power management system to manage charging, discharging, and power consumption. The power supply 103 can also include one or more DC or AC power supplies, a recharging system, a power failure detection circuit, a power converter or inverter, a power status indicator, and other arbitrary components.

[0120] The electronic device may further include an input unit 104, which may be configured to receive input digital or character information and generate keyboard, mouse, joystick, optical or trackball signal inputs related to user settings and function control.

[0121] Although not shown, the electronic device may also include a display unit, an image acquisition component, etc., which will not be described in detail here. Specifically, in this embodiment, the processor 101 in the electronic device will load the executable code corresponding to one or more computer programs into the memory 102 according to the following instructions, and the processor 101 will execute the steps of the endoscope operation monitoring method provided in this application, such as: Obtain the current part category to be retained, wherein the current part category to be retained is a part category to be retained in a preset part category set, and the parts to be retained in the preset part category set are arranged in a preset order; obtain the current endoscopic image acquired by the target endoscope; determine the target predicted part category corresponding to the current endoscopic image based on the current endoscopic image; determine whether the target predicted part category corresponding to the current endoscopic image is the same as the current part category to be retained; if the target predicted part category corresponding to the current endoscopic image is the same as the current part category to be retained, issue a first notification. The first prompt information is used to prompt the user to observe for a preset time and keep the image; a plurality of images to be kept collected by the target endoscope are obtained as a set of images to be kept for the current part category to be kept; when the total retention time of the plurality of images to be kept collected by the target endoscope is greater than the preset time, the next part category to be kept in the preset part category set is determined as the current part category to be kept, and a set of images to be kept for each part category to be kept in the preset part category set is obtained; the endoscope operation monitoring parameters are determined based on the set of images to be kept for each part category to be kept in the preset part category set.

[0122] It should be noted that the electronic device provided in the embodiment of the present application and the endoscope operation monitoring method in the above embodiment belong to the same concept, and its specific implementation process is detailed in the above related embodiments and will not be repeated here.

[0123] This application also provides a computer-readable storage medium having a computer program stored thereon. When the stored computer program is executed on a processor of an electronic device provided in an embodiment of this application, the processor of the electronic device performs the steps of the endoscopic operation monitoring method provided in this application. The storage medium may be a magnetic disk, an optical disk, a read-only memory (ROM), or a random access memory (RAM).

[0124] The present application also provides a computer program product or computer program, which includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform various optional implementations of the above-mentioned endoscopic operation monitoring method.

[0125] The above is a detailed introduction to an endoscopic operation monitoring method and device provided by the present application. Specific examples are used herein to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method and core idea of ​​the present application. At the same time, for technical personnel in this field, based on the ideas of the present application, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as a limitation on the present application.

[0126] It should be noted that when the above embodiments of this application are applied to specific products or technologies, the relevant user data is involved, and the user's permission or consent must be obtained, and the collection, use and processing of the relevant data must comply with the relevant laws, regulations and standards of the relevant countries and regions.

Claims

1. A method for monitoring endoscopic operation, characterized in that: include: Obtaining a current part category to be retained, wherein the current part category to be retained is a part category to be retained in a preset part category set, and the part categories to be retained in the preset part category set are arranged in a preset order; Acquire the current endoscopic image acquired by the target endoscope; determining, based on the current endoscopic image, a target prediction part category corresponding to the current endoscopic image; Determining whether the target predicted part category corresponding to the current endoscopic image is the same as the part category of the current image to be retained; If the target predicted part category corresponding to the current endoscopic image is the same as the current part category to be retained, a first prompt message is issued, wherein the first prompt message is used to prompt the user to observe for a preset time and retain the image; Acquire multiple retained images collected by the target endoscope as a retained image set of the current part category to be imaged; When the total retention time of the target endoscope acquiring the plurality of images is greater than a preset time length, the next part category to be imaged in the preset part category set to be imaged is determined as the current part category to be imaged, and a set of retained images of each part category to be imaged in the preset part category set to be imaged is obtained; Endoscopic operation monitoring parameters are determined based on the image sets of each part category to be retained in the preset part category set to be retained.

2. The method for monitoring endoscopic operation according to claim 1, wherein: The determining, based on the current endoscopic image, a target prediction part category corresponding to the current endoscopic image includes: determining the current endoscopic image and a plurality of endoscopic images preceding the current endoscopic image as a plurality of time-sequence images, wherein the plurality of time-sequence images are arranged in sequence according to the order of shooting time; When a plurality of time-series images are respectively input into a preset convolutional neural network model, a plurality of first predicted part categories corresponding to the plurality of time-series images are obtained; The plurality of time series images and the corresponding plurality of the first predicted part categories are input into a preset long short-term memory model to obtain the target predicted part category.

3. The method for monitoring endoscopic operation according to claim 2, wherein: The determining of the endoscopic operation monitoring parameters based on the image sets of each part category to be retained in the preset part category set includes: Respectively obtaining the target predicted part category of each of the retained images in the retained image set of the part category to be retained; Determine the retained image whose target prediction part category is the same as the part category to be retained as the correctly identified retained image, obtain the correct recognition ratio of the correctly identified retained images in the retained image set, and obtain the correct recognition ratio of each retained image set; Determine the part category to be retained corresponding to the retained image set whose correct recognition ratio is greater than a preset ratio as a successfully recognized part category and put it into a successfully recognized part category set; Determine the part category to be retained corresponding to the retained image set whose correct recognition ratio is not greater than a preset ratio as an unrecognized part category and put it into an unrecognized part category set; Endoscopic operation monitoring parameters are determined based on the information of the successfully identified part category set and the information of the unidentified part category set.

4. The method for monitoring endoscopic operation according to claim 3, wherein: The endoscopic operation monitoring method comprises: If the target predicted part category corresponding to the current endoscopic image is different from the part category of the current image to be retained, a second prompt message is issued, wherein the second prompt message is used to prompt the user to move the target endoscope; When movement of the target endoscope is detected, acquiring a first endoscopic image recaptured by the target endoscope and recording the number of acquisitions; If the target predicted part category corresponding to the first endoscopic image is different from the part category of the current image to be retained, determining whether the number of acquisitions has reached a preset number; If the number of acquisitions reaches the preset number, the current part category to be retained is determined as an unidentified part category and placed in an unidentified part category set, and the next part category to be retained in the preset part category set to be retained is determined as the current part category to be retained, to obtain a retained image set of the next part category to be retained.

5. The method for monitoring endoscopic operation according to claim 4, wherein: The endoscopic operation monitoring method comprises: If the target predicted part category corresponding to the first endoscopic image is the same as the part category of the current image to be retained, a first prompt message is issued.

6. The method for monitoring endoscopic operation according to claim 5, characterized in that: The determining of endoscopic operation monitoring parameters based on the information of the successfully identified part category set and the information of the unidentified part category set includes: Obtaining the number of successfully identified part categories in the successfully identified part category set; Obtaining the number of unidentified part categories in the unidentified part category set; The endoscopic operation monitoring parameter is determined based on the number of successfully identified site categories and the number of unidentified site categories.

7. The method for monitoring endoscopic operation according to claim 6, wherein: The determining of the endoscopic operation monitoring parameter based on the number of successfully identified part categories and the number of unidentified part categories includes: Obtaining the total duration of the gastroscopic operation of the target endoscope; Determine the time difference between two adjacent parts of the preset part category set to be retained and determined as the current part category to be retained as the part stay time of the part category to be retained, and obtain the average part stay time of each part category to be retained in the preset part category set; The endoscopic operation monitoring parameters are determined based on the total duration of the gastroscopic operation, the average site residence time, the number of successfully identified site categories, and the number of unidentified site categories.

8. An endoscope operation monitoring device, characterized in that: include: A first acquisition module is configured to acquire a current part category to be retained, wherein the current part category to be retained is a part category to be retained in a preset part category set, and the part categories to be retained in the preset part category set are arranged in a preset order; A second acquisition module is used to acquire the current endoscopic image acquired by the target endoscope; A first determination module is configured to determine, based on the current endoscopic image, a target prediction part category corresponding to the current endoscopic image; A judgment module, configured to judge whether the target predicted part category corresponding to the current endoscopic image is the same as the part category of the current image to be retained; an issuing module, configured to issue a first prompt message if the target predicted part category corresponding to the current endoscopic image is the same as the current part category to be retained, wherein the first prompt message is used to prompt the user to observe for a preset time period and retain the image; A third acquisition module is used to acquire a plurality of retained images collected by the target endoscope as a retained image set of the current part category to be imaged; A second determining module is configured to, when the total retention time of the plurality of images acquired by the target endoscope is greater than a preset time length, determine the next part category to be imaged in the preset part category set as the current part category to be imaged, and obtain a set of retained images of each part category to be imaged in the preset part category set; The third determining module is configured to determine endoscopic operation monitoring parameters based on the image sets of each part category to be imaged in the preset set of part categories to be imaged.

9. An electronic device, characterized in that: The method comprises a memory and a processor, wherein the memory stores a computer program, and the processor is used to run the computer program in the memory to execute the steps in the endoscopic operation monitoring method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a plurality of instructions, and the instructions are suitable for being loaded by a processor to execute the steps in the endoscopic operation monitoring method according to any one of claims 1 to 7.

Images