Endoscope device, method for controlling endoscope device, and computer program

Abstract

The invention provides an endoscope device, a method of controlling the endoscope device, and a computer program. A control method of an endoscope device according to one embodiment of the present disclosure comprises: a step of obtaining an image of the interior of a body from an image sensor; inputting the images into a trained artificial neural network model to classify the obtained images; classifying the images into an air supply condition image, a water supply condition image and / or an inspiration condition image by using the artificial neural network model, and outputting the air supply condition image, the water supply condition image and / or the inspiration condition image; and a step of controlling the endoscope device to drive an air supply unit, a water supply unit and / or an air suction unit of the endoscope device on the basis of the output classification result.

Description

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application claims priority to and the benefit of Korean Patent Application No. 10-2024-0026747, filed on February 23, 2024, which is hereby incorporated by reference for all purposes as if fully set forth herein. Technical Field

[0003] Embodiments of the present disclosure relate to a method for controlling an endoscopic device using an artificial neural network model, an endoscopic device, and a computer program. Background Art

[0004] Endoscope is a general term for medical instruments used to observe organs by inserting the endoscope into the body without surgery or autopsy. Endoscopes are inserted into the human body and illuminated with light to visualize the light reflected from the inner surface. Endoscopes are broadly categorized by purpose and body part, and can be rigid endoscopes or flexible endoscopes. Rigid endoscopes have metal endoscope tubes, while flexible endoscopes are typically used for digestive endoscopes.

[0005] When an endoscopist enters the esophagus during endoscopic examination, they must press the Air button if a stricture develops in the esophagus, the Water button if foreign matter sticks to the screen, and the Suction button when gastric fluid is visible on the screen and the procedure is complete and the instrument is removed from the esophagus. This requires multiple button presses for a single endoscopic examination, making it difficult. Summary of the Invention

[0006] The present disclosure aims to solve the above-mentioned problems in the prior art and relates to a method for controlling an endoscopic device, an endoscopic device, and a computer program, wherein the air supply, water supply, and / or air suction conditions of the endoscope are identified without user manipulation, the identified results are transmitted to the endoscope system, and the air supply, water supply, and / or air suction are automatically performed.

[0007] However, the technical problems to be solved by this embodiment are not limited to the technical problems described above, and there may also be other technical problems.

[0008] According to an embodiment of the present disclosure, a method for controlling an endoscopic device including one or more processors includes: a step of obtaining an image of the inside of a body from an image sensor; a step of inputting the image into a trained artificial neural network model in order to classify the obtained image; a step of classifying the image into an air supply condition image, a water supply condition image, and / or an inhalation condition image by the artificial neural model and outputting the image; and a step of controlling the endoscopic device to drive the air supply unit, water supply unit and / or inhalation unit of the endoscopic device based on the output classification result.

[0009] The artificial neural network model may include a plurality of artificial neural network models, and the plurality of artificial neural network models may be configured to classify and output at least one condition image among the air delivery condition image, the water delivery condition image and / or the inhalation condition image.

[0010] In the step of classifying the image into an air supply status image, a water supply status image and / or an inhalation status image by the artificial neural network model and outputting the image, the artificial neural network model can be trained to judge the image of organ stenosis as an air supply status image when the endoscope device enters the human body.

[0011] In the step of classifying the image into an air supply condition image, a water supply condition image and / or an inhalation condition image by the artificial neural network model and outputting the image, the artificial neural network model can be trained to judge the image in which impurities are stuck on the lens of the endoscope device as a water supply condition image, an air supply condition image and / or an inhalation condition image.

[0012] In the step of classifying the image into an air supply status image, a water supply status image and / or an inhalation status image by the artificial neural network and outputting the image, the artificial neural network model can be trained to judge the image of the endoscope device completing the operation and separating from the organ or the image of confirming the substance to be removed inside the organ as the inhalation status image.

[0013] The artificial neural network model may include a convolutional neural network (CNN).

[0014] The artificial neural network model may include a convolutional layer and a fully connected layer. The convolutional layer may be configured to extract features of an input image of the interior of the body through a convolution operation, and the fully connected layer may be configured to output a classification result as to which of the air supply condition image, the water supply condition image, and / or the inhalation condition image the extracted image features ultimately belong to.

[0015] The artificial neural network model classifies the image into an air supply condition image, a water supply condition image, and / or

[0016] The step of generating and outputting an inhalation condition image or an inhalation condition image may further include the step of applying a threshold value to the image classified by the artificial neural network model.

[0017] The artificial neural network model classifies the image into an air supply condition image, a water supply condition image, and / or

[0018] The step of generating and outputting the classified image or the inhalation condition image may further include the step of confirming the matching degree between the classified image and the classification results of the previous and next images.

[0019] The step of classifying the image into a gastric gas supply condition image, a water supply condition image and / or an inhalation condition image by the artificial neural network model and outputting the image may further include the step of labeling according to the gas supply condition image, the water supply condition image and / or the inhalation condition image, and in this case, the label used for labeling may be a multi-label (Multi-label) in which a label of information about an internal part of the body is attached to the corresponding image.

[0020] According to one embodiment of the present disclosure, an endoscopic device includes: a memory configured to store images of the interior of a body photographed by the endoscopic device; and a processor configured to input the images of the interior of the body into a trained artificial neural network model, classify the images into air supply condition images, water supply condition images and / or inhalation condition images, and control the endoscopic device to drive the air supply unit, the water supply unit and / or the inhalation unit based on the output classification result.

[0021] The processor may be further configured to obtain an internal body image from the image sensor through the image obtaining unit.

[0022] The processor can also be configured to transmit a control signal of the control unit to the driving unit, and the driving unit can open and close the valve of the air pump connected to the air supply unit, open and close the valve of the air pump connected to the water supply unit, or open and close the valve of the air pump connected to the air suction unit in response to the control signal.

[0023] The processor may be further configured to perform post-processing on the image classified as the air supply condition image, the water supply condition image and / or the inhalation condition image and output by the post-processing unit.

[0024] The post-processing unit may include: a threshold comparison unit configured to apply a threshold to the classified image; and a matching degree confirmation unit configured to confirm a matching degree between the classified image and classification results of previous and next images.

[0025] According to an aspect of the present invention, there is provided a computer program configured to be stored in a recording medium to execute the method using a computer. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] The above and other aspects, features and advantages of certain embodiments of the present disclosure will become more apparent from the following description taken in conjunction with the accompanying drawings, in which:

[0027] Figure 1 FIG. 1 is a diagram illustrating an endoscope device according to an embodiment of the present disclosure.

[0028] Figure 2 A diagram illustrating a method for controlling an endoscope device according to an embodiment of the present disclosure.

[0029] Figure 3 FIG. 4 is a block diagram of an endoscope system according to an embodiment of the present disclosure.

[0030] Figure 4 FIG. 1 is a conceptual diagram specifically illustrating an endoscope system according to an embodiment of the present disclosure.

[0031] Figure 5 FIG2 is a conceptual diagram illustrating an artificial neural network model according to an embodiment of the present disclosure.

[0032] Figure 6 (a) and Figure 6 (b) is a conceptual diagram illustrating an artificial neural network model according to other embodiments of the present disclosure.

[0033] Figure 7 is a flowchart showing the sub-operations of an artificial intelligence model classifying an image into an air supply condition image, a water supply condition image, and an inhalation condition image and outputting them in a method for controlling an endoscopic device according to an embodiment of the present disclosure.

[0034] FIG8 is a conceptual diagram illustrating applying a threshold to an image classified by an artificial neural network according to an embodiment of the present disclosure.

[0035] FIG9 is a conceptual diagram illustrating confirming the matching degree of the classified image with the classification results of the previous and next images according to an embodiment of the present disclosure; and

[0036] picture 10 FIG. 4 is a conceptual diagram illustrating the display of multiple labels on a screen of an endoscope device according to an embodiment of the present disclosure.

[0037]

Explanation of Figure Numbers

[0038] 100: Endoscopic device

[0039] 110: Output unit

[0040] 400: Control unit

[0041] 130: Drive unit

[0042] 150: Endoscopy

[0043] 151: Image Sensor

[0044] 153: Light Source

[0045] 154: Lens

[0046] 155, 156: working channel

[0047] 101: Processor

[0048] 102: Memory

[0049] 200: Artificial Neural Network Model

[0050] 300: Image acquisition unit

[0051] 400: Control unit

[0052] 500: Post-processing unit DETAILED DESCRIPTION

[0053] The terms used in the present invention are only used to describe specific embodiments and may not be intended to limit the scope of other embodiments. Unless otherwise clearly defined in the context, expressions in the singular may also include expressions in the plural. Including technical terms or scientific terms, the terms used herein may have the same meaning as those generally understood by those of ordinary skill in the art to which the present invention belongs. Among the terms used in the present invention, the terms defined in conventional dictionaries may be interpreted as having the same or similar meaning as the meanings in the context of the relevant technology, and, unless clearly defined in the present invention, should not be interpreted as ideal or overly formal meanings. In some cases, even the terms defined in the present invention cannot be interpreted as excluding embodiments of the present invention.

[0054] Hereinafter, various embodiments will be described in detail with reference to the accompanying drawings so that a person skilled in the art can easily implement the present invention. However, the technical ideas of the present invention can be changed and implemented in various forms and are therefore not limited to the embodiments described in this specification. When describing the embodiments disclosed in this specification, if it is considered that a specific description of the relevant known technologies may obscure the gist of the technical ideas of the present invention, the specific description of the known technologies will be omitted. The same reference symbols are given to the same or similar components, and repeated descriptions are omitted.

[0055] The term "unit" as used in this embodiment refers to a component configured to perform a specific function that is executed by software or hardware such as a field programmable gate array (FPGA) or an application specific integrated circuit (ASIC). However, the "unit" is not limited to being executed by software or hardware. The "unit" can be in the form of data stored in an addressable storage medium or implemented as instructions to cause one or more processors to perform a specific function.

[0056] Software may include a computer program, code, instructions, or a combination of one or more thereof, and may configure a processing device to operate as needed, or may command the processing device independently or collectively. In order for the processing device to interpret or provide instructions or data to the processing device, the software and / or data may be permanently or temporarily embodied in some type of machine, component, physical device, virtual equipment, computer storage medium or device, or transmitted signal wave. The software may be distributed across a computer system connected to a network and may be stored or executed in a decentralized manner. The software and data may be stored in one or more computer-readable recording media. The software may be read into main memory from other devices via other computer-readable media such as a data storage device or a communication interface. The software instructions stored in the main memory may cause the processor to execute a program or step to be described in detail later. Alternatively, fixed wiring circuits may be used instead of or in combination with software instructions to execute programs consistent with the principles of the present invention. Therefore, embodiments consistent with the principles of the present invention are not limited to some specific combination of hardware circuits and software.

[0057] The terms used in this application are only used to describe specific embodiments and are not used for the purpose of limiting the present invention. Unless otherwise clearly defined in the context, expressions in the singular also include expressions in the plural. In this application, terms such as "including" or "having" specify the presence of features, numbers, steps, operations, constituent elements, parts, or combinations thereof recorded in the specification, rather than excluding in advance the presence or additional possibilities of one or more other features, numbers, steps, operations, constituent elements, parts, or combinations thereof. Terms such as first and second can be used to describe various constituent elements, but the constituent elements should not be limited to the terms. The terms are only used to distinguish one constituent element from other constituent elements.

[0058] The 'learning model' referred to in the present invention may include all forms of computational methods or methodologies used to learn or understand specific patterns or structures based on data. That is, learning models may include not only machine learning models such as regression models, decision trees, random forests, support vector machines, K-nearest neighbors, naive Bayes, and clustering algorithms, but also deep learning models such as neural networks, convolutional neural networks, recurrent neural networks, Transformer-based neural networks, generative adversarial networks (GANs), and autoencoders. A 'learning model' may refer to a set of learned parameters or weights used to predict or classify outputs for specific inputs. Such models may be trained using methods such as supervised learning, unsupervised learning, semi-supervised learning, and reinforcement learning. Furthermore, learning models may include not only single models but also various learning methods and structures, such as ensemble models, multimodal models, and models learned through transfer learning. Such learning models may be pre-trained on a separate computer device from the computer device configured to predict outputs for inputs and may be used by other computer devices.

[0059] Figure 1 FIG. 1 is a diagram illustrating an endoscope device according to an embodiment of the present disclosure.

[0060] Reference Figure 1 The endoscope device 100 according to one embodiment of the present disclosure may be a flexible endoscope, specifically a digestive organ endoscope. The endoscope device 100 may include a configuration capable of obtaining medical video footage of the interior of the digestive organ and, if necessary, inserting a device and performing treatment or manipulation while viewing the medical video.

[0061] The endoscope device 100 may include an output unit 110 , a control unit 400 , a driving unit 130 , an endoscope 150 , and a control unit 400 .

[0062] The output unit 110 may include a display for displaying medical videos. The output unit 110 may include a display module capable of outputting visual information or implementing a touch screen, such as a liquid crystal display (LCD), a thin film transistor-liquid crystal display (TFT LCD), an organic light-emitting diode (OLED), a flexible display, a 3D display, etc.

[0063] The output unit 110 may include various tools configured to provide information about the medical video, ie, the image inside the body, and may include, for example, a speaker configured to provide information about the image inside the body through hearing.

[0064] The output unit 110 may be configured to display an internal body image obtained from the endoscope 150 or an internal body image processed by the control unit 400. The output unit 110 may be configured to display the internal body image and may further display a result of classification and analysis of the internal body image by the artificial neural network.

[0065] For example, the output unit 110 may display information indicating that a corresponding frame in the internal body image requires inhalation, information indicating that inhalation is in progress, and the like.

[0066] although Figure 1 Although one output unit 110 is shown, the number of output units 110 may be plural. In this case, the output unit for displaying the image of the interior of the body obtained by the endoscope 150 can be distinguished from the output unit for displaying the information processed by the control unit 1400. Furthermore, a first screen displaying the image of the interior of the body obtained by the endoscope 150 and a second screen displaying the information processed by the controller 400 and the image of the interior of the body can be distinguished on the one output unit.

[0067] The control unit 400 can be configured to control the entire operation of the endoscopic device 100. The control unit 400 can include all types of devices capable of processing data. For example, the control unit 400 can be a data processing device embedded in hardware, which has a physically structured circuit to execute the functions expressed by the code or instructions included in the program. As an example of a data processing device embedded in hardware, it can include a microprocessor, a central processing unit (CPU), a processor core (processor core), a multiprocessor (multiprocessor), an application-specific integrated circuit (ASIC), a field programmable gate array (field programmable gate array) and other processing devices, but the technical ideas of the present disclosure are not limited to this.

[0068] The control unit 400 may control the movement of the endoscope 150 through the driving unit 130 connected to the endoscope 150. The control unit 400 may perform various control operations through the endoscope 150 to photograph the inside of the body, specifically, the inside of the digestive organs.

[0069] The control unit 400 can perform various processes on the medical video obtained through the endoscope 150. The control unit 400 can control the endoscope device 100 based on the control signal received from the computing device. For example, the control unit 120 can open or close the suction pump in response to the opening and closing signal of the valve connected to the suction pump.

[0070] The drive unit 130 can be configured to provide the power required for inserting the endoscope 150 into the body or moving it within the body. For example, the drive unit 130 may include a plurality of motors connected to wires within the endoscope 150 and a tension adjustment unit configured to adjust the tension of the wires. The drive unit 130 can control the endoscope 150 in various directions using the power of each of the plurality of motors.

[0071] although Figure 1 The control unit 400 and the drive unit 130 are shown as separate and independent hardware, but are not limited thereto. For example, the control unit 400 and the drive unit 130 may be implemented in a single hardware component, or in two or more hardware components. When implemented in two or more hardware components, a portion of the control unit 400 and a portion of the drive unit 130 may be physically separated.

[0072] Endoscope 150 may include an insertion portion 151a, a curved portion 150b, and a distal end 150c. Insertion portion 150a, curved portion 150b, and distal end 150c may include at least one of an air supply pump, an air suction pump, and a water supply pump. The air supply pump injects air into the body through endoscope 150. The air suction pump draws air from the body through endoscope 150 by applying negative pressure or vacuum. The water supply pump injects wash water into the body through endoscope 150. Each pump may include a valve for controlling the movement of fluid. The valve of each pump may be opened and closed by control unit 400.

[0073] According to one embodiment of the present disclosure, the valve of the intake pump can be opened and closed based on a control signal from a computing device or control by the control unit 400. According to one embodiment, the control unit 400 can open the intake valve based on the intake valve opening time calculated by the computing device. The control unit 400 can also close the intake valve in response to an intake valve blocking signal from the computing device or after the time calculated by the computing device has elapsed.

[0074] The endoscope 150 may include an insertion portion 150a inserted into the body, specifically, into the digestive organ, and an operation unit 160 configured to control movement of the insertion portion 150a and receive input from a user to perform various procedures inside the digestive organ.

[0075] The insertion portion 150a is configured to be flexibly bendable, and one end thereof is connected to the drive unit 130. Therefore, the degree or direction of bending of the insertion portion 150a can be determined by the drive unit 130. Since imaging of the interior of the body and performing procedures are performed at the distal end of the insertion portion 150a, the endoscope 150 may include a plurality of cables and tubes extending to the distal end of the insertion portion 150a.

[0076] The curved portion 150b and the distal end 150c may be formed at the distal end of the insertion portion 150a. The distal end 150c may include an image sensor 151 configured to obtain images of the interior of the body. The distal end 150c can adjust its rotation angle via the drive unit 130, which receives control signals from the control unit 400, to identify various internal body parts, capture images of the interior of the body, and perform procedures within the body.

[0077] The curved portion 150b may be connected to the distal end portion 150c. The driving unit 130 may adjust the degree of curvature of the curved portion 150b in response to a control signal from the control unit 400. The rotation angle of the distal end portion 150c may be adjusted according to the degree of curvature of the curved portion 150c.

[0078] According to this embodiment, when the first bending steering unit 161 and the second bending steering unit 162 are operated, a signal is transmitted to the control unit 400 via the signal transmission system mounted on the operating unit 160. The signals processed by the control unit 400 are transmitted to the drive unit 130 to drive the motor, thereby controlling the bending portion 150b. The first bending steering unit 161 can control the vertical movement of the bending portion 150b and the distal end 150c, while the second bending steering unit 162 can control the horizontal movement of the bending portion 150b and the distal end 150c.

[0079] Reference Figure 1 The distal end portion 150c of the endoscope 150 may include an image sensor 151 therein, and may include a light source 153, a lens 154, a first working channel 155, and a second working channel 156 at its distal end. The image of the interior of the body photographed through the lens 154 can be perceived by the image sensor 151. The light source 153 can illuminate the dark interior of the body by emitting light adjacent to the lens 154, so that the interior of the body can be photographed more clearly and accurately through the lens 154.

[0080] Instruments used to treat and manage lesions during endoscopic procedures can be inserted through first working channel 155. For example, wash water can be supplied to the interior of the body through operating unit working channel 163 via first working channel 155. Furthermore, first working channel 155 can be used to draw fluid from the interior of the body, specifically, from the digestive organs, through the valve of the suction pump of suction unit 430.

[0081] In addition to the first working channel 155, Figure 1 Also shown is a second working channel 156. In the second working channel 156, an operation different from that performed in the first working channel 155 can be performed. For example, while the operation of inhaling fluid is performed in the first working channel 155, the operation of injecting air for ventilation into the body can be performed in the second working channel 156.

[0082] The operation unit 160 may include a plurality of input buttons providing various functions allowing the endoscopist to control the direction of the insertion portion 150b and perform an operation through the first working channel 155. Regardless of the control of the computer device, the endoscopist can use the operation unit 160 to suction inside the digestive organ.

[0083] The aforementioned operations of the control unit 400 may be performed by the drive unit 130. To this end, the drive unit 130 may include a necessary processing device, such as a microprocessor, a CPU, etc. In this case, the control unit 400 may process images of the interior of the body obtained through the endoscope 150, and the drive unit 130 may control the endoscopic device 100 based on control signals received from the computing device.

[0084] Figure 2 A diagram illustrating a method of controlling an endoscope device according to an embodiment of the present disclosure. Figure 3 FIG. 4 is a block diagram of an endoscope system according to an embodiment of the present disclosure. Figure 4 FIG. 1 is a conceptual diagram specifically illustrating an endoscope system according to an embodiment of the present disclosure.

[0085] Reference Figures 1 to 4 According to an embodiment of the present disclosure, a method for controlling an endoscope device includes a step of obtaining an image of the interior of a body from an image sensor 151 (S100), a step of inputting the image of the interior of the body into an artificial neural network model 200 trained to classify the obtained image (S200), a step of classifying the image of the interior of the body into an air supply condition image, a water supply condition image and / or an inhalation condition image by the artificial neural network model 200 and outputting the image (S300), and a step of controlling the endoscope device 100 based on the output classification result to drive the air supply unit 410, the water supply unit 420 and / or the inhalation unit 430 of the endoscope device 100 (S400).

[0086] The endoscopic device 100 according to an embodiment of the present disclosure may include a hardware device or a portion of a hardware device configured for integrated data processing and computation, and may also include a software-based computing environment connected to a communication network. For example, the endoscopic device 100 may include a server, which is the entity that performs intensive data processing functions and shares resources, and may also include a user (client) who shares resources by interacting with the server.

[0087] Furthermore, the endoscope device 100 may further include a cloud system configured to enable multiple servers and clients to interactively and comprehensively process data. The above description is merely an example of the configuration of the endoscope device 100. Therefore, the endoscope device 100 may be configured in various forms within the scope of ordinary skill in the art based on the present disclosure.

[0088] Reference Figure 3 According to an embodiment of the present disclosure, an endoscope device 100 may include a processor 101 , a memory 102 , an artificial neural network model 200 , an image acquisition unit 300 , a control unit 400 , and a post-processing unit 500 .

[0089] The processor 101 according to an embodiment of the present disclosure can be understood as including a constituent unit, which includes hardware and / or software for computing operations. For example, the processor 101 can perform data processing for machine learning by interpreting a computer program. The processor 101 can process computing processes, for example, processing input data for machine learning, extracting features for machine learning, performing error calculation based on backpropagation, etc. The processor 101 for performing such data processing may include a central processing unit (CPU), a general purpose graphics processing unit (GPGPU), a tensor processing unit (TPU), an application specific integrated circuit (ASIC), or a field programmable gate array (FPGA). The above-mentioned type of processor 101 is only an example, and therefore, the type of processor 101 can have various structures within the scope that can be understood by ordinary technicians based on the content of this disclosure.

[0090] Processor 101 can use internal body images, such as endoscopic images, as learning data to train artificial neural network model 200. Specifically, processor 101 can train artificial neural network model 200 to determine in real time whether air supply, water supply, and / or suction are required within the body based on endoscopic images obtained by photographing the interior of the body. In this case, the endoscopic images can include endoscopic videos.

[0091] Furthermore, images of the interior of the body, i.e., endoscopic videos, can be input into a trained artificial neural network model, and the images can be classified into air supply status images, water supply status images, and / or inhalation status images and output, and the endoscopic device can be controlled to drive the air supply unit / water supply unit and / or inhalation unit according to the output classification results.

[0092] In this specification, the interior of the body may refer to organs such as the stomach, duodenum, small intestine, and large intestine, and more specifically, may refer to a portion of an organ, such as the cardia, angulo, and pylorus of the stomach. The following description will take the digestive organs as an example.

[0093] During an endoscopic procedure, the endoscopist must perform the following operations: when entering the esophagus, if esophageal stricture occurs, press the air button; when foreign matter sticks to the lens 154, press the water button; and when gastric fluid is visible on the lens 154 (i.e., the screen) and when the procedure is complete and the object is removed from the esophagus, press the suction button. This requires multiple button operations for a single endoscopic diagnosis, which can be difficult.

[0094] Specifically, during endoscopic procedures, it is necessary to ensure a safe distance between endoscope 150, which includes lens 154, and the digestive organs. Therefore, medical personnel can inject air into the digestive organs during endoscopic procedures to expand them. If insufficient air is injected, lesions located between the wrinkles of the organs may not be observed, or the lesions may become trapped. Furthermore, as the amount of air injected changes, the shape of the organs and lesions also changes. Therefore, for accurate diagnosis, it is essential to inject the appropriate amount of air, determined in real time during endoscopic procedures. Furthermore, if impurities adhere to lens 154, the accuracy of the medical video will be reduced. Therefore, air can be sprayed into the lens to remove the impurities.

[0095] Furthermore, when performing endoscopic surgery, it may be necessary to clean the lesion and its surrounding area, i.e., the affected area. In addition, in order to cope with the situation where foreign matter is stuck on the lens 154 and the screen cannot be seen clearly, it may be necessary to clean the lens 154. In this case, cleaning water can be sprayed onto the affected area or the lens.

[0096] Furthermore, during endoscopic procedures, suction is frequently performed to ensure a clear field of view and accurately detect lesions by aspirating and expelling fluids or impurities within the digestive organs. For example, if impurities or accumulated fluids are present around the affected area, ensuring accurate video is difficult, so suction is performed. Suction is also performed to remove debris from biopsies for surgical procedures or tissue examinations. If excessive air or gas is injected during expansion of the organ under observation, suction unit 430 appropriately aspirates the air or gas.

[0097] As described above, water supply, air supply and air suction are required in various situations. According to an embodiment of the present disclosure, the endoscope device 100 can be configured to determine whether water supply, air supply and / or air suction are required inside the body based on the internal image of the body. If necessary, the air supply unit 410, the water supply unit 420 and the air suction unit 430 of the endoscope device 100 can be controlled.

[0098] For example, the processor 101 may train an artificial neural network model to determine that the presence of impurities around an affected area in an image of the interior of the body or excessive expansion of the interior of the body is a situation requiring inspiration.

[0099] At this time, the processor 101 can train an artificial neural network model, which is configured to judge the situation where the lens 154 for photographing the internal body image needs to be washed, the situation where impurities exist around the affected part represented in the internal body image, and the situation where the body is excessively expanded, or it can train multiple artificial neural network models corresponding to each situation.

[0100] The processor 101 can train the artificial neural network model 200 through supervised learning using learning data as input values. Alternatively, the artificial neural network model can be trained through unsupervised learning. Unsupervised learning is a learning method that discovers a benchmark for data recognition by spontaneously learning the data type required to recognize the data without specific guidance. Alternatively, the artificial neural network model can be trained through reinforcement learning, which is a learning method that uses feedback on whether the result recognized based on the learned data is correct.

[0101] The artificial neural network model 200 according to the present disclosure may include a convolutional neural network (CNN). However, the present embodiment is not limited thereto, and the artificial neural network 200 according to the present disclosure may include a deep neural network (DNN), a recurrent neural network (RNN), a bidirectional recurrent deep neural network (BRDNN), a multilayer perception (MLP), a transformer, and other network models.

[0102] The processor 101 can input the internal body image into the trained artificial neural network model to determine the need for air supply, water supply, and suction in the digestive organs. The processor 101 can generate a signal for controlling the endoscope device 100 based on the output value of the artificial neural network model.

[0103] Specifically, the processor 101 can generate control signals for executing air supply, water supply, and / or suction. The processor 101 can generate control signals for selectively opening and closing valves of the air supply pump, water supply pump, and / or suction pump connected to the working channels 155 and 156 to synchronize the operation of the endoscope 150 of the endoscopic device 100 inserted into the digestive organ with the endoscopic devices 155 and 156 for suction. The processor 101 can directly generate control signals for controlling the endoscopic device 100 based on the output results of the artificial neural network model or instruct the endoscopic device 100 to generate control signals.

[0104] For example, the processor 101 can control the valve of the suction pump of the endoscope device 100 by distinguishing between situations where the lens for photographing internal body images needs to be cleaned, situations where impurities exist around the affected area shown in the internal body images, and situations where the digestive organs are excessively expanded.

[0105] During the endoscopic procedure, the processor 101 may receive images of the body's interior in frames and determine whether air supply, water supply, and / or suction are required for each frame. The above operations may be repeated until the endoscopic procedure is completed.

[0106] The memory 102 according to an embodiment of the present disclosure can be understood as a component unit including hardware and / or software for storing and managing data processed by the endoscopic device 100. That is, the memory 102 can store any form of data generated or determined by the processor 101 and any form of data received by a network unit (not shown). For example, the processor 102 can include at least one type of storage medium selected from the group consisting of flash memory type, hard disk type, multimedia card micro type, card type memory, random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic storage, magnetic disk, or optical disk. Furthermore, the memory 102 can also include a database system configured to control and manage data using a certain system. The above-mentioned type of memory 120 is only an example. Therefore, the type of memory 102 may have various structures within the scope that can be understood by ordinary technicians based on the content of this disclosure.

[0107] The memory 102 can structure, organize, and manage data and data combinations required for the processor 101 to perform operations, as well as program codes that can be executed by the processor 101. In addition, the memory 102 can store program codes that enable the processor 101 to generate learning data.

[0108] According to the present disclosure, the memory 102 may be configured to store an internal body image photographed by the endoscope device 100 and store a control signal generated by the processor 101 based on the internal body image.

[0109] According to an embodiment of the present disclosure, a network unit can be understood as a component that transmits / receives data via any form of known wireless communication system. For example, the network unit can use a wireless communication system such as a local area network (LAN), wideband code division multiple access (WCDMA), long term evolution (LTE), wireless broadband internet (WiBro), fifth generation mobile communication technology (5G), ultra wide-band, ZigBee, radio frequency (RF) communication, wireless local area network (WLAN), wireless fidelity, near field communication (NFC), or Bluetooth to transmit / receive data. The above-mentioned communication systems are merely examples, and therefore, the wireless communication systems used by the network unit to transmit / receive data can be widely applied to fields beyond the examples described above.

[0110] The network unit can be configured to receive data required for processor 101 to perform operations through wireless communication with any system or client. Furthermore, the network unit can be configured to transmit data generated by operations performed by processor 101 through wireless communication with any system or client. For example, the network unit can be configured to receive medical videos through communication with a medical video storage and transmission system, a cloud server configured to perform tasks such as medical data standardization, or a separate computing device. The network unit can transmit various data generated by operations performed by processor 101 through communication with the aforementioned systems, servers, or separate computing devices.

[0111] The processor 101 can be configured to control the endoscopic device to input internal body images into the trained artificial neural network model 200, classify the images into air supply condition images, water supply condition images and / or inhalation condition images and output them, and drive the air supply unit, water supply unit and / or inhalation unit based on the output classification results.

[0112] The processor 101 may be configured to obtain an internal body image from the image sensor via the image acquisition unit 300. Furthermore, the processor 101 may be configured to transmit a control signal from the control unit 400 to the drive unit 130, which may

[0113] The control unit 410 is configured to respond to opening and closing of a valve of an air supply pump connected to the air supply unit 410, opening and closing of a valve of a water supply pump connected to the water supply unit 420, or opening and closing of a valve of an air suction pump connected to the air suction unit 430. The air supply unit 410, the water supply unit 420, and the air suction unit 430 may be disposed inside the insertion portion 150a of the endoscope 150 and may be configured to supply air, water, or suction into the body based on driving information controlled by the control unit.

[0114] The processor 101 may be configured to perform post-processing on the images classified as the air delivery condition image, the water delivery condition image, and / or the inhalation condition image and outputted through the post-processing unit 500 .

[0115] According to the present disclosure, as will be described later, the post-processing unit 500 may include a threshold comparison unit 510 and a matching degree confirmation unit 520, wherein the threshold comparison unit 510 is configured to apply the threshold to the classified image, and the matching degree confirmation unit 520 is configured to confirm the matching degree of the classified image with the classification results of the previous and next images.

[0116] Reference Figure 4 The image acquisition unit 300 can acquire images of the interior of the body. For example, it can acquire image A of a narrowed esophagus E, image B of foreign matter S adhering to the lens L, and image C of gastric fluid GJ around the esophagus E.

[0117] The images thus acquired by the image acquisition unit 300 can be input into the artificial neural network model 200. The artificial neural network model 200 can classify the various input internal body images into air supply status images 301, water supply status images 302, and / or inhalation status images 303. Images that do not fall into the aforementioned three status images can also be classified separately.

[0118] The artificial neural network model 200 can output three classified condition images. These images can then be transmitted to one of the air supply unit 410, water supply unit 420, and air intake unit 430 in the control unit 400. Images that do not fall into these three categories may not be transmitted to the control unit 400.

[0119] For example, it is assumed that the endoscope apparatus 100 located inside the body captures a specific image at a specific time point. In this case, when the image sensor 151 of the endoscope apparatus 100 located inside the body senses the image A entering the esophagus E where a stenosis occurs, the image acquisition unit 300 can acquire the image A entering the esophagus E where a stenosis occurs through the image sensor 151. AThe image A of the esophagus E obtained by the image obtaining unit 300 is input into the artificial neural network model 200 , and the artificial neural network model 200 can determine that the image is the air supply status image 301 based on the trained content.

[0120] If the image is determined to be the air supply status image 301, the artificial neural network model 200 may output information that the image is the air supply status image 301 to the control unit 400. The control unit 400 may open and close the valve of the air supply pump connected to the air supply unit 410 through the driving unit 130 based on the information received from the artificial neural network model 200. The air supply unit 410 may be connected to the working channel 301. 15 5 and 156 spray air supplied from the air pump, and the air is sprayed to the esophagus where the stenosis occurs. E The esophagus E is dilated, and the distal end portion 150 c of the endoscope device 100 can be easily inserted into the esophagus E.

[0121] According to one embodiment of the present disclosure, in the step (S300) in which the artificial neural network model 200 classifies an image into an air supply status image, a water supply status image and / or an inhalation status image and outputs the image, the artificial neural network model 200 can be trained to judge an image in which organ stenosis occurs as an air supply status image when the endoscope device 100 enters the human body.

[0122] For example, the artificial neural network model 200 may be trained to determine an image of esophageal stenosis as an air supply status image when the endoscope device 100 enters the esophagus.

[0123] Furthermore, according to one embodiment of the present disclosure, in the step (S300) in which the artificial neural network model 200 classifies an image into an air supply condition image, a water supply condition image and / or an inhalation condition image and outputs the image, the artificial neural network model 200 can judge the image with impurities stuck on the lens of the endoscope device 100 as a water supply condition image, an air supply condition image and / or an inhalation condition image.

[0124] In order to remove the impurities stuck on the lens, the impurities stuck on the lens can be removed by starting one or more operations of water supply, air supply and / or suction.

[0125] Furthermore, according to one embodiment of the present disclosure, in the step (S300) in which the artificial neural network model 200 classifies an image into an air supply status image, a water supply status image and / or an inhalation status image and outputs the image, the artificial neural network model 200 can be trained to judge an image in which the endoscopic device 100 completes the operation and leaves the organ or an image in which impurities inside the organ are confirmed as an inhalation status image.

[0126] For example, the artificial neural network model 200 can be trained to identify an image showing the endoscope device 100 completing a procedure and being removed from the esophagus as an image in the suction state. Furthermore, the artificial neural network model 200 can be trained to identify an image showing the endoscope device 100 confirming impurities within an organ as an image in the suction state. This allows the artificial neural network model 200 to inhale and remove substances within the organ through suction when substances to be removed, such as excess gastric juice, digestive products, nasal discharge, or lesions, are confirmed.

[0127] although Figure 4 An artificial neural network model 200 is shown to determine whether air supply, water supply and / or suction are required in various situations. The artificial neural network model 200 may also include a plurality of artificial neural network models for judging various situations.

[0128] As described above, the use of the artificial neural network model 200 according to the present disclosure can greatly reduce the physical burden of the endoscopic surgery subject and significantly improve the convenience of endoscopic diagnosis by automatically executing water supply, air supply and / or suction operations according to the situation, compared with the existing technology that requires complex manipulation to perform water supply, air supply and / or suction operations.

[0129] Figure 5 FIG2 is a conceptual diagram illustrating an artificial neural network model according to an embodiment of the present disclosure.

[0130] Reference Figure 5 According to an embodiment of the present disclosure, the artificial neural network model 200 may include a convolutional neural network (CNN). The image transmitted from the image acquisition unit 300 may be input into the artificial neural network model 200. At this time, the convolutional layer 210 of the artificial neural network model 200 may analyze the image through a convolution operation. The convolutional layer 210 may extract the feature portion of the image by performing operations on a plurality of layers. The extracted feature portion may be reduced in dimension of the image data through a pooling layer 220 to improve the computational efficiency of the neural network. The convolutional layer and the pooling layer may be repeatedly constructed to concretize the feature portion of the image.

[0131] The features extracted through the above process can be passed to the fully connected (FC) layer 230. The FC layer 230 can use the input extracted features to output a classification result regarding which category each image belongs to (water delivery status image, water delivery status image, or air inhalation status image). The output content can be passed to the control unit 400.

[0132] Figure 6 (a) and Figure 6(b) is a conceptual diagram illustrating an artificial neural network model according to other embodiments of the present disclosure. Figure 6 (a) is a conceptual diagram illustrating an artificial neural network model according to another embodiment of the present disclosure. Figure 6 (b) is a conceptual diagram illustrating an artificial neural network model according to another embodiment of the present disclosure.

[0133] Reference Figure 6 (a), the artificial neural network model 200' may include a plurality of artificial neural network models 201' and 202'. In this case, the plurality of artificial neural network models 201' and 202' may be configured to classify and output at least one of an air delivery status image, a water delivery status image, and / or an inhalation status image.

[0134] For example, the first artificial neural network model 201' may classify the input image into an air supply condition image and a water supply condition image and output the images, and the second artificial neural network model 202' may classify the input image into an inhalation condition image and output the images.

[0135] Reference Figure 6 (b), the artificial neural network model 200" may include a plurality of artificial neural network models 201", 202", and 203". At this time, the plurality of artificial neural network models 201", 202", and 203 may be configured to classify and output at least one of an air supply condition image, a water supply condition image, and / or an inhalation condition image.

[0136] For example, the first artificial neural network model 201" can be configured to classify the input image as an air supply condition image and output it, the second artificial neural network model 202" can be configured to classify the input image as a water supply condition image and output it, and the third artificial neural network model 203" can be configured to classify the input image as an inhalation condition image and output it.

[0137] The plurality of artificial neural network models according to the present disclosure are not limited to Figure 6 (a) and Figure 6 The content disclosed in (b) and the number of artificial neural network models and the range of the situation images classified by each artificial neural network model can be implemented in various ways.

[0138] Figure 7 It is a flow chart showing the detailed steps in which an artificial intelligence model classifies images into air supply status images, water supply status images, and inhalation status images and outputs them in a method for controlling an endoscopic device according to one embodiment of the present disclosure.

[0139] Reference Figure 7In the method for controlling an endoscope device according to an embodiment of the present disclosure, the step (S300) of classifying an image into an air supply condition image, a water supply condition image, and / or an inhalation condition image by the artificial neural network model 200 and outputting the image may include the step (S310) of applying a threshold to the image classified by the artificial neural network model 200 and the step (S311) of confirming the matching degree between the classified image and the classification results of the previous and next images.

[0140] (S320).

[0141] Figure 8 FIG2 is a conceptual diagram illustrating applying a threshold to an image classified by an artificial neural network according to an embodiment of the present disclosure.

[0142] Reference Figure 8 A threshold value can be applied as a post-processing step to improve the classification accuracy of the gas supply condition images, water supply condition images, and / or inhalation condition images determined using the artificial neural network model 200. In this case, by applying a threshold value for a specific criterion to the classification judgment criterion of the gas supply condition images, water supply condition images, and / or inhalation condition images classified by the artificial neural network model 200, it can be determined that only images that meet the threshold value belong to the specific condition image.

[0143] At this time, the threshold value may be determined based on data learned using the air supply condition image, the water supply condition image, and / or the inhalation condition image. Figure 7 As shown, the threshold values ​​according to classification can be set differently as the first threshold value 301a, the second threshold value 301b, and the third threshold value 301c, thereby applying different threshold values ​​according to the patient's condition during endoscopic surgery to control water supply, air supply, and suction differently according to the situation.

[0144] Figure 9 is a conceptual diagram illustrating confirmation of a matching degree between a classified image and classification results of previous and next images according to an embodiment of the present disclosure; and

[0145] Reference Figure 9The degree of matching can be confirmed as a post-processing step to improve the classification accuracy of the air supply status image, water supply status image, and / or inhalation status image determined by the artificial neural network model 200. In other words, the endoscope captures images of the interior of the body in frames. By comparing the image captured in the preceding image frame and the image captured in the subsequent image frame with the image of the current frame, the degree of output matching between the images can be confirmed. In this case, the output matching can be improved by confirming the degree of matching between the image corresponding to each frame input into the artificial neural network and the image of the adjacent frame after classification.

[0146] Figure 10 FIG. 1 is a conceptual diagram illustrating the display of multiple labels on a screen of an endoscope device according to an embodiment of the present disclosure.

[0147] The artificial neural network model 200 can receive labels (Labels) related to endoscopic images as learning data. The endoscopic video can be an endoscopic video captured by an endoscopic device. The label can include information indicating that the corresponding internal body image is in a state requiring air supply, water supply, and / or suction. For example, the state requiring suction can include various states, such as the presence of impurities around an affected area in the internal body image and excessive distension of the digestive organs. In this case, the internal body image can be labeled as requiring suction.

[0148] Furthermore, for example, if the internal body image is in sharp focus or if there are no foreign objects around the affected area, the internal body image can be labeled as normal, meaning that inhalation is not necessary. This labeling can be performed by medical personnel, or a separate artificial neural network model can be used for labeling.

[0149] According to an embodiment of the present disclosure, the step (S300) of classifying the image into an air supply image, a water supply image, and / or an inhalation image by the artificial neural network model 200 and outputting the classification may further include the step (S330) of labeling the air supply image, the water supply image, and / or the inhalation image. In this case, the labeling may be a multi-label, in which a label is attached that contains information about a portion of the body's interior in the image.

[0150] Figure 10 An example of multi-label images is shown. Figure 9As shown, when labeling specific internal body parts, in addition to label information regarding air delivery, water delivery, and inhalation, the labels can also include information regarding the corresponding location, such as the esophagus, duodenum, or stomach, for output. Specifically, when classifying images, the artificial neural network model 200 can be configured to further classify and determine the water delivery, air delivery, and inhalation status of the internal body location and transmit the corresponding information to the control unit 400.

[0151] The control unit 400 can further determine information regarding the internal organ within the body where the distal end portion 150c of the endoscope is currently located, and information regarding whether the distal end portion 150c is currently inserted into or removed from the body. This allows for more precise control of the degree of air supply, water supply, and suction based on the position and advancement direction of the distal end portion 150c during air, water, and suction control operations of the endoscope.

[0152] The memory 102 can store multiple application programs to be driven, data used for operating the computer device, and instructions. The memory 102 can be embodied as internal memory such as read-only memory (ROM) and random access memory (RAM) included in the processor 101, or it can be embodied as a memory separate from the processor 101. According to one embodiment, the memory 102 can store neural networks and learning data.

[0153] Specifically, the processor 101 can use various programs stored in the memory 102 of the computer device to control the operation of the computer device. The processor 101 may include a CPU, a random access memory (RAM), a read-only memory (ROM), a system bus, etc. The processor 101 may be embodied as a single CPU or a plurality of CPUs (or DSPs, SoCs). According to one embodiment, the processor 101 may be embodied as a digital signal processor (DSP) that processes digital signals, a microprocessor (microprocessor), or a time controller (TCON). However, this embodiment is not limited to this, and the processor 101 may include a central processing unit (CPU), a microcontroller (MCU), a microprocessor (MPU), a processor (controller), an application processor (AP), or a communication processor (CP), one or more of an ARM processor, or may be defined by these terms. Furthermore, the processor 101 may be implemented as a system on chip (SoC) or large scale integration (LSI) with embedded processing algorithms, or in the form of a field programmable gate array (FPGA).

[0154] The apparatus described above can be embodied as hardware components, software components, and / or a combination of hardware components and software components. For example, the apparatus and components described in the embodiments can be implemented using one or more general-purpose computers or special-purpose computers, such as a processor, a controller, an arithmetic logic unit (ALU), a digital signal processor (DSP), a microcomputer, a field programmable gate array (FPGA), a programmable logic unit (PLU), a microprocessor, or any other device capable of executing and responding to instructions. The processing device can execute an operating system (OS) and one or more software applications executed on the operating system. Furthermore, the processing device can approach, store, operate, process, and generate data in response to software execution. For ease of understanding, the use of a processing device can be described, however, a person skilled in the art of the present disclosure should be able to understand that the processing device can include a plurality of processing elements and / or a plurality of types of processing elements. For example, the processing device can include a plurality of processors or a processor and a controller. Furthermore, other processing configurations, such as parallel processors, may also be used.

[0155] Software may include a computer program, code, instruction, or one or more combinations thereof, and may configure a processing device to operate as needed, or may independently or collectively command the processing device. In order to be interpreted by the processing device or to provide instructions or data to the processing device, the software and / or data may be permanently or temporarily embodied in some type of machine, component, physical device, virtual device, computer storage medium or device, or transmitted signal wave. The software may be dispersed across computer systems connected to a network and may be stored or executed in a decentralized manner. The software and data may be stored in one or more computer-readable recording media.

[0156] The method according to the embodiment can be embodied as a program instruction form that can be executed by various computer means and recorded in a computer-readable medium. The computer-readable medium can include program instructions, data files, data structures, etc. individually or collectively. The program instructions recorded in the medium may be specially designed and constructed for the embodiment, or they may be known and usable by those of ordinary skill in the field of computer software. Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks, and magnetic tapes, optical recording media such as CD-ROMs and DVDs, magneto-optical media such as floppy disks, and hardware devices such as read-only memories (ROMs), random access memories (RAMs), and flash memories that are specifically configured to store and execute program instructions. Examples of program instructions include machine language codes formed by compilers, as well as high-level language codes that can be executed by computers using interpreters. The hardware device can be configured to operate as one or more software modules, or vice versa.

[0157] According to the embodiment of the present disclosure, during endoscopic surgery, when esophageal stenosis occurs, when foreign matter sticks to the screen, when gastric fluid is seen on the screen, and when the surgery using the endoscopic device is completed and detached from the esophagus, the image obtained by the image sensor can be automatically classified into the air supply condition, water supply condition and suction condition through the image classification process of the artificial neural network model without the manipulation of the doctor, and the air supply unit, water supply unit and suction unit can be automatically driven based on the classification results.

[0158] The effects of the present invention are not limited to the effects mentioned above.

[0159] Although the embodiments have been described above with reference to the exemplary embodiments and accompanying drawings, those skilled in the art will be able to make various modifications and variations based on the above description. For example, even if the described techniques are performed in a different order than the described methods, and / or the described systems, structures, devices, circuits, and other components are combined in a different manner than the described methods, or are replaced or substituted with other components or equivalents, appropriate results can still be achieved.

[0160] Accordingly, other implementations, other embodiments, and equivalents of the claims are within the scope of the following claims.

[0161] Although certain exemplary embodiments and implementations have been described herein, other embodiments and modifications will be apparent from this description. Therefore, the inventive concept is not limited to these embodiments, but rather to the broader scope of the appended claims, and various obvious modifications and equivalent arrangements will be apparent to those skilled in the art.

Claims

1. A method of controlling an endoscopic device comprising one or more processors, the method comprising: obtaining an image of the interior of the body from an image sensor; In order to classify the obtained image, the image is input into a trained artificial neural network model; The artificial neural network model classifies the image into an air supply condition image, a water supply condition image and / or an inhalation condition image and outputs the image; as well as The endoscope apparatus is controlled to drive an air supply unit, a water supply unit, and / or an air suction unit of the endoscope apparatus according to the output classification result.

2. The method according to claim 1, wherein The artificial neural network model includes a plurality of artificial neural network models, and The plurality of artificial neural network models classify and output at least one of the air supply condition image, the water supply condition image and / or the inhalation condition image.

3. The method according to claim 1, wherein The artificial neural network model classifies the image into an air supply condition image, a water supply condition image and / or an inhalation condition image and performs input, The artificial neural network model is trained to determine an image showing organ stenosis as a gas supply status image when the endoscope apparatus enters a human body.

4. The method according to claim 1, wherein In the process of the artificial neural network model classifying the image into an air supply condition image, a water supply condition image and / or an inhalation condition image and outputting the image, The artificial neural network model is trained to judge an image of impurities adhering to the lens of the endoscope device as a water supply condition image, an air supply condition image, and / or an inhalation condition image.

5. The control method according to claim 1, wherein: In the process of the artificial neural network model classifying the image into an air supply condition image, a water supply condition image and / or an inhalation condition image and outputting the image, The artificial neural network model is trained to identify an image showing the endoscope being removed from an organ after completing an operation or an image showing a substance to be removed from an organ as an inspiration status image.

6. The method according to claim 1, wherein The artificial neural network model includes a convolutional neural network.

7. The method according to claim 6, wherein: The artificial neural network model includes a convolutional layer and a fully connected layer. The convolution layer is configured to extract features of the input internal body image through a convolution operation, and, The fully connected layer is configured to output a classification result as to which condition image the extracted features of the image ultimately correspond to: the air supply condition image, the water supply condition image, and / or the inhalation condition image.

8. The method according to claim 1, wherein The artificial neural network model classifies the image into an air supply condition image, a water supply condition image and / or an inhalation condition image and outputs the image. A threshold is applied to the images classified by the artificial neural network model.

9. The method according to claim 1, wherein The artificial neural network model classifies the image into an air supply condition image, a water supply condition image and / or an inhalation condition image and outputs the image. Confirm the matching degree between the classified image and the classification results of the previous and next images.

10. The method according to claim 1, wherein The artificial neural network model classifies the image into an air supply condition image, a water supply condition image and / or an inhalation condition image and outputs the image. Labeling is performed based on the air supply status image, water supply status image and / or inhalation status image. In this case, the label used for labeling is a multi-label label (Multi-label), in which labels for information about internal body parts in the image are added.

11. An endoscopic device comprising: a memory configured to store images of the interior of a body photographed by the endoscope device; as well as a processor configured to input the internal body image into a trained artificial neural network model, The image is classified into an air supply condition image, a water supply condition image and / or an inhalation condition image and outputted, and The endoscope apparatus is controlled to drive an air supply unit / water supply unit and / or an air suction unit according to the output result of the classification.

12. The endoscopic device according to claim 11, wherein The processor is configured to obtain an image of the inside of the body from the image sensor through the image obtaining unit.

13. The endoscopic device according to claim 11, wherein The processor is configured to transmit a control signal of the control unit to the drive unit, and The driving unit is configured to open and close a valve of an air pump connected to the air supply unit, open and close a valve of a water pump connected to the water supply unit, or open and close a valve of an air suction pump connected to the air suction unit in response to the control signal.

14. The endoscopic device according to claim 11, wherein The processor is configured to perform post-processing on the images classified as the air supply condition image, the water supply condition image and / or the inhalation condition image and outputted through a post-processing unit, and, The post-processing unit comprises: a threshold comparison unit configured to apply a threshold to the classified image; and A matching degree confirmation unit is configured to confirm the matching degree between the classified image and the classification results of the previous and next images. 15 . A computer program configured to be stored in a recording medium to execute the control method according to claim 1 using a computer.

Images