Endoscope scope posture estimation method and device

Abstract

To provide a method and a device for estimating an end part of an endoscope scope.SOLUTION: A method for estimating a posture of an endoscope scope 150 performed by a computing device including at least one processor includes a step which estimates a posture of a scope on the basis of posture information of a drive part 130 controlling movements of the scope by using a backlash model 121 by which a control medium variable is identified with respect to backlash occurring to the scope.SELECTED DRAWING: Figure 2

Description

Technical Field

[0001] The present disclosure relates to a method and an apparatus for estimating the posture of the end of an endoscope scope.

Background Art

[0002] An endoscope is a general term for a medical instrument that observes organs by inserting a scope into the body without performing surgery or autopsy. An endoscope inserts a scope into the human body, irradiates light, and visualizes the light reflected from the surface of the inner wall. Depending on the purpose and the body part, the types of endoscopes are classified, and roughly, they can be classified into a rigid endoscope in which the endoscope tube is formed of metal and a flexible endoscope typified by a gastrointestinal endoscope.

[0003] Since a flexible endoscope device includes various devices inside, it is vulnerable to impact, and the inside of the digestive tract into which the flexible endoscope is inserted is also a very soft tissue and has an irregular shape. In addition, since the shape of the inside of the digestive tract varies from patient to patient, even experienced medical staff may not find the process of inserting the endoscope easy.

[0004] Here, as the endoscope surgery is performed, the endoscope scope is inserted into the digestive tract while being twisted or bent according to the shape of the digestive tract. At the initial stage of the surgery, since the shape of the scope is relatively simple, the operator can bend or move the scope to a desired angle without applying much force. However, as the surgery progresses or when complicated movements are involved during the surgery, controlling the scope while considering the characteristics of the scope that is deformed every moment may cause great operational inconvenience to the operator. In addition, due to the characteristics of the endoscope surgery inserted into the body, fine control of the scope is required, so a technique for controlling the scope by reflecting the physical changes that occur in the endoscope device in the surgical situation is required.

[0005] In order to automatically control an endoscope scope to a desired position and angle, it is first required to accurately grasp the posture of the endoscope scope. However, due to the characteristic that the endoscope scope must be inserted into the body and the thickness of the scope tube is limited, it is difficult to mount various sensors for obtaining posture information. Therefore, a technique for estimating the accurate posture of the scope without attaching a separate sensor to the scope is required.

Summary of the Invention

Problems to be Solved by the Invention

[0006] The present disclosure is for solving the problems of the above-described prior art, and relates to a method and apparatus for estimating the posture of an endoscope scope by modeling the backlash of the endoscope scope or using an artificial neural network model.

[0007] However, the technical problems to be achieved by the present embodiment are not limited to the technical problems as described above, and other technical problems may exist.

Means for Solving the Problems

[0008] According to an embodiment of the present disclosure for realizing the problems as described above, a method for estimating the posture of an endoscope scope performed by a computing device including at least one processor is disclosed. The method includes estimating the posture of the scope based on the posture information of a driving unit that controls the movement of the scope, using a backlash model in which a control medium variable is identified for the backlash generated in the scope.

[0009] As an alternative, the backlash model can be determined by the angle range of the scope.

[0010] As an alternative, the backlash model can be set such that the amount of backlash is constant when the angle of the scope is within a first range.

[0011] As an alternative, the backlash model can be set such that the amount of backlash increases as the angle of the scope increases within a second range.

[0012] As an alternative, the step of estimating the posture of the scope may include a step of determining whether the motor of the driving unit is in a backlash state based on the backlash model and the state of the driving unit.

[0013] As an alternative, in the step of estimating the posture of the scope, it can be estimated that the posture of the scope is constant in the backlash state.

[0014] As an alternative, in the step of estimating the posture of the scope, when not in the backlash state, the bending angle of the scope can be estimated based on the angle of the motor.

[0015] According to an embodiment of the present disclosure for realizing the problems described above, a method for estimating the posture of an endoscope scope, performed by a computing device including at least one processor, is disclosed. The method includes estimating the posture of the scope based on the posture information of the driving unit using an artificial neural network model trained to infer the posture of the scope using the posture information of the driving unit that controls the movement of the scope as learning data.

[0016] As an alternative, the posture information of the driving unit may include the posture information of the motor that provides power to the scope and the direction information of the motor.

[0017] According to an embodiment of the present disclosure for realizing the problems described above, a computing device for estimating the posture of an endoscope scope is disclosed. The device includes a processor including at least one core, and a memory including program code executable by the processor. The processor estimates the posture of the scope based on the posture information of a driving unit that controls the movement of the scope, using a backlash model in which a control medium variable is identified for the backlash generated in the scope.

[0018] According to an embodiment of the present disclosure for realizing the problems described above, a computing device for estimating the posture of an endoscope scope is disclosed. The device includes a processor including at least one core, and a memory including program code executable by the processor. The processor estimates the posture of the scope based on the posture information of the driving unit, using a model learned to infer the posture of the scope with the posture information of the driving unit that controls the movement of the scope as learning data.

Advantages of the Invention

[0019] According to the embodiment of the present disclosure, in the actual situation where the endoscope device operates, since the scope does not move in the backlash section, the risk that the endoscope operator will perform an operation of excessively moving the endoscope scope in the backlash section is reduced, and the endoscope operator can precisely perform the scope operation to the desired degree.

[0020] Also, according to the embodiment of the present disclosure, since various non-linear characteristics that may occur in addition to backlash are reflected in the artificial neural network model learning process, it is possible to estimate the scope posture with higher accuracy compared to estimating the scope posture using only a simple linear model.

Brief Description of the Drawings

[0021]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Mode for Carrying Out the Invention

[0022] Hereinafter, embodiments of the present disclosure will be described in detail with reference to the accompanying drawings so that those having ordinary knowledge in the technical field of the present disclosure (hereinafter referred to as those skilled in the art) can easily implement them. The embodiments presented in the present disclosure are provided so that those skilled in the art can use or implement the content of the present disclosure. Therefore, various modifications to the embodiments of the present disclosure will be apparent to those skilled in the art. That is, the present disclosure can be embodied in various different forms and is not limited to the following embodiments.

[0023] Throughout the specification of the present disclosure, the same or similar reference numerals refer to the same or similar components. Also, for the sake of clarity in explaining the present disclosure, the reference numerals of the parts not related to the description of the present disclosure can be omitted from the drawings.

[0024] The term "or" as used in this disclosure is intended to mean an inclusive "or" rather than an exclusive "or". That is, in this disclosure, unless otherwise specified or the meaning is not clear from the context, "x uses a or b" should be understood to mean one of the natural inclusive substitutions. For example, in this disclosure, unless otherwise specified or the meaning is not clear from the context, "x uses a or b" can be interpreted as either x uses a, x uses b, or x uses both a and b.

[0025] The term "and / or" as used in this disclosure should be understood to include all possible combinations of one or more of the related concepts listed.

[0026] The terms "comprising" and / or "including" as used in this disclosure should be understood to mean that a particular feature and / or component is present. However, the terms "comprising" and / or "including" should be understood not to exclude the presence or addition of one or more other features, other components, and / or combinations thereof.

[0027] In this disclosure, unless otherwise specified or indicating a singular form and the context is not clear, a singular form should generally be interpreted to include "one or more".

[0028] The term "the Nth (N is a natural number)" as used in this disclosure can be understood as an expression used to distinguish the components of this disclosure from each other according to a predetermined criterion such as a functional perspective, a structural perspective, or for the convenience of explanation. For example, in this disclosure, components that perform different functional roles can be distinguished as the first component or the second component. However, components that are substantially the same within the technical idea of this disclosure but need to be distinguished for the convenience of explanation can also be distinguished as the first component or the second component.

[0029] On the one hand, the terms "module" or "unit" used in the present disclosure can be understood as terms indicating an independent functional unit that processes computing resources such as a computer-related entity, firmware, software or a part thereof, hardware or a part thereof, or a combination of software and hardware. Here, a "module" or "unit" may be a unit composed of a single element, or may be a unit represented as a combination or set of multiple elements. For example, as a concept of negotiation, a "module" or "unit" can indicate a hardware element or a set thereof of a computing device, an application program that performs a specific function of software, a processing procedure embodied by the execution of software, or a set of instruction words for the execution of a program. Also, in a broad sense, a "module" or "unit" may indicate the computing device itself that constitutes a system, or an application executed on the computing device. However, since the above concepts are only examples, the "module" or "unit" concept can be defined in various ways within the scope understandable by those skilled in the art based on the content of the present disclosure.

[0030] The term "model" used in the present disclosure can be understood as a system embodied using mathematical concepts and language to solve a specific problem, a set of software units for solving a specific problem, or an abstract model for a processing procedure for solving a specific problem. For example, a neural network "model" can indicate the entire system embodied as a neural network having problem-solving ability through learning. Here, the neural network can have problem-solving ability by optimizing parameters that connect nodes or neurons through learning. A neural network "model" can include a single neural network, or can also include a set of neural networks combined from multiple neural networks.

[0031] The above explanations of terms are for helping understand the present disclosure. Therefore, it should be noted that unless the above terms are explicitly described as limiting matters of the content of the present disclosure, the content of the present disclosure is not used in the sense of limiting the technical idea.

[0032] FIG. 1 is a configuration diagram of an endoscope apparatus according to an embodiment of the present disclosure.

[0033] Referring to FIG. 1, an endoscope apparatus 100 according to an embodiment of the present disclosure can be a flexible endoscope, specifically, a gastrointestinal endoscope. The endoscope apparatus 100 can include a configuration capable of acquiring a medical video of the inside of the digestive tract, and, if necessary, a configuration capable of inserting a tool and performing a treatment or procedure while viewing the medical video.

[0034] The endoscope apparatus 100 can include an output unit 110, a control unit 120, a drive unit 130, a pump unit 140, and a scope 150, and can further include a light source unit (not shown).

[0035] The output unit 110 can include a display for displaying a medical video. The output unit 110 can include a display module capable of outputting visualized information such as a liquid crystal display (LCD), a thin film transistor-driven liquid crystal display (TFTLCD), an organic light-emitting diode (OLED), a flexible display, a three-dimensional display (3D display), etc., or implementing a touch screen.

[0036] The output unit 110 can include various means for providing medical images or information about medical images. The output unit 110 can display the medical images acquired by the scope 150 or the medical images processed by the control unit 120. In addition to visual means, the output unit 110 can provide information via auditory means, and can include, for example, a speaker that audibly provides an alarm for the medical image. On the other hand, in FIG. 2, a single output unit 110 is shown, but the number of output units 110 can be plural. In this case, it is possible to distinguish between the output unit 110 that displays the medical images acquired by the scope 150 and the output unit 110 that displays the information processed by the control unit 120.

[0037] The control unit 120 can control the overall operation of the endoscope device 100. For example, the control unit 120 can perform operations such as a medical image shooting operation by the scope 150, a processing operation of the acquired medical image, a control operation for performing medical operations such as washing water injection and suction, and a series of calculations for controlling the movement of the scope 150. The control unit 120 can include all types of devices that can process data. According to an exemplary embodiment, the control unit 120 can be a data processing device built into hardware that has a physically structured circuit for performing functions represented by codes or instructions included in a program. As an example of a data processing device built into hardware, it can include processing devices such as a microprocessor, a central processing unit (CPU), a processor core, a multiprocessor, an ASIC (application-specific integrated circuit), an FPGA (field programmable gate array), etc., but the technical idea of the present disclosure is not limited thereto.

[0038] The endoscope device 100 of the present invention can include a computing device including a processor and a memory. Exemplarily, the computing device can constitute the control unit 120 of the endoscope device 100. That is, the computing device can be configured to execute the operations of the control unit 120.

[0039] The processor 110 according to an embodiment of the present disclosure can be understood as a constituent unit including hardware and / or software for performing computing operations. For example, the processor can read a computer program and execute data processing for machine learning. The processor can process operation processes such as processing of input data for machine learning, feature extraction for machine learning, and error calculation based on backpropagation. The processor for executing such data processing can include a central processing unit (CPU), a general purpose graphics processing unit (GPGPU), a tensor processing unit (TPU), an application specific integrated circuit (ASIC), or a field programmable gate array (FPGA), etc. Since the above-described types of processors are merely examples, the types of processors can be variously configured within the scope understandable by those skilled in the art based on the content of the present disclosure.

[0040] A memory according to an embodiment of the present disclosure can be understood as a component unit including hardware and / or software for storing and managing data processed by a computing device. That is, the memory can store any form of data generated or determined by the processor and any form of data received by the network unit. For example, the memory can include at least one type of storage medium such as a flash memory type, a hard disk type, a multimedia card micro type, a card type memory, a RAM (random access memory), an SRAM (static random access memory), a ROM (read-only memory), an EEPROM (electrically erasable programmable read-only memory), a PROM (programmable read-only memory), a magnetic memory, a magnetic disk, and an optical disk. In addition, the memory can also include a database system for controlling and managing data in a predetermined system. Since the types of memories described above are only examples, the types of memories can be configured in various ways within the scope understandable by those skilled in the art based on the content of the present disclosure.

[0041] The memory can structure and organize for management data, combinations of data, program code executable by the processor, etc. that are necessary for the processor to execute operations. In addition, the memory can store program code for operating the processor to generate learning data.

[0042] The computing device can further include a network unit for transmitting and receiving data to and from an external computing device, another endoscope device, a hospital server, or the like.

[0043] The control unit 120 can control the movement of the scope 150 via the drive unit 130 connected to the scope 150. That is, the control unit 120 can generate a control signal provided to the drive unit 130 to control the movement of the scope 150.

[0044] Exemplarily, a series of operations for the endoscope device 100 of the present disclosure to control the scope 150 can be executed as follows. The user can input the degree of curvature or the direction of curvature of the scope 150 via the operation unit 151. The input information is transmitted to the control unit 120, and the control unit 120 can process the input information to generate a signal provided to the drive unit 130. For example, the control unit 120 can calculate the position, angle, angular velocity, etc. of the motor corresponding to the degree of curvature or the direction of curvature set by the user and provide them to the drive unit 130. The drive unit 130 can generate power based on the signal of the control unit 120 and transmit it to the scope 150. Therefore, the scope 150 can move or curve corresponding to the value input by the user.

[0045] The endoscope device 100 according to the present disclosure can estimate the posture of the scope 150. Here, the posture of the scope 150 may mean the posture of the insertion portion inserted into the digestive tract as at least a part of the scope 150. The posture of the scope 150 can include at least one of the x-axis position, y-axis position, z-axis position, roll, pitch, yaw values of the end of the scope 150, and the bending angle measured according to a predetermined reference.

[0046] The endoscope device 100 can model the backlash generated at the end of the scope 150 to estimate the posture of the scope 150 and estimate the posture of the scope 150 based on the backlash model.

[0047] In the present disclosure, backlash means a phenomenon in which the force generated by the driving unit 130 is transmitted through the wire inside the scope 150, but the end of the scope 150 does not move. The driving unit 130 generates power so that the endoscope operator can control the posture of the scope 150 to a desired degree by operating, and the generated power is provided to the wire inside the scope 150. Here, backlash may occur where the posture of the scope 150 does not change. Such backlash is a non-linear element that makes it impossible to predict the operation of controlling the scope 150. By modeling the backlash generated at the end of the scope 150 according to the present disclosure and reflecting this in the control of the scope 150, the scope 150 can be controlled more precisely and accurately.

[0048] On the other hand, the endoscope apparatus 100 can train an artificial neural network model to infer the posture of the scope 150 using the posture information of the driving unit 130 that controls the movement of the scope 150 as learning data. Alternatively, the endoscope apparatus 100 can receive an artificial neural network model trained by an external computing device wirelessly or wiredly connected to the endoscope apparatus 100. Alternatively, the endoscope apparatus 100 can provide learning data to an external computing device, provide input data after learning is completed, and receive an inference result.

[0049] Here, the artificial neural network model 200 for inferring the posture of the scope 150 can include at least one neural network. The neural network can include, but is not limited to, network models such as DNN (Deep Neural Network), RNN (Recurrent Neural Network), BRDNN (Bidirectional Recurrent Deep Neural Network), MLP (Multilayer Perceptron), CNN (Convolutional Neural Network), and transformer.

[0050] The artificial neural network model 200 can be trained by supervised learning with the learning data 220 as input values. Alternatively, it can be trained by unsupervised learning to find the criteria for data recognition by learning the types of data necessary for data recognition without any teacher. Alternatively, it can be trained by reinforcement learning that uses feedback on whether the result of data recognition by learning is correct. Specific details of each learning will be described later with reference to FIG. 2.

[0051] Therefore, when considering the actual usage environment where the information obtained from the endoscope scope 150 is limited, the posture of the scope 150 can be estimated using the available information. Also, the operation styles of the endoscope scopes 150 vary among endoscope operators, the irregular twisting of the scopes 150, and the body structures of the patients targeted for endoscope procedures are not uniform and are difficult to predict environments. Therefore, according to the present disclosure, since various non-linear characteristics that can occur other than backlash are reflected in the artificial neural network model learning process, it is possible to estimate the posture of the scope 150 with high accuracy compared to estimating the posture of the scope 150 using only a simple linear model.

[0052] The drive unit 130 can provide the power necessary for the process of inserting the scope 150 into the body or moving while curving inside the body. For example, the drive unit 130 can include a motor connected to a wire inside the scope 150 and a tension adjustment unit that adjusts the tension of the wire.

[0053] The drive unit 130 can control the power of the motor to control the scope 150 in various directions. For example, a plurality of motors can be configured corresponding to the direction in which the insertion part 152 at the end of the scope 150 is to be bent. Alternatively, a plurality of motors can be configured corresponding to the wires inside the scope 150. Specifically, the drive unit 130 can include a first motor that determines the x-axis movement of the scope 150 and a second motor that determines the y-axis movement of the scope 150. By controlling the drive unit 130, the x-axis position, y-axis position, z-axis position, roll, pitch, and yaw values of the end of the scope 150 can be determined, but the configuration of the drive unit 130 is not limited to this.

[0054] The tension adjustment unit can receive power from the motor and pull the wires inside the scope 150 to generate tension. Thereby, the scope 150 can be bent. The tension adjustment unit 330 can adjust the tension acting on the plurality of wires 1000 inside the scope 150 so that the scope 150 can be bent according to the determined amount of curvature and the direction of curvature.

[0055] The pump unit 140 can include at least one of an air pump that injects air into the body through the scope 150, a suction pump that provides negative pressure or vacuum and inhales air from the body through the scope 150, and a water pump that injects cleaning water into the body through the scope 150. Each pump can include a valve for controlling the flow of fluid. The pump unit 140 can be opened and closed by the control unit 120. At least one of the suction pump, the water pump, and the air pump can be opened and closed by a control signal of the computing device or the control of the control unit 120.

[0056] The scope 150 can include an insertion part 152 that is inserted into the digestive tract and an operation part 151 that controls the movement of the insertion part 152 and receives input from the user to perform various operations.

[0057] The insertion part 152 is configured to bend flexibly, and one end is connected to the driving part 130, so that the driving part 130 can determine the degree of curvature or the direction of curvature. Since medical imaging and surgery are performed at the end of the insertion part 152, the scope 150 can include a plurality of cables and tubes extending to the end of the insertion part 152. Inside the scope 150, a light source lens 153, an objective lens 154, a working channel 155, and an air and water channel 156 can be provided. Through the working channel 155, tools for treating and disposing of lesions can be inserted during the endoscopic surgery process. Air can be injected and washing water can be supplied through the air and water channel 156. On the other hand, in FIG. 2, the air and water channel 156 is shown as the passage for supplying washing water, but it is not limited thereto. Exemplarily, a separate water jet channel (not shown) can be provided inside the scope 150, and washing water can also be supplied through the water jet channel.

[0058] On the other hand, the expression described in this specification that the scope 150 is curved by the control part 120 or the driving part 130 may mean that at least a part of the scope 150, for example, the insertion part 152, is curved.

[0059] The operation part 151 can include a plurality of input buttons that provide various functions (such as image shooting, washing water injection, etc.) so that the endoscopic surgeon can control the orientation of the insertion part 152 and perform the surgery through the working channel 155 and the air and water channel 156. For example, the operation part 151 can include a plurality of buttons for indicating the direction of the scope 150 or an input device in the form of a joystick.

[0060] The light source unit can include a light source that irradiates light into the body through the endoscope scope 150. The light source unit can include an illumination device that generates white light, or can include a plurality of illumination devices that generate lights with different wavelength bands. Through the light source unit, the type of light source, the intensity of light, the white balance, etc. can be set. On the other hand, the above-described setting items can also be set through the control unit 120. The light generated by the light source unit can be transmitted to the scope 150 through a path such as an optical fiber.

[0061] FIG. 2 is a block diagram showing a partial configuration of an endoscope apparatus according to an embodiment of the present disclosure, FIG. 3 is a flowchart showing a method for estimating the posture of an endoscope scope according to an embodiment of the present disclosure, FIG. 4 is a graph exemplarily showing a backlash model according to an embodiment of the present disclosure, and FIG. 5 is a flowchart showing a method for estimating the posture of an endoscope scope using an artificial neural network model according to an embodiment of the present disclosure.

[0062] Referring to FIGS. 1 to 5, the control unit 120 of the endoscope apparatus 100 can estimate the posture of the scope 150 based on the posture information of the driving unit 130 that controls the movement of the scope 150 by using the backlash model 121 in which the control medium variable is identified for the backlash generated in the scope 150 (S110).

[0063] Hereinafter, it will be described on the assumption that the control unit 120 generates the backlash model 121, but the modeling work for the backlash can be executed by an external server connected to the endoscope apparatus 100 with or without a wire.

[0064] The control unit 120 can collect the posture information of the scope 150 and the posture information of the motor in order to model the backlash.

[0065] In the graph of FIG. 4, the x-axis represents the posture information of the motor, and the y-axis represents the posture information of the scope 150. The black line represents the actual value, and the gray line represents the estimated value. Here, there is an interval where the posture of the scope 150 does not change even with the change in the posture of the motor, and this interval can be defined as the backlash interval. That is, in the graph, the interval where the y-axis value does not change even when the value on the x-axis changes, the interval that forms a parallel line with respect to the x-axis can be defined as the backlash interval. The backlash interval can become longer as the posture of the motor moves further away from 0. Therefore, the control unit 120 can generate a backlash model 121 having a hysteresis form. Here, the form of the backlash model 121 can be formed in various ways depending on the type of the motor, the type of the scope, and components other than the motor of the drive unit 130, etc.

[0066] The backlash model 121 is determined by the angular range of the scope 150 and can be embodied as a mathematical formula in other forms depending on the angular range. That is, the backlash model 121 can be set such that the backlash amount is constant when the angle of the scope 150 is within the first range, and the backlash amount increases as the angle of the scope 150 increases when the angle of the scope 150 is within the second range. When the angle of the scope 150 is within the second range, the backlash model 121 can be embodied in the form of an nth-degree polynomial function. The degree of the polynomial function and the values of the coefficients constituting the polynomial function can be determined by regression analysis. The control medium variables constituting the backlash model 121 can have their values determined in the graph fitting process by regression analysis.

[0067] The control unit 120 can determine whether the scope 150 is in the backlash state based on the backlash model 121 in which the control intermediate variable is identified by the above-described process and the state of the drive unit 130 (S120). Exemplarily, referring to the graph of FIG. 4, backlash can occur in a section where the moving direction of the motor changes. Therefore, the control unit 120 can determine whether the scope 150 is in the backlash state based on the sign of the moving direction of the motor. Exemplarily, the control unit 120 can define a certain time section in which the sign of the moving direction of the motor changes as the section in the backlash state. Then, the control unit 120 can determine that the scope 150 is not in the backlash state in the remaining section excluding the section in the backlash state.

[0068] The control unit 120 can estimate that the attitude of the scope 150 is constant in the backlash state (S130). That is, in the section in the backlash state, the control unit 120 can determine that the attitude of the scope 150 is the same as the attitude of the scope 150 in the previous time section. Here, the attitude of the scope 150 may mean the bending angle of the scope 150.

[0069] When it is not in the backlash state, the control unit 120 can estimate the bending angle of the scope 150 based on the angle of the motor (S140). Exemplarily, the control unit 120 can estimate the bending angle of the scope 150 in the current time section based on the angle of the motor and the bending angle of the scope 150 in the previous time section. Specifically, using the gradient of the graph of the backlash model 121 in FIG. 4, the bending angle of the scope 150 can be estimated by the following mathematical formula. [Number] JPEG2025084129000003.jpg9112JPEG2025084129000004.jpg9122JPEG2025084129000005.jpg870 means the moving speed of the motor. The gradient m of the graph can be determined by the angle of the motor.

[0070] That is, the control unit 120 can model backlash and, based on this, estimate the posture of the scope 150 according to the state of the motor that provides power to the scope 150. In the actual situation where the endoscope apparatus 100 operates, the state of the motor (for example, the posture information of the motor) can be input into the backlash model 121 in real time according to the movement of the scope 150 of the endoscope. Since the scope 150 does not move in the backlash section, the endoscope operator may perform an operation of excessively moving the endoscope scope 150 in the backlash section. Here, if force is generated by the drive unit 130 by the amount of the input operation amount, rather, the movement of the scope 150 may deviate from the predictable range. The endoscope apparatus 100 according to the present disclosure can accurately perform an operation of the scope 150 to a desired degree by estimating the posture of the scope 150 using the backlash model 121 that takes non-linearity into consideration.

[0071] Referring to FIGS. 2 and 4 together, the control unit 120 can use an artificial neural network model 122 that estimates the posture of the scope 150.

[0072] The control unit 120 can cause the artificial neural network model 122 to learn to infer the posture of the scope 150 using the posture information of the drive unit 130 that controls the movement of the scope 150 as learning data (S210). On the other hand, hereinafter, the case where the artificial neural network model 122 is learned inside the endoscope apparatus 100 will be described as an example, but the artificial neural network model 122 can also be learned by an external server.

[0073] The artificial neural network model 122 can be trained to output the attitude information of the scope 150 through supervised learning, using the attitude information of the scope 150, the attitude information of the motor, the direction information of the motor, etc. as learning data. The learning data can be provided in units of a predetermined time interval. The artificial neural network model 122 can be implemented by a Multi Layer Perceptron (MLP), Vanilla RNN, LSTM, Piecewise regression with decision tree, etc., but is not limited thereto.

[0074] Here, the attitude information of the scope 150 output as time passes after acquiring the learning data may change. That is, instead of receiving information about the backlash of the motor as learning data, the artificial neural network model 122 can be configured and used with time-series data according to the passage of time as the learning data. Thereby, the artificial neural network model 122 can perform learning in consideration of the backlash section of the scope 150. For this purpose, the artificial neural network model 122 can be implemented by models such as RNN and LSTM that learn in consideration of time-series data characteristics.

[0075] The control unit 120 can estimate the attitude of the scope 150 based on the attitude information of the drive unit 130 using the trained artificial neural network model 122 (S220). The control unit 120 can input the attitude information acquired by the drive unit 130 into the artificial neural network model 122 and receive the attitude information of the scope 150 from the artificial neural network model 122.

[0076] After S130 or S140 in FIG. 3 and after S220 in FIG. 5, the control unit 120 can control the scope 150 based on the estimated attitude information of the scope 150. Exemplarily, in order to provide power to the scope 150 or compensate for the generated power based on the estimated attitude information of the scope 150, an operation can be performed on the force to be generated by the drive unit 130.

[0077] The above description of the present disclosure is for illustrative purposes, and it will be understandable to those with ordinary knowledge in the technical field to which the present disclosure belongs that they can easily transform it into other specific forms without changing the technical idea and essential features of the present disclosure. Therefore, it should be understood that the embodiments described above are illustrative in all aspects and not restrictive. For example, each component described as a single type can also be implemented in a distributed manner, and similarly, the components described as distributed can also be implemented in a combined form.

[0078] The scope of the present disclosure is determined by the claims described later rather than the above detailed description, and all changes or modifications derived from the meaning, scope, and equivalent concept of the claims should be construed as being included in the scope of the present disclosure.

Description of Reference Numerals

[0079] 100 Endoscope device 110 Output unit 120 Control unit 121 Backlash model 122 Artificial neural network model 130 Drive unit 140 Pump unit 150 Scope 151 Operation unit 152 Insertion unit 153 Light source lens 154 Objective lens 155 Working channel 156 Air and water channel

Claims

1. 1. A method for estimating a pose of an endoscope scope, the method being performed by a computing device including at least one processor, the method comprising: The method includes a step of estimating an attitude of the scope based on attitude information of a drive unit that controls a movement of the scope, using a backlash model in which a control parameter for backlash generated in the scope is identified.

2. The method of claim 1 , wherein the backlash model is determined by an angular range of the scope.

3. The method of claim 2 , wherein the backlash model is set such that the amount of backlash is constant within a first range of angles of the scope.

4. The method of claim 2 , wherein the backlash model is established such that an amount of backlash increases as the scope angle increases within a second range of the scope angle.

5. The step of estimating the attitude of the scope includes: The method of claim 1 , further comprising determining if the scope is in a backlash condition based on the backlash model and a state of the drive.

6. The method of claim 5 , wherein the step of estimating the scope attitude estimates that the scope attitude is constant under backlash conditions.

7. The method according to claim 5 , wherein the step of estimating the attitude of the scope estimates a bending angle of the scope based on an angle of a motor of the drive unit when there is no backlash state.

8. 1. A method for estimating a pose of an endoscope scope, the method being performed by a computing device including at least one processor, the method comprising: estimating the attitude of the scope based on attitude information of a drive unit that controls movement of the scope using an artificial neural network model trained to infer the attitude of the scope using attitude information of the drive unit as training data.

9. The method of claim 8 , wherein the drive attitude information includes attitude information of a motor that powers the scope and direction information of the motor.

10. 1. A computing device for estimating a pose of an endoscope scope, comprising: A processor including at least one core; a memory containing program code executable by the processor; Including, The processor estimates the attitude of the scope based on attitude information of a drive unit that controls the movement of the scope, using a backlash model in which a control parameter is identified for backlash occurring in the scope.

11. 1. A computing device for estimating a pose of an endoscope scope, comprising: A processor including at least one core; a memory containing program code executable by the processor; Including, The processor estimates the attitude of the scope based on attitude information of a drive unit that controls the movement of the scope, using a model that has been trained to infer the attitude of the scope using attitude information of the drive unit that controls the movement of the scope as learning data.

Images