Endoscope device and method for obtaining size information of lesion

Abstract

The present invention relates to an endoscope device for recognizing a lesion based on a pre-trained model and calculating size information of the recognized lesion, and a method for obtaining size information of a lesion. An embodiment of the present invention discloses a method for obtaining size information of a lesion, comprising a step of obtaining an image of the inside of a body from an image sensor, a step of identifying at least one lesion from the image based on a pre-trained model, and a step of calculating size information of the lesion.

Description

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application claims priority to and the benefit of Korean Patent Application No. 10-2024-0026746, filed on February 23, 2024, which is hereby incorporated by reference for all purposes as if fully set forth herein. Technical Field

[0003] The present invention relates to an endoscopic device and a method for obtaining size information of a lesion. More specifically, it relates to an endoscopic device configured to identify lesions based on a pre-trained model and calculate size information of the identified lesions, and a method for obtaining size information of a lesion. Background Art

[0004] An endoscope is a medical device used to insert an endoscope into the body to observe organs and perform treatment or procedures as needed. When using an endoscope, users often simultaneously manipulate the endoscope and perform diagnostic and treatment procedures to confirm internal abnormalities. When the user's concentration is distracted, they often miss lesions that require observation or treatment due to lack of experience or fatigue.

[0005] When a lesion is found during diagnosis and treatment using an endoscopic device, the treatment method varies depending on the size of the lesion found, so it is necessary to measure the size of the lesion. In the prior art, the size of the lesion was once measured with the naked eye, but this method has the problem of poor accuracy.

[0006] The background technology is technical information that the inventor possesses in order to derive the present invention or is known in the process of deriving the present invention, and is not necessarily known technology disclosed to the general public before the application of the present invention. Summary of the Invention

[0007] The present invention is intended to solve the above-mentioned problem, and its object is to provide an endoscope device configured to identify a lesion based on a pre-trained model and calculate size information of the identified lesion, and a method for obtaining the size information of the lesion.

[0008] However, such problems are merely exemplary, and the technical problems of the present invention are not limited thereto. Problems not mentioned above can be clearly understood by those skilled in the art based on this specification and the accompanying drawings.

[0009] An embodiment of the present invention discloses a method for obtaining size information of a lesion, comprising: obtaining an image of the interior of a body from an image sensor, identifying at least one lesion from the image based on a pre-trained model, and calculating the size information of the lesion.

[0010] According to this embodiment, the method may further include a step of calculating a first distance, and in the step of calculating the size information of the lesion, the size information of the lesion may be calculated based on the first distance.

[0011] According to this embodiment, the step of calculating the first distance may include the step of calculating the first distance based on brightness information of the lesion expressed in the image.

[0012] According to this embodiment, the step of calculating the first distance may include the step of obtaining a first image when the front end portion is at a first angle and a second image when the front end portion is at a second angle, and the step of measuring the difference between the first angle and the second angle and the visual deviation between the first image and the second image.

[0013] According to this embodiment, the step of identifying the lesion may include the step of calculating lesion size information on the image, wherein the lesion size information on the image represents size information of the lesion on the image.

[0014] According to this embodiment, the lesion size information on the image may include a bounding box, and the bounding box is displayed to surround the lesion.

[0015] According to this embodiment, the method may further include the step of controlling the distal end portion so that the distal end portion tracks the lesion.

[0016] According to this embodiment, the viewing angle of the image sensor may include a first area, and the step of controlling the front end portion may include the step of controlling the front end portion so that the lesion is located within the first area.

[0017] According to this embodiment, the step of controlling the front end portion may further include the step of calculating a first value.

[0018] Another embodiment of the present invention discloses an endoscopic device, comprising: a front end portion having an image sensor; and a control unit that obtains an image of the interior of a body from the image sensor, identifies at least one lesion from the image based on a pretrained model, and calculates size information of the lesion.

[0019] According to this embodiment, the control unit may be further configured to calculate a first distance, and may calculate the size information of the lesion based on the first distance.

[0020] According to this embodiment, the control unit may be further configured to calculate the first distance based on brightness information of the lesion expressed in the image.

[0021] According to this embodiment, the control unit can also be configured to obtain a first image when the front end portion is at a first angle and a second image when the front end portion is at a second angle, and measure the difference between the first angle and the second angle and the visual deviation between the first image and the second image.

[0022] According to this embodiment, the control unit may calculate on-image lesion size information, which represents size information of the lesion on the image.

[0023] According to this embodiment, the lesion size information on the image may include a bounding box, and the bounding box is displayed to surround the lesion.

[0024] According to this embodiment, the control unit may be further configured to control the front end portion so that the front end portion tracks the lesion.

[0025] According to this embodiment, the viewing angle of the image sensor may include a first area, and the control unit may be further configured to control the front end portion so that the lesion is located within the first area.

[0026] According to this embodiment, the control unit may be further configured to calculate a first value.

[0027] Other aspects, features, and advantages not mentioned above will become apparent based on the following detailed description, claims, and drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] The above and other aspects, features and advantages of certain embodiments of the present disclosure will become more apparent from the following description taken in conjunction with the accompanying drawings, in which:

[0029] Figure 1 FIG. 1 is a diagram illustrating an endoscope device according to an embodiment of the present invention.

[0030] Figure 2 FIG2 is a diagram illustrating an example of a pre-trained model identifying lesions from images of the interior of a body according to an embodiment of the present invention.

[0031] Figure 3 A diagram showing an image output on a display unit when a lesion is identified.

[0032] Figure 4 is a diagram illustrating a first distance inside a body.

[0033] Figure 5 A diagram illustrating the concept of obtaining lesion size information based on lesion size information on an image.

[0034] Figure 6 FIG. 1 is a diagram showing an image output on a display unit when a lesion is located within a first region.

[0035] Figure 7 A diagram illustrating a method of calculating a first value based on a position difference on an image according to an angular difference of a distal end portion.

[0036] Figure 8 FIG. 1 is a flow chart illustrating a method for obtaining size information of a lesion according to an embodiment of the present invention.

[0037] Figure 9 FIG. 4 is a flowchart of sub-operations in the step of calculating the first distance according to an embodiment of the present invention.

[0038] Figure 10 FIG. 1 is a flow chart illustrating a method for obtaining size information of a lesion according to another embodiment of the present invention.

[0039] Figure 11 Flowchart of sub-operations in the step of controlling the front end portion according to one embodiment of the present invention.

[0040]

Explanation of Figure Numbers

[0041] 1: Endoscopic device

[0042] 110: Display unit 120: Control unit

[0043] 130: Drive unit 140: Operation unit

[0044] 150: Endoscope 151: Insertion part

[0045] 152: Bend portion 153: Front end

[0046] 153a: Image sensor 153b: Lens

[0047] 153c: Lighting

[0048] 200: Model

[0049] M1, M2: Methods for obtaining information on the size of lesions

[0050] LE: lesions

[0051] B: Bounding box

[0052] D: First distance

[0053] S: Lesion size information on the image

[0054] R: Lesion size information

[0055] K: first value DETAILED DESCRIPTION

[0056] The terms used in the present invention are only used to describe specific embodiments and may not be intended to limit the scope of other embodiments. Unless otherwise clearly defined in the context, expressions in the singular may also include expressions in the plural. Including technical terms or scientific terms, the terms used herein may have the same meaning as those generally understood by those of ordinary skill in the art to which the present invention belongs. Among the terms used in the present invention, the terms defined in conventional dictionaries may be interpreted as having the same or similar meaning as the meanings in the context of the relevant technology, and, unless clearly defined in the present invention, should not be interpreted as ideal or overly formal meanings. In some cases, even the terms defined in the present invention cannot be interpreted as excluding embodiments of the present invention.

[0057] Hereinafter, various embodiments will be described in detail with reference to the accompanying drawings so that a person skilled in the art can easily implement the present invention. However, the technical ideas of the present invention can be changed and implemented in various forms and are therefore not limited to the embodiments described in this specification. When describing the embodiments disclosed in this specification, if it is considered that a detailed description of the relevant known technologies may obscure the gist of the technical ideas of the present invention, the detailed description of the known technologies will be omitted. The same reference symbols are given to the same or similar components, and repeated descriptions are omitted.

[0058] At this point, the terms "part" and "unit" used in this embodiment refer to a component that is configured to perform a specific function that is executed by software or hardware such as a field programmable gate array (FPGA) or an application-specific integrated circuit (ASIC). However, the "part" and "unit" are not limited to being executed by software or hardware. The "part" and "unit" can be in the form of data stored in an addressable storage medium or implemented as instructions to cause one or more processors to perform a specific function.

[0059] Software may include a computer program, code, instructions, or a combination of one or more thereof, and may configure a processing device to operate as needed, or may command the processing device independently or collectively. In order for the processing device to interpret or provide instructions or data to the processing device, the software and / or data may be permanently or temporarily embodied in some type of machine, component, physical device, virtual equipment, computer storage medium or device, or transmitted signal wave. The software may be distributed across a computer system connected to a network and may be stored or executed in a decentralized manner. The software and data may be stored in one or more computer-readable recording media. The software may be read into main memory from other devices via other computer-readable media such as a data storage device or a communication interface. The software instructions stored in the main memory may cause the processor to execute a program or step to be described in detail later. Alternatively, fixed wiring circuits may be used instead of or in combination with software instructions to execute programs consistent with the principles of the present invention. Therefore, embodiments consistent with the principles of the present invention are not limited to some specific combination of hardware circuits and software.

[0060] The terms used in this application are only used to describe specific embodiments and are not used for the purpose of limiting the present invention. Unless otherwise clearly defined in the context, expressions in the singular also include expressions in the plural. In this application, terms such as "including" or "having" specify the presence of features, numbers, steps, operations, constituent elements, parts, or combinations thereof recorded in the specification, rather than excluding in advance the presence or additional possibilities of one or more other features, numbers, steps, operations, constituent elements, parts, or combinations thereof. Terms such as first and second can be used to describe various constituent elements, but the constituent elements should not be limited to the terms. The terms are only used to distinguish one constituent element from other constituent elements.

[0061] The 'learning model' referred to in the present invention may include all forms of algorithms or methodologies used to learn or understand specific patterns or structures based on data. That is, learning models may include not only machine learning models such as regression models, decision trees, random forests, support vector machines, K-nearest neighbors, naive Bayes, and clustering algorithms, but also deep learning models such as neural networks, convolutional neural networks, recurrent neural networks, Transformer-based neural networks, generative adversarial networks (GANs), and autoencoders. A 'learning model' may refer to a set of learned parameters or weights that is used to predict or classify the output for a specific input. Such models may be trained using methods such as supervised learning, unsupervised learning, semi-supervised learning, and reinforcement learning. Furthermore, learning models may include not only a single model but also various learning methods and structures, such as ensemble models, multimodal models, and models learned through transfer learning. Such learning models may be pre-trained on a computer device separate from the computer device configured to predict the output for the input and may be used by other computer devices.

[0062] The learning model according to an embodiment of the present invention may include models related to object detection and position estimation.

[0063] Below, refer to Figures 1 to 11 An endoscopic apparatus and a method for obtaining lesion size information according to an embodiment of the present invention are described.

[0064] Figure 1 FIG. 1 is a diagram illustrating an endoscope device 100 according to an embodiment of the present invention.

[0065] The endoscope device 100 is a medical tool configured to insert an endoscope 150 into a body to observe an organ and perform treatment or disposal as needed. Figure 1 The endoscope device 100 may include a display unit 110 , a control unit 120 , a driving unit 130 , an operation unit 140 , and an endoscope 150 .

[0066] The display unit 110 may be configured to output an image. In other words, the display 110 may be configured to output an internal body image obtained from the image sensor 153a described later. The output internal body image may include an x-axis coordinate and a y-axis coordinate (see Figure 3 ).

[0067] The display unit 110 may include a display module capable of outputting visual information or implementing a touch screen, such as a liquid crystal display (LCD), a thin film transistor-liquid crystal display (TFT LCD), an organic light-emitting diode (OLED), a flexible display, a three-dimensional display, etc.

[0068] The display unit 110 may be configured to output on-image lesion size information S described later.

[0069] The control unit 120 can be configured to control the entire operation of the endoscopic device 100. The control unit 120 can include all types of components capable of processing data. According to one embodiment, the control unit 120 may include a data processing device embedded in hardware, which has physically structured circuits to execute functions expressed in codes or instructions included in a program. The data processing device embedded in hardware may include a microprocessor, a central processing unit (CPU), a processor core, a multiprocessor, an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA), etc.

[0070] The control unit 120 may calculate size information R of the identified lesion. Details of the control unit 120 will be described later.

[0071] The drive unit 130 may be configured to provide the power required for inserting the endoscope 150 described later into the body or moving it within the body. For example, the drive unit 130 may include a plurality of motors connected to wires inside the endoscope 150 and a tension adjustment unit configured to adjust the tension of the wires.

[0072] The operation unit 140 may be configured to input a user command. The operation unit 140 may include a plurality of buttons configured to provide various functions to control the angle of the front end portion 153 described later or perform various procedures inside the body.

[0073] The endoscope 150 can be directly inserted into the body. Specifically, the endoscope 50 can include an insertion portion 151 , a bending portion 152 , and a front end portion 153 .

[0074] The insertion portion 151 may be used to insert a front end portion 153 described later into any position inside the body to be observed and treated. The insertion portion 151 may be connected to one end of the operation unit 140.

[0075] The bending portion 152 may be connected to one end of the insertion portion 151. The bending portion 152 may be used to change the angle of the front end portion 153 described later. The bending portion 150 may be flexibly bent. As the bending portion 152 bends, the angle of the front end portion 153 may be changed. The bending portion 152 may be connected to the drive unit 130 and receive the force required to change the angle of the front end portion 153. The degree or direction of the bending of the bending portion 152 may be determined by the drive unit 130.

[0076] The front end portion 153 can be connected to one end of the curved portion 152. The front end portion 153 can image the inside of the body and perform treatment or treatment as needed. The front end portion 153 may include an image sensor 153a, a lens 153b, an illumination 153c, a working channel 153d, and an air / water channel 153e.

[0077] The image sensor 153a can be used to obtain an image of the interior of the body. The image of the interior of the body may include an image composed of a plurality of consecutive frames. The image of the interior of the body may be output through the display unit 110.

[0078] The viewing angle of the image sensor 153a may include a first area. The first area may include the center of the viewing angle. In the image 300 of the interior of the body, the portion 310 corresponding to the first area may include the center of the image 300 (see FIG. Figure 3 ). Therefore, when the object is located within the first area, the object may be close to the center of the image output through the display unit 110.

[0079] According to an embodiment, the number of image sensors 153 a may be plural.

[0080] Lens 153b can serve as a pathway for light reflected from the body's interior to enter image sensor 153a. Illuminating elements 153c can illuminate the body's interior, enabling image sensor 153a to obtain images of the interior. The number of illuminating elements 153c is not particularly limited. Tools for treating and managing lesions LE can be inserted into working channel 153d. Air or washing water can be supplied through air / water channel 153e.

[0081] Hereinafter, the control unit 120 will be described in detail.

[0082] Figure 2 FIG. 1 is a diagram illustrating an example of a pre-trained model 200 identifying a lesion LE from an image A of an interior of a body according to an embodiment of the present invention. Figure 3 3 is a diagram showing an image 300 output on the display unit 110 when the lesion LE is identified.

[0083] The control unit 120 may include a processor configured to perform various calculations or operations described later. The processor may perform data processing for machine learning by interpreting a computer program. The processor may process an operation process, for example, processing input data for machine learning, extracting features for machine learning, performing error calculation based on backpropagation, etc. The processor 101 for performing such data processing may include a central processing unit (CPU), a general purpose graphics processing unit (GPGPU), a tensor processing unit (TPU), an application specific integrated circuit (ASIC), or a field programmable gate array (FPGA). However, this is merely an example, and the type of processor may have various configurations within the scope that can be understood by a person of ordinary skill in the art based on the present content.

[0084] The control unit 120 may obtain an image of the inside of the body from the image sensor 153 a , identify at least one lesion LE from the image based on the pre-trained 200 , and calculate lesion size information R.

[0085] The control unit 120 may pre-train the model. The model may be trained to receive images of the interior of the body, identify lesions LE from the images, and calculate lesion size information S on the images. The model may receive images of the interior of the body as training data. Furthermore, the model may receive label data corresponding to each image of the interior of the body, representing the location of the lesion using a bounding box.

[0086] The model can include network structures such as Fully Convolutional Network (FCN), Conditional Adversarial Network (CAN), Recurrent Neural Network (RNN) or Matching Cost-CNN (MC-CNN).

[0087] Reference Figure 2 , the control unit 120 can identify the lesion LE from the image A based on the pre-trained model 200. The control unit 120 can input the image A of the inside of the body obtained from the image sensor 153a into the pre-trained model 200. The pre-trained model 200 inputted with the image A of the inside of the body can identify the lesion LE, calculate and output the lesion size information S on the image. The lesion size information S on the image can be information on the size of the lesion LE on the image outputted on the display unit 110. According to one embodiment, as Figure 2 As shown, the lesion size information S on the image may include a bounding box B, which may be represented as surrounding the lesion LE.

[0088] For example, Figure 2 As shown, when an image A1 of the interior of the body is input to the pre-trained model 200, the pre-trained model 200 can identify the lesion LEa included in the image A1 of the interior of the body and output a bounding box Ba surrounding the lesion LEa. When an image A2 of the interior of the body is input to the pre-trained model 200, there is no lesion in the image A2 of the interior of the body, so the pre-trained model 200 does not output a bounding box. When an image A3 of the interior of the body is input to the pre-trained model 200, the pre-trained model 200 can identify the lesion LEb included in the image A3 of the interior of the body and output a bounding box Bb surrounding the lesion LEb.

[0089] According to one embodiment, the lesion size information S on the image may include image coordinate information about four line segments of the bounding box B. That is, the pre-trained model 200 may calculate the image coordinate information about the four line segments of the bounding box B.

[0090] For example, refer to Figure 3 As the lesion size information S on the image, the pre-trained model 200 can calculate the x-coordinate x1 of the left segment of the bounding box B, the x-coordinate x2 of the right segment of the bounding box B, the y-coordinate y1 of the bottom segment of the bounding box B, and the y-coordinate y2 of the top segment of the bounding box B. The horizontal size xw of the bounding box B is the value obtained by subtracting the x-coordinate x1 of the left segment of the bounding box B from the x-coordinate x2 of the right segment of the bounding box B (xw = x2 - x1). The vertical size yh of the bounding box B is the value obtained by subtracting the y-coordinate y1 of the bottom segment of the bounding box B from the y-coordinate y2 of the top segment of the bounding box B (yh = y2 - y1). The lesion size information S on the image can include the horizontal size xw of the bounding box B and the vertical size yh of the bounding box B.

[0091] Figure 4 is a diagram showing a first distance D inside the body.

[0092] The control unit 120 may calculate a first distance D. The first distance D may be a value required to calculate the lesion size information R. Specifically, the first distance D may serve as a medium connecting the numerical information on the image with the numerical information actually inside the body. The first distance D may include the distance between the lesion LE and the front end portion 153. More specifically, the first distance D may include the distance between the lesion LE and the lens 153b.

[0093] In one example, the control unit 120 can calculate the first distance based on the brightness information of the lesion LE shown in the image. The control unit 120 may pre-store the distance from the distal end portion 153 to any object inside the body, as well as the brightness data corresponding to the distance, as reflected by the illumination 153c on the image of the object inside the body. Therefore, by substituting the brightness of the lesion LE in the image into this data, the distance from the distal end portion 153 to the lesion LE, i.e., the first distance D, can be calculated.

[0094] According to an embodiment, the first distance D may be calculated by averaging. Specifically, the first distance D may be calculated by averaging distances obtained by respectively substituting the brightness of a plurality of locations within the lesion LE into the data.

[0095] According to another embodiment, the control unit 120 can obtain a first image i1 when the front end portion 153 is at a first angle a and a second image i2 when the front end portion 153 is at a second angle b, and measure the difference Δθ between the first angle a and the second angle b and the visual deviation ΔL between the first image i1 and the second image i2.

[0096] The first angle a and the second angle b are two arbitrary angles different from each other for calculating the first distance D, and may refer to the angle between the front end portion 153 and the insertion portion 151, which angle is formed as the bending portion 152 bends. As the angle of the front end portion 153 changes, the position of the image sensor 153a may change, and as the position of the image sensor 153a changes, the position of the arbitrary location W on the image may appear different. Visual disparity may refer to the degree to which the position of the arbitrary location W on the image appears different as the position of the image sensor 153a changes. Specifically, the control unit 120 may calculate the first distance D using the following [Formula 1].

[0097] [Formula 1]

[0098]

[0099] In Equation 1, Δθ is the difference between the first angle a and the second angle b, and ΔL is the visual deviation between the first image i1 and the second image i2. R is the rotation radius of the curved portion 152 when it is bent. Therefore, RΔθ represents the displacement of the image sensor 153a. f is the focal length of the lens 153b.

[0100] According to another embodiment, the control unit 120 may calculate the first distance D using a stereo method based on a plurality of image sensors 153 a .

[0101] According to another embodiment, the control unit 120 may calculate the first distance D by using light with a specific pattern such as stripes or grids as structured light to illuminate the lesion LE, and measuring distortion of the light pattern returning from the lesion LE to the image sensor 153a.

[0102] As another embodiment, the control unit 120 may calculate the first distance D by irradiating light onto the lesion LE and measuring the moving time of light reflected from the lesion LE.

[0103] Figure 5 FIG2 is a diagram illustrating the concept of obtaining lesion size information R based on lesion size information S on an image.

[0104] The control unit 120 may calculate the lesion size information R based on the first distance D. The control unit 120 may calculate the size information based on the lesion size information S and the first distance D on the image.

[0105] According to one embodiment, the control unit 120 may calculate the lesion size information R using a regression model and a ratio of the size of a pixel in the image to the actual size corresponding to the location at the first distance D from the front end portion 153. Figure 3 and Figure 5 The number of pixels corresponding to the size information of the bounding box B on the image surrounding the lesion LE can be multiplied by the ratio to obtain the lesion size information R. Specifically, the ratio can be multiplied by the horizontal size xw of the bounding box B to calculate the actual horizontal size RW of the lesion. The ratio can be multiplied by the vertical size yh of the bounding box B to calculate the actual vertical size KH of the lesion.

[0106] According to another embodiment, the control unit 120 may use the first distance D and the focal length f of the lens 153b to calculate the lesion size information R. The focal length f of the lens 153b, the sizes xw and yh of the bounding box B on the image, the first distance D, and the actual sizes RW and RH of the lesion can be expressed by the following proportional expression [Equation 2].

[0107] [Formula 2]

[0108] Xw f=RW:D

[0109] yh f=RH:D

[0110] Therefore, the actual lesion sizes RW and RH included in the lesion size information R can be calculated using the following [Formula 3] obtained by modifying the above-mentioned [Formula 2].

[0111] [Formula 3]

[0112]

[0113] Figure 6 3 is a diagram illustrating an image 300 ′ output on the display unit 110 when the lesion LE is located in the first area. Figure 7 The method of calculating the first value K based on the position difference on the image according to the angle difference of the front end portion 153 is specifically shown.

[0114] The control unit 120 can control the front end portion 153 so that the front end portion 153 tracks the lesion LE. Specifically, the control unit 120 can control the front end portion 153 so that the lesion LE is located in the first area, such as Figure 6 That is, by controlling the front end portion 153 so that the identified lesion LE is located within the first area, the identified lesion LE can be located near the center of the image 300 ′ output through the display unit 110 . As a result, the identified lesion LE can be continuously output on the display unit 110 .

[0115] According to an embodiment, the center coordinates of the bounding box B may be calculated based on the on-image coordinate information about the four line segments of the bounding box B. Figure 3 The center coordinates (x0, y0) of the bounding box B of the lesion LE identified by the pre-trained model 200 in the internal body image 300 output on the display unit 110 can be calculated as shown in the following [Equation 4] based on the on-image coordinate information about the four line segments of the bounding box B. The center coordinates (x0, y0) of the bounding box B can be referred to as the center coordinates of the identified lesion LE.

[0116] [Formula 4]

[0117]

[0118] The control unit 120 can calculate the first value K, calculate the total movement angle TMA of the front end portion based on the center coordinates (x0, y0) of the lesion on the image and the first value K, calculate the target angle ag at the current time considering the movement angular velocity as and control period cp of the front end portion 153, and move the front end portion 153 at the target angle ag at the current time.

[0119] The control unit 120 may calculate a first value K. The first value K is a ratio of the position difference on the image to the angle change of the front end portion 153. The first value K may be a value required to calculate the total movement angle TMA of the front end portion 153.

[0120] According to one embodiment, referring to Figure 7 The control unit 120 can obtain a third image i3 when the front end portion 153 is at a third angle c and a fourth image i4 when the front end portion 153 is at a fourth angle d, and measure the difference Δδ between the third angle c and the fourth angle d and the position difference ΔX between the third image i3 and the fourth image i4.

[0121] The fourth angle c and the fourth angle d are two different arbitrary angles used to calculate the first value K. They may refer to the angles between the distal end portion 153 and the insertion portion 151, which are formed as the bending portion 152 bends. In this case, the position difference ΔX between the third image i3 and the fourth image i4 may include a coordinate difference on the images for an arbitrary location X inside the body. That is, it may include the difference between the coordinates of the arbitrary location X on the third image i3 and the coordinates of the arbitrary location X on the fourth image i4.

[0122] Reference Figure 7 , the control unit 120 can use the image sensor 153a to obtain a third image i3 by manipulating the bending portion 152 so that the front end portion 153 is at a third angle c. In this case, the coordinates of an arbitrary location X inside the body on the image may be (xc, yc). The method for specifying the arbitrary location X is not particularly limited. According to one embodiment, the arbitrary location X may be the location of a blood vessel of a specific shape found inside the body on the image.

[0123] When the bending portion 152 is manipulated again to make the distal end portion 153 at the fourth angle d, the control unit 120 may use the image sensor 153a to obtain a fourth image i4. At this time, the coordinates of any point X inside the body on the image may be (xd, yd).

[0124] The coordinate difference between the third image i3 and the fourth image i4 can be expressed as (Δx, Δy), which is the coordinate difference of any location X on the image. In this case, Δx = xd - xc, and Δy = yd - yc. Based on this, the position difference ΔX between the third image i3 and the fourth image i4 is expressed as [Equation 5] below.

[0125] [Formula 5]

[0126]

[0127] The first value K can be calculated by substituting the difference Δδ between the third angle c and the fourth angle d and the position difference ΔX between the third image i3 and the fourth image i4 into the following [Formula 6].

[0128] [Formula 6]

[0129]

[0130] According to one embodiment, in the step of calculating the first value (S241), the first image i1 and the second image i2 obtained in the step of calculating the first distance (S130) can be used. Specifically, the difference Δθ between the first angle a and the second angle b is used as the difference Δδ between the third angle c and the fourth angle d. In addition, the coordinates (x ) on the first image i1 of the arbitrary location W can be used. a ,y a ) and the difference between the coordinates (xb, yb) on the second image i2 at the arbitrary location W is taken as the position difference ΔX between the third image i3 and the fourth image i4.

[0131] The control unit 120 can calculate the total movement angle TMA of the front end portion based on the center coordinates (x0, y0) of the image lesion and the first value K. The total movement angle TMA refers to the total angle that the front end portion 153 must move to move the center coordinates (x0, y0) of the lesion LE to the center coordinates (0, 0) on the image. The length Q from the center of the lesion LE to the center on the image is as shown in the following [Equation 7].

[0132] [Formula 7]

[0133]

[0134] The total movement angle TMA can be calculated by substituting the length Q from the center of the lesion LE to the center on the image and the calculated first value K into the following [Formula 8].

[0135] [Formula 8]

[0136]

[0137] The control unit 120 may calculate the target angle ag at the current time in consideration of the movement speed as and the control period cp of the front end portion 153. The movement angular speed as and the control period cp of the front end portion 153 may be set in advance.

[0138] According to one embodiment, the current target angle ag can be calculated using multiple nominal trajectories. As a specific example, the front end 153 can be configured to move at a constant speed through the total movement angle TMA. The preset angular velocity as of the front end 153 can be 30° / second, and the preset control period cp can be 2 ms. In this case, the current target angle ag can be 30° / s × 2 ms = 0.2°. However, this is merely an illustrative embodiment and the present disclosure is not limited thereto.

[0139] According to another embodiment, the current target angle ag can be calculated using a Bezier curve trajectory. That is, the method for calculating the current target angle ag is not limited to multiple nominal trajectories. In other words, the current target angle ag can be calculated using various trajectories that can be devised by a person of ordinary skill.

[0140] The control unit 120 may move the front end portion 153 by the current target angle ag. The control unit 120 may calculate the force required to move the front end portion 153 by the current target angle ag, taking into account the dynamic characteristics of the bending portion 152. The control unit 120 may move the front end portion 153 by the current target angle ag by transmitting information about the calculated force to the driving unit 130.

[0141] like Figure 6 As shown, the control unit 120 can repeatedly perform the process of identifying the lesion, calculating the first distance D, and controlling the front end 153 until the lesion LE is located in the first area. In this way, the front end 153 can track the lesion LE, and the lesion LE can be continuously output on the display unit 110.

[0142] By controlling the distal end portion 153 so that the identified lesion LE is located within the first region, the lesion LE is located near point C, the center of the image sensor 153a's field of view. This improves the convenience and accuracy of calculating the first distance D. This also enhances the reliability of the lesion size information R. Furthermore, even users with limited experience can easily track and observe the lesion LE, allowing them to focus on observing, treating, and managing the identified lesion LE without worrying about endoscope manipulation, thereby improving the quality of diagnosis and treatment.

[0143] Figure 8 FIG. 1 is a flow chart illustrating a method M1 for obtaining lesion size information according to an embodiment of the present invention.

[0144] The method M1 for obtaining lesion size information is a method for recognizing a lesion LE in an image of the interior of a body obtained from the endoscope apparatus 100 and obtaining size information of the recognized lesion.

[0145] Reference Figure 8, the method M1 for obtaining the size information of a lesion may include a step of obtaining an internal body image from an image sensor 153a (S110), a step of identifying at least one lesion LE from the image based on a pre-trained model 200 (S120), a step of calculating a first distance D (S130), and a step of calculating lesion size information R (S140).

[0146] Endoscope device 100 can obtain an image A of the interior of the body from image sensor 153a (S110). Endoscope 150 can be inserted into the body and obtain an image of the interior of the body through image sensor 153a provided in distal end portion 153. The image of the interior of the body can include an image composed of a plurality of consecutive frames. The image of the interior of the body can be output via display unit 110.

[0147] Reference Figure 2 , the endoscopic device 100 can identify the lesion LE from the image A based on the pre-trained model 200 (S120). The endoscopic device 100 can input the image A of the inside of the body obtained from the image sensor 153a into the pre-trained model 200. The pre-trained model 200 having the image A of the inside of the body inputted thereto can identify the lesion LE, calculate and output the lesion size information S on the image. The lesion size information S on the image can be information indicating the size of the lesion LE on the image outputted on the display unit 110. According to one embodiment, as Figure 2 As shown, the lesion size information S on the image may include a bounding box B surrounding the lesion LE.

[0148] For example, Figure 2 As shown, when an image A1 of the interior of the body is input to the pre-trained model 200, the pre-trained model 200 can identify the lesion LEa included in the image A1 of the interior of the body and output a bounding box Ba surrounding the lesion LEa. When an image A2 of the interior of the body is input to the pre-trained model 200, there is no lesion in the image A2 of the interior of the body, so the pre-trained model 200 does not output a bounding box. When an image A3 of the interior of the body is input to the pre-trained model 200, the pre-trained model 200 can identify the lesion LEb included in the image A3 of the interior of the body and output a bounding box Bb surrounding the lesion LEb.

[0149] The model can be trained to receive images of the interior of the body, identify lesions LE from the images of the interior of the body, and calculate lesion size information S on the images. The model can receive images of the interior of the body as training data. Furthermore, the model can receive label data corresponding to each image of the interior of the body, which represents the location of the lesion using a bounding box.

[0150] The model can include network structures such as Fully Convolutional Network (FCN), Conditional Adversarial Network (CAN), Recurrent Neural Network (RNN) or Matching Cost-CNN (MC-CNN).

[0151] According to one embodiment, the lesion size information S on the image may include image coordinate information about four line segments of the bounding box B. That is, the pre-trained model 200 may calculate the image coordinate information about the four line segments of the bounding box B.

[0152] For example, refer to Figure 3 As the lesion size information S on the image, the pre-trained model 200 can calculate the x-coordinate x1 of the left segment of bounding box B, the x-coordinate x2 of the right segment of bounding box B, the y-coordinate y of the bottom segment of bounding box B, and the y-coordinate y2 of the top segment of bounding box B. The horizontal size xw of bounding box B is the value obtained by subtracting the x-coordinate x1 of the left segment of bounding box B from the x-coordinate x2 of the right segment of bounding box B (xw = x2 - x1). The vertical size yh of bounding box B is the value obtained by subtracting the y-coordinate y1 of the bottom segment of bounding box B from the y-coordinate y2 of the top segment of bounding box B (yh = y2 - y1). The lesion size information S on the image can include the horizontal size xw of bounding box B and the vertical size yh of bounding box B.

[0153] Figure 9 4 is a flowchart of sub-operations in the step (S130) of calculating the first distance according to an embodiment of the present invention.

[0154] The endoscope device 100 may calculate a first distance D (S130). The first distance D may be a value required to calculate the lesion size information R. Specifically, the first distance D may serve as a medium connecting numerical information on the image with numerical information actually inside the body. The first distance D may include the distance between the lesion LE and the front end portion 153. More specifically, the first distance D may include the distance between the lesion LE and the lens 153b.

[0155] According to one embodiment, referring to Figure 9Calculating the first distance (S130) may include calculating the first distance based on the brightness of the lesion LE represented in the image (S131). Data may be pre-stored regarding the distance from the distal end portion 153 to any object within the body, along with corresponding brightness data for the image of the lesion LE reflected by the illumination 153c. Therefore, by substituting the brightness of the lesion LE in the image into this data, the distance from the distal end portion 153 to the lesion LE, i.e., the first distance D, can be calculated.

[0156] According to an embodiment, the first distance D may be calculated by averaging. Specifically, the first distance D may be calculated by averaging distances obtained by respectively substituting the brightness of a plurality of locations within the lesion LE into the data.

[0157] According to another embodiment, referring to Figure 2 The step of calculating the first distance (S130) may include a step of calculating a first image i1 when the front end portion 153 is at a first angle a and a second image i2 when the front end portion 153 is at a second angle b (S132a), and a step of measuring a difference Δθ between the first angle a and the second angle b and a visual deviation ΔL between the first image i1 and the second image i2 (S132b).

[0158] The first angle a and the second angle b are two different arbitrary angles used to calculate the first distance D, and can refer to the angle between the front end portion 153 and the insertion portion 151, which is formed as the bending portion 152 bends. As the angle of the front end portion 153 changes, the position of the image sensor 153a can change, and as the position of the image sensor 153a changes, the position of the arbitrary location W seen on the image may appear different. Visual disparity can refer to the degree to which the position of the arbitrary location W seen on the image appears different as the position of the image sensor 153a changes. Specifically, the first distance D can be calculated using [Equation 1].

[0159] According to another embodiment, the first distance D may be calculated using a stereo method based on a plurality of image sensors 153 a .

[0160] According to another embodiment, the first distance D may be calculated by using light of a specific pattern such as stripes or grids as structured light to illuminate the lesion LE, and measuring distortion of the light pattern returning from the lesion LE to the image sensor 153a.

[0161] According to another embodiment, the first distance D may be calculated by measuring the travel time of light reflected from the lesion LE after irradiating light onto the lesion LE.

[0162] The endoscope apparatus 100 may calculate lesion size information R based on the first distance D ( S140 ). The size information may be calculated based on the lesion size information S and the first distance D on the image.

[0163] According to one embodiment, the endoscope apparatus 100 may calculate the lesion size information R using a regression model and a ratio of the size of one pixel in the image to the actual size corresponding to the location at the first distance D from the front end portion 153. Figure 3 and Figure 5 The number of pixels corresponding to the size information of the bounding box B on the image surrounding the lesion LE can be multiplied by the ratio to obtain the lesion size information R. Specifically, the ratio can be multiplied by the horizontal size xw of the bounding box B to calculate the actual horizontal size RW of the lesion. The ratio can be multiplied by the vertical size yh of the bounding box B to calculate the actual vertical size RH of the lesion.

[0164] According to another embodiment, the endoscope device 100 can use the first distance D and the focal length f of the lens 153b to calculate the lesion size information R. The focal length f of the lens 153b, the sizes xw and yh of the bounding box B on the image, the first distance D, and the actual sizes RW and RH of the lesion can be expressed by the proportional expression of [Equation 2].

[0165] Therefore, the actual lesion sizes RW and RH included in the lesion size information R can be calculated using the [Formula 3] obtained by modifying the [Formula 2].

[0166] Figure 10 FIG. 1 is a flow chart illustrating a method M2 for obtaining lesion size information according to another embodiment of the present invention. Figure 11 15. FIG. 15 is a flowchart of sub-operations in the step (S240) of controlling the front end portion 153 according to an embodiment of the present invention.

[0167] 10 , the method M2 for obtaining lesion size information may include the steps of obtaining an image A of the interior of the body from the image sensor 153a (S210), identifying at least one lesion LE from the image A based on the pre-trained model 200 (S220), calculating a first distance D (S230), controlling the front end portion 153 so that the front end portion 153 tracks the lesion LE (S240), and calculating the size information of the lesion LE (S250). The steps of obtaining the image (S210), identifying the lesion (S220), calculating the first distance (S230), and calculating the size information of the lesion LE (S250) are the same or similar to those described in the method M1 for obtaining lesion size information above, and therefore detailed descriptions are omitted. The description will focus on the distinguishing features.

[0168] The endoscope device 100 may control the front end portion 153 so that the front end portion 153 tracks the lesion LE (S240). The step of controlling the front end portion (S240) may include controlling the front end portion so that the lesion LE is located in the first area.

[0169] According to an embodiment, the center coordinates of the bounding box B may be calculated based on the on-image coordinate information about the four line segments of the bounding box B. Figure 3 The center coordinates (x0, y0) of the bounding box B of the lesion LE identified by the pre-trained model 200 in the internal body image 300 output on the display unit 110 can be calculated as described in [Equation 4] based on the image coordinate information of the four line segments of the bounding box B. The center coordinates (x0, y0) of the bounding box B can be referred to as the center coordinates of the identified lesion LE.

[0170] The viewing angle of the image sensor 153a may include a first area. The first area may include the center of the viewing angle. Figure 6 , the portion 310 corresponding to the first region in the image 300 ′ inside the body may include the center of the image 300 ′. Therefore, when an object is located within the first region, the object may be close to the center of the image output through the display unit 110 .

[0171] That is, by controlling the front end portion 153 so that the identified lesion LE is located within the first area, the identified lesion LE can be close to the center of the image output through the display unit 110. As a result, the identified lesion LE can be continuously output on the display unit 110.

[0172] Reference Figure 11 The step of controlling the front end portion (S240) may include a step of calculating a first value K (S241), calculating a total movement angle TMA of the front end portion based on the center coordinates (x0, y0) of the lesion on the image and the first value K (S242), a step of calculating a target angle ag at the current time considering the movement angular velocity as and the control period cp of the front end portion (S243), and a step of moving the front end portion 153 to the target angle ag at the current time (S244).

[0173] The endoscope device 100 may calculate a first value K (S241). The first value K is the ratio of the position difference on the image to the angle change of the distal end portion 153. The first value K is a value required to calculate the total movement angle TMA of the distal end portion 153.

[0174] According to one embodiment, the step of calculating the first value (S241) may include the steps of obtaining a third image i3 when the front end portion 153 is at a third angle c and a fourth image i4 when the front end portion 153 is at a fourth angle d, and the steps of measuring the difference Δδ between the third angle c and the fourth angle d and the position difference ΔX between the third image i3 and the fourth image i4.

[0175] The fourth angle c and the fourth angle d are two different arbitrary angles used to calculate the first value K. They may refer to the angle between the distal end portion 153 and the insertion portion 151, which is formed as the bending portion 152 bends. In this case, the position difference ΔX between the third image i3 and the fourth image i4 may include the difference in the image coordinates of an arbitrary location X inside the body. That is, it may include the difference between the coordinates of the arbitrary location X on the third image i3 and the coordinates of the arbitrary location X on the fourth image i4.

[0176] Reference Figure 7 When the bending portion 152 is manipulated so that the distal end portion 153 has a third angle c, the image sensor 153a can be used to obtain a third image i3. In this case, the coordinates of an arbitrary location X inside the body on the image can be (xc, yc). The method for specifying the arbitrary location X is not particularly limited. According to one embodiment, the arbitrary location X can be the location of a blood vessel of a specific shape found inside the body on the image.

[0177] When the bending portion 152 is manipulated again to make the distal end portion 154 have a fourth angle d, the image sensor 153a can be used to obtain a fourth image i4. At this time, the coordinates of any point X inside the body on the image can be (xd, yd).

[0178] The coordinate difference between the third image i3 and the fourth image i4 can be expressed as (Δx, Δy), which is the difference between the coordinates of any location X on the image. In this case, Δx = xd - xc, and Δy = yd - yc. Based on this, the position difference ΔX between the third image i3 and the fourth image i4 is expressed as follows [Equation 5].

[0179] The first value K can be calculated by substituting the difference Δδ between the third angle c and the fourth angle d and the position difference ΔX between the third image i3 and the fourth image i4 into the above [Equation 6].

[0180] According to one embodiment, in the step of calculating the first value (S241), the first image i1 and the second image i2 obtained in the step of calculating the first distance (S130) can be used. Specifically, the difference Δθ between the first angle a and the second angle b is used as the difference Δδ between the third angle c and the fourth angle d. In addition, the coordinates (x ) on the first image i1 of the arbitrary location W can be used. a ,y a) and the coordinates (x b ,y b ) as the position difference ΔX between the third image i3 and the fourth image i4.

[0181] The endoscope apparatus 100 can calculate the total movement angle TMA of the distal end portion based on the center coordinates (x0, y0) of the lesion on the image and the first value K (S242). The total movement angle TMA is the total angle that the distal end portion 153 must move to move the center coordinates (x0, y0) of the lesion LE to the center coordinates (0, 0) on the image. The length Q from the center of the lesion LE to the center on the image is as described in [Equation 7].

[0182] The total movement angle TMA can be calculated by substituting the length Q from the center of the lesion LE to the center on the image and the calculated first value K into the [Equation 8].

[0183] The endoscope apparatus 100 can calculate the target angle ag at the current time (S243) in consideration of the movement angular velocity as and the control cycle cp of the distal end portion 153. The movement angular velocity as and the control cycle cp of the distal end portion 153 can be set in advance.

[0184] According to one embodiment, the current target angle ag can be calculated using multiple nominal trajectories. As a specific example, the front end 153 can be set to move at a constant speed through a total movement angle TMA. The preset angular velocity as of the front end 153 can be 30° / second, and the preset control period cp can be 2 ms. In this case, the current target angle ag can be 30° / s × 2 ms = 0.2°. However, this is merely an illustrative embodiment and the present disclosure is not limited thereto.

[0185] According to another embodiment, the current target angle ag can be calculated using a Bezier curve trajectory. That is, the method for calculating the current target angle ag is not limited to multiple nominal trajectories. In other words, the current target angle ag can be calculated using various trajectories that can be devised by a person of ordinary skill.

[0186] The endoscope device 100 may move the distal end portion 153 to the current target angle ag (S244). The endoscope device 100 may calculate the force required to move the distal end portion 153 to the current target angle ag, taking into account the dynamic characteristics of the bending portion 152. The endoscope device 100 may move the distal end portion 153 to the current target angle ag by transmitting information about the calculated force to the drive unit 130.

[0187] The steps of identifying the lesion ( S220 ), calculating the first distance ( S230 ), and controlling the front end ( S240 ) may be repeated until the lesion LE is located in the first area. Thus, the front end 153 can track the lesion LE, and the lesion LE can be continuously output on the display unit 110 .

[0188] By further including the step of controlling the front end portion ( S242 ), the lesion LE is brought closer to the point C located at the center of the viewing angle of the image sensor 153 a , thereby enhancing the convenience and accuracy of calculating the first distance D. This improves the reliability of the lesion size information R. Furthermore, even users with limited experience can easily track and observe the lesion, allowing the user to focus on observing, treating, and managing the identified lesion without being distracted by endoscope manipulation, thereby improving the quality of diagnosis and treatment.

[0189] In the endoscope apparatus and the method for obtaining lesion size information according to an embodiment of the present invention, by identifying lesions using a pre-trained model, lesions inside the body can be discovered without being affected by the user's convergence or fatigue.

[0190] In an endoscopic device and a method for obtaining lesion size information according to an embodiment of the present invention, the endoscopic device automatically calculates the size information of the identified lesion, so that the lesion size information can be obtained correctly and consistently without deviation based on user experience or proficiency, and an appropriate treatment method for the lesion can be selected based on this, thereby improving the quality of diagnosis and treatment.

[0191] The effects of the present invention are not limited to the effects mentioned above, and other effects not mentioned should be clearly understood by those skilled in the art in the technical field to which the present invention belongs based on this specification and the accompanying drawings.

[0192] While the present invention has been described above with reference to the embodiments shown in the accompanying drawings, such descriptions are intended to be illustrative only. Those skilled in the art will appreciate that various modifications and equivalent embodiments may be made based on the embodiments. Therefore, the true scope of the present invention should be determined by the appended claims.

[0193] The specific technical contents described in the embodiments are only some embodiments and do not limit the technical scope of the embodiments. In order to concisely and clearly record the content of the invention, the description of existing conventional technologies and structures may be omitted. In addition, the line connections or connecting parts between the constituent elements shown in the drawings only illustrate functional connections and / or physical or circuit connections by way of example. In actual devices, they may be expressed as various functional connections, physical connections, or circuit connections that can be replaced or added. In addition, unless otherwise specifically mentioned as "necessary", "important", etc., it may not be an essential constituent element for the application of the present invention.

[0194] The "said" or similar referents recorded in the summary of the invention and the claims, unless otherwise specified, can refer to the singular and plural. Furthermore, if a range is recorded in the embodiment, including the invention of the application of individual values ​​falling within the range (unless otherwise stated to the contrary), this is equivalent to recording the individual values ​​constituting the range in the summary of the invention. Furthermore, for the steps constituting the method according to the embodiment, unless the order is clearly stated or otherwise stated to the contrary, the steps can be performed in an appropriate order. The embodiment is not limited to the order in which the steps are recorded. The use of all illustrative or exemplary terms (such as, etc.) in the embodiment is only for describing the embodiment in detail, and unless limited by the claims, the scope of the embodiment is not limited to the illustrative or exemplary terms. Furthermore, a person of ordinary skill should be able to understand that various modifications, combinations and changes can be made according to design conditions and factors within the scope of the appended claims or their equivalents.

[0195] Although certain exemplary embodiments and implementations have been described herein, other embodiments and modifications will be apparent from this description. Therefore, the inventive concept is not limited to these embodiments, but rather to the broader scope of the appended claims, and various obvious modifications and equivalent arrangements will be apparent to those skilled in the art.

Claims

1. A method for obtaining size information of a lesion, comprising: obtaining image information of the interior of the body from an image sensor; identifying at least one lesion from the image based on a pre-trained model; as well as Calculate the size information of the lesion.

2. The method according to claim 1, further comprising Calculate the first distance, where Calculate the size information of the lesion, The size information of the lesion is calculated based on the first distance.

3. The method according to claim 2, wherein: Calculating the first distance includes calculating the first distance based on brightness information of the lesion represented in the image.

4. The method according to claim 2, wherein: Calculating the first distance also includes obtaining a first image when the front end portion is at a first angle and a second image when the front end portion is at a second angle; and A difference between the first angle and the second angle and a visual deviation between the first image and the second image are determined.

5. The method according to claim 1, wherein Identifying the lesion includes calculating lesion size information on the image, wherein: The lesion size information on the image represents the size of the lesion in the image.

6. The method according to claim 5, wherein: The lesion size information on the image includes a bounding box, which is represented as surrounding the lesion. The method according to claim 1 , further comprising controlling the distal end portion so that the distal end portion tracks the lesion.

8. The method according to claim 7, wherein: The image sensor has a viewing angle including a first area, and Controlling the front end portion includes controlling the front end portion so that the lesion is located within the first area.

9. The method according to claim 7, wherein: Controlling the front end portion includes calculating a first value.

10. An endoscopic device comprising: a front end portion including an image sensor; as well as A control unit is configured to obtain an image of the interior of a body from the image sensor, identify at least one lesion from the image based on a pre-trained model, and calculate size information of the lesion.

11. The endoscopic device according to claim 10, wherein: The control unit is configured to calculate a first distance and calculate size information of the lesion based on the first distance.

12. The endoscopic device according to claim 11, wherein The control unit is configured to calculate the first distance based on brightness information of the lesion represented by the image.

13. The endoscopic device according to claim 11, wherein The control unit is configured to obtain a first image when the front end portion is at a first angle and a second image when the front end portion is at a second angle, and A difference between the first angle and the second angle and a visual deviation between the first image and the second image are determined.

14. The endoscopic device according to claim 10, wherein: The control unit is configured to calculate lesion size information on the image, where the lesion size information on the image represents the size of the lesion on the image.

15. The endoscopic device according to claim 14, wherein The lesion size information on the image includes a bounding box, which is represented as surrounding the lesion.

16. The endoscopic device according to claim 10, wherein The control unit is configured to control the distal end portion so that the distal end portion tracks the lesion.

17. The endoscopic device according to claim 16, wherein: The image sensor has a viewing angle including a first area, and The control unit is configured to control the distal end portion so that the lesion is located within the first region.

18. The endoscopic device according to claim 16, wherein The control unit is configured to calculate a first value.

Images