Endoscope system and method for operating the same

Abstract

To provide an endoscope system that can determine whether a length measurement mode can be executed depending on connection of an endoscope and a method of operating the endoscope system.SOLUTION: A processor determines whether an endoscope 12 is a length measurement-compatible endoscope in a case where the endoscope 12 is connected to a processor device 14, and enables switching to a length measurement mode where the endoscope 12 emits measurement light and causes a display to display a virtual scale based on the measurement light in a case where the endoscope 12 is the length measurement-compatible endoscope. In a case where an observation mode is set as a special observation mode, special light is turned on, which is used to enhance a specific region of an object to be observed, while the measurement light is turned off.SELECTED DRAWING: Figure 14

Description

[Technical field]

[0001] The present invention relates to an endoscope system that displays a virtual scale for measuring the size of an object, and an operating method thereof. [Background technology]

[0002] In an endoscope system having a light source device, an endoscope, and a processor device, the distance to a subject, the size of the subject, etc. are acquired. For example, in Patent Document 1, illumination light and measurement light are irradiated onto the subject, and a measurement light irradiation area such as a spot light appears on the subject due to the irradiation of the measurement light. Then, a virtual scale for measuring the size of the subject is displayed on the image in correspondence with the position of the spot light. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] International Publication No. 2018 / 051680 Summary of the Invention [Problem to be solved by the invention]

[0004] When displaying a virtual scale using measurement light as in Patent Document 1, a length measurement-enabled endoscope capable of irradiating measurement light, etc. is required. In an endoscope system, in order to execute a length measurement mode in which a virtual scale is displayed, it is necessary to determine whether a length measurement-enabled endoscope is connected.

[0005] An object of the present invention is to provide an endoscope system and an operating method thereof that can determine whether or not a length measurement mode can be executed based on the connection of an endoscope. [Means for solving the problem]

[0006] The endoscopic system of the present invention comprises an endoscope, a processor device to which the endoscope is connected, and a processor. When the endoscope is connected to the processor device, the processor determines whether the endoscope is a length-measurement-capable endoscope, and if the endoscope is a length-measurement-capable endoscope, enables switching to a length-measurement mode in which measurement light is emitted from the endoscope and a virtual scale based on the measurement light is displayed on the display, and when the observation mode is set to a special observation mode, controls the turning on of a special light used to highlight a specific area of ​​the object of observation and the turning off of the measurement light.

[0007] When a switching operation to the length measurement mode is performed while the special observation mode is set, the processor preferably prohibits switching to the length measurement mode. When a switching operation to the length measurement mode is performed while the special observation mode is set, the processor preferably cancels the special observation mode and switches to the length measurement mode.

[0008] In the special observation mode, the processor preferably displays a pseudo-color image in which the colors of the subject image are assigned to different color channels on the display. When the length measurement mode is set, it is preferable that illumination light and measurement light different from the special light are irradiated from the endoscope. In accordance with an operation for switching to the length measurement mode, the processor preferably displays a scale display icon on the display.

[0009] When the mode switching condition is not satisfied during switching to the length measurement mode, the processor preferably turns on a length measurement function unavailable display indicating that the virtual scale is not being displayed. The mode switching condition is preferably a setting condition related to the endoscope, the processor device, and the processor that is used for executing the length measurement mode. In the length measurement mode, the display mode of the virtual scale is preferably changed by selecting from a plurality of scale patterns.

[0010] When switching to the length measurement mode is enabled, it is preferable that the processor, by switching to the length measurement mode, performs at least one of the following: switching the measurement light ON or OFF, switching the length measurement image display setting related to the length measurement image displaying the virtual scale ON or OFF, switching the length measurement function operation status display indicating that the virtual scale is being displayed on the display ON or OFF, and switching the display of the virtual scale ON, OFF, or changing the display mode.

[0011] It is preferable that the processor switches the measurement light ON, the measurement image display setting ON, the measurement function operation status display ON, and the display of the virtual scale ON in response to a switching operation to the measurement mode. If the mode switching condition is not satisfied in the switching operation to the measurement mode, it is preferable that the processor prohibits switching the measurement light ON, the measurement image display setting ON, the measurement function operation status display ON, and the display of the virtual scale ON. Instead of prohibiting switching the measurement function operation status display ON, it is preferable to turn ON a measurement function operation status disabled display indicating that the virtual scale is not being displayed.

[0012] When the processor turns the length measurement image display setting ON, it is preferable that the processor saves the image display setting before switching to the length measurement mode. The display mode of the virtual scale is preferably changed by selecting from among a plurality of scale patterns. When the processor switches from the length measurement mode to another mode, it is preferable that the processor switches the measurement light OFF, the length measurement image display setting OFF, the length measurement function operation status display OFF, and the virtual scale display OFF. When the processor turns the length measurement image display setting OFF, it is preferable that the processor switches to the image display setting saved before switching to the length measurement mode.

[0013] The present invention provides an operating method for an endoscopic system including an endoscope, a processor device to which the endoscope is connected, and a processor, in which, when the endoscope is connected to the processor device, the processor determines whether the endoscope is a length-measurement-capable endoscope, and if the endoscope is a length-measurement-capable endoscope, enables switching to a length-measurement mode in which measurement light is emitted from the endoscope and a virtual scale based on the measurement light is displayed on the display, and when the observation mode is set to a special observation mode, turns on a special light used to highlight a specific area of ​​the object of observation and turns off the measurement light. Effect of the Invention

[0014] According to the present invention, it is possible to determine whether or not the length measurement mode can be executed depending on the connection of the endoscope. [Brief description of the drawings]

[0015] [Figure 1] FIG. 1 is a schematic diagram of an endoscope system. [Diagram 2] FIG. [Diagram 3] FIG. 2 is a front view of the balloon. [Figure 4] FIG. 2 is an explanatory diagram showing a balloon in a deflated state within the intestinal tract. [Diagram 5] FIG. 2 is an explanatory diagram showing an inflated balloon in the intestinal tract. [Figure 6] FIG. 2 is a front view of the tip portion of the endoscope. [Figure 7] FIG. 2 is a perspective view of the tip of the endoscope. [Figure 8] 2 is a block diagram showing the functions of the endoscope system. FIG. [Figure 9] 1A is an image diagram showing a digital zoom function when it is OFF and when it is ON when it is ON. [Figure 10] FIG. 2 is a schematic diagram showing a measurement light emitting unit. [Figure 11] 2 is a cross-sectional view of a tip portion of an endoscope having a measurement light emitting portion. [Figure 12] FIG. [Figure 13]FIG. 4 is a schematic diagram showing the traveling direction of measurement light. [Figure 14] FIG. 2 is a block diagram showing the functions of a system control unit. [Figure 15] FIG. 13 is an image showing an icon indicating that a scale is being displayed or an icon indicating that a scale is not being displayed. [Figure 16] FIG. 2 is a block diagram showing the functions of a system control unit. [Figure 17] FIG. 4 is an explanatory diagram showing a first control. [Figure 18] FIG. 13 is an image diagram showing a message displayed when the first control is performed. [Figure 19] 13 is an explanatory diagram showing the release of the special observation mode and the change to the length measurement mode. FIG. [Figure 20] FIG. 11 is an explanatory diagram showing a second control. [Figure 21] FIG. 13 is an image diagram showing a message displayed when the second control is performed. [Figure 22] FIG. 11 is an explanatory diagram showing a third control. [Diagram 23] FIG. 13 is an image diagram showing a message displayed when the third control is performed. [Figure 24] FIG. 2 is a block diagram showing the functions of a system control unit. [Diagram 25] FIG. 11 is an explanatory diagram showing a first light emission control table. [Figure 26] FIG. 2 is an explanatory diagram showing coordinate areas 1 to 5. [Figure 27] FIG. 11 is an explanatory diagram showing a second light emission control table. [Figure 28] FIG. 11 is an explanatory diagram showing light emission control in a length measurement mode. [Figure 29] 11 is an explanatory diagram showing light emission and imaging control of a first pattern in a length measurement mode. FIG. [Diagram 30] 13 is an explanatory diagram showing a second pattern of light emission and imaging control in the length measurement mode. FIG. [Diagram 31] FIG. 2 is a block diagram showing the functions of a signal processing unit. [Diagram 32] FIG. 13 is an image diagram showing a virtual scale displayed in the case of the near end Px. [Diagram 33] FIG. 13 is an image diagram showing the virtual scale displayed when Py is near the center. [Diagram 34] FIG. 13 is an image diagram showing a virtual scale displayed in the case of the far end Pz. [Diagram 35] 1A to 1C are explanatory diagrams showing virtual scales of various shapes. [Diagram 36] FIG. 13 is an explanatory diagram showing virtual scales having different sizes. [Figure 37] FIG. 13 is an explanatory diagram showing a virtual scale with different colors. [Figure 38] FIG. 13 is an explanatory diagram showing a distorted concentric virtual scale; [Figure 39] FIG. 11 is an explanatory diagram showing representative point data. [Diagram 40] FIG. 11 is an explanatory diagram showing the processing of a table update unit. [Diagram 41] 13 is an image diagram showing a virtual scale displayed when a planar measurement light is irradiated. FIG. [Diagram 42] FIG. 2 is an explanatory diagram showing planar light including two first characteristic lines. [Diagram 43] FIG. 2 is an explanatory diagram showing functions of a signal processing unit. [Diagram 44] 11 is an image diagram showing an image displayed when planar light including two first characteristic lines is irradiated. FIG. [Diagram 45] 11 is an explanatory diagram showing a traveling direction of planar light when the planar light including two first characteristic lines is irradiated. FIG. [Figure 46] FIG. 2 is an explanatory diagram showing a diffraction spot. [Figure 47] 11 is an explanatory diagram of a method for calculating two-dimensional information and three-dimensional information of a subject when a diffraction spot is used. FIG. [Figure 48] FIG. 2 is a block diagram showing the functions of a signal processing unit. [Figure 49] FIG. 4 is an explanatory diagram showing a process for obtaining a first captured image from which noise has been removed. [Figure 50] FIG. 11 is an explanatory diagram showing a binarized first color information image. [Figure 51]FIG. 13 is an explanatory diagram showing a binarized second color information image. [Figure 52] FIG. 13 is an explanatory diagram showing a binarized second color information image. [Figure 53] FIG. 11 is an explanatory diagram showing a binarized first color information image. [Figure 54] FIG. 2 is a block diagram showing the functions of a signal processing unit. [Figure 55] FIG. 4 is an explanatory diagram showing a process for obtaining a first differential image or a second differential image. [Figure 56] FIG. 11 is an explanatory diagram showing a first difference process. [Figure 57] FIG. 11 is an explanatory diagram showing a second difference process. [Figure 58] FIG. 2 is a block diagram showing the functions of a signal processing unit. [Figure 59] FIG. 2 is a schematic diagram showing a spot including a white central region and a surrounding region. [Figure 60] 1 is a graph showing distribution of pixel values ​​of various images in a captured image. [Figure 61] 11 is a graph showing the relationship between the transmission distribution of each color filter and the wavelength range of measurement light. [Figure 62] 4 is a block diagram showing functions of an irradiation area recognition unit; FIG. [Figure 63] FIG. 13 is an explanatory diagram showing an example of a spot pattern deformed from a circle. [Figure 64] FIG. 2 is a block diagram showing the functions of a signal processing unit. [Figure 65] FIG. 1 is a schematic diagram showing a convex polyp. [Figure 66] FIG. 13 is an explanatory diagram showing the height of a spot. [Figure 67] FIG. 13 is an explanatory diagram regarding calculation of an offset distance D6. [Figure 68] 1 is a schematic diagram of a virtual scale with lines of different widths. [Figure 69] FIG. 2 is a schematic diagram showing a concentric virtual scale. [Figure 70] FIG. 13 is a schematic diagram of a virtual scale in which gradation is added to lines. [Figure 71]FIG. 13 is a schematic diagram of a virtual scale with different dashed gaps. [Figure 72] 1 is a schematic diagram of a virtual scale having different numbers of lines. [Figure 73] FIG. 2 is a block diagram showing the functions of a signal processing unit. [Figure 74] FIG. 13 is an image showing an illuminated spot on the periphery of a polyp. [Figure 75] FIG. 13 is an explanatory diagram showing a virtual scale whose base end is aligned with the position of a spot. [Figure 76] FIG. 13 is an explanatory diagram showing two virtual scales whose base ends are aligned with the positions of spots. [Figure 77] FIG. 13 is an explanatory diagram showing a virtual scale of a circle and a line segment whose base end is aligned with the position of a spot. [Figure 78] FIG. 13 is an explanatory diagram showing a virtual scale whose base end is aligned with the position of a spot. [Figure 79] FIG. 2 is a block diagram showing the functions of a signal processing unit. [Figure 80] 4 is a block diagram showing the functions of a reference scale setting unit; FIG. [Figure 81] FIG. 13 is an image diagram showing a virtual scale superimposed on a polyp. [Figure 82] 4 is a block diagram showing functions of a measurement value scale generating unit; FIG. [Figure 83] FIG. 2 is an image diagram showing a region of interest. [Figure 84] FIG. [Figure 85] FIG. 13 is an image diagram of a measurement value scale. [Figure 86] FIG. 13 is an image showing a measurement scale superimposed on a polyp. [Figure 87] FIG. 13 is an explanatory diagram showing a distorted lattice region. [Figure 88] FIG. 2 is an explanatory diagram showing a square lattice region. [Figure 89] 11 is an explanatory diagram showing an example of displaying virtual scales of different shapes inside and outside an effective measurement area; FIG. [Figure 90]11 is an explanatory diagram showing an example of displaying a virtual scale with different line types inside and outside the effective measurement area; FIG. [Figure 91] FIG. 11 is an explanatory diagram regarding acquisition of a still image in a length measurement mode. [Figure 92] FIG. 4 is an explanatory diagram showing first to third captured images. [Figure 93] FIG. 11 is an image diagram showing the second and third captured images displayed when a still image is acquired. [Figure 94] FIG. 11 is an explanatory diagram regarding acquisition of a still image in a length measurement mode. [Figure 95] FIG. 11 is an explanatory diagram regarding acquisition of a still image in a length measurement mode. [Figure 96] FIG. 2 is a block diagram showing the functions of a signal processing unit. [Figure 97] FIG. 2 is a block diagram showing the functions of a calibration device. [Figure 98] FIG. 1 is a schematic diagram of an inspection system. [Figure 99] FIG. 2 is a plan view showing a test chart. [Figure 100] 1 is an image diagram showing an inspection reference position, a spot, and a virtual scale displayed on an inspection image. [Figure 101] 1 is an image diagram showing an inspection reference position, a spot, and a virtual scale displayed on an inspection image. [Figure 102] This is a chart image with 5mm ruled lines. [Figure 103] This is a chart image with 5 mm lines (taken at a more distal end than FIG. 102). [Figure 104] FIG. 13 is an explanatory diagram showing pixel positions of spots in the X direction. [Figure 105] FIG. 13 is an explanatory diagram showing pixel positions of spots in the Y direction. [Fig. 106] FIG. 13 is an explanatory diagram showing pixel positions of spots in the X direction. [Figure 107] FIG. 13 is an explanatory diagram showing pixel positions of spots in the Y direction. [Figure 108] 11 is an explanatory diagram showing a striped light pattern ZPL. FIG. [Fig. 109]13 is an explanatory diagram showing an emission pattern of striped pattern light ZPL of phases X, Y, and Z. FIG. [Figure 110] 11 is an explanatory diagram showing a measurement light LPL having a grid pattern. FIG. [Figure 111] 11 is an explanatory diagram showing a light emission pattern for intermittently irradiating measurement light in a grid pattern. FIG. [Figure 112] FIG. 2 is an explanatory diagram showing three-dimensional planar light TPL. [Figure 113] 10 is an explanatory diagram showing a light emission pattern for intermittently emitting three-dimensional planar light TPL. FIG. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0016] As shown in Fig. 1, the endoscope system 10 includes an endoscope 12, a light source device 13, a processor device 14, a display 15, a user interface 16, an extended processor device 17, and an extended display 18. The endoscope 12 is optically connected to the light source device 13 and electrically connected to the processor device 14. The endoscope 12 includes an insertion section 12a to be inserted into the body of an observation subject, an operation section 12b provided at a base end portion of the insertion section 12a, and a bending section 12c and a tip section 12d provided at the tip side of the insertion section 12a. The bending section 12c is bent by operating the operation section 12b. The tip section 12d is directed in a desired direction by the bending operation of the bending section 12c.

[0017] The operation unit 12b also includes an observation mode switching switch 12f used to switch between observation modes, a still image acquisition instruction switch 12g used to instruct the acquisition of a still image of the object of observation, and a zoom operation unit 12h used to operate the zoom lens 21b.

[0018] The processor device 14 is electrically connected to a display 15 and a user interface 16. The display 15 outputs and displays images or information of the observation target processed by the processor device 14. The user interface 16 has a keyboard, mouse, touchpad, microphone, etc., and has a function of accepting input operations such as function settings. The extended processor device 17 is electrically connected to the processor device 14. The extended display 18 outputs and displays images or information processed by the extended processor device 17.

[0019] The endoscope 12 has a normal observation mode, a special observation mode, and a length measurement mode, which are switched by an observation mode changeover switch 12f. The normal observation mode is a mode in which the observation target is illuminated with illumination light. The special observation mode is a mode in which the observation target is illuminated with special light different from the illumination light. The length measurement mode illuminates the observation target with illumination light or measurement light, and displays a virtual scale used to measure the size of the observation target, etc., on the subject image obtained by capturing an image of the observation target. The subject image without the virtual scale superimposed is displayed on the display 15, while the subject image with the virtual scale superimposed is displayed on the extended display 18.

[0020] The illumination light is light used to provide brightness to the entire object to be observed and to observe the entire object. The special light is light used to highlight a specific area of ​​the object to be observed. The measurement light is light used to display a virtual scale. In this embodiment, a virtual scale displayed on an image is described, but a real scale may be provided in an actual lumen so that the real scale can be confirmed through an image. In this case, the real scale may be inserted through the forceps channel of the endoscope 12 and protruded from the tip 12d.

[0021] When the user operates the still image acquisition instruction switch 12g, the screen of the display 15 freezes and an alert sound (e.g., a beep) is generated to notify the user that a still image will be acquired. Then, still images of the subject image obtained around the time of operating the still image acquisition instruction switch 12g are stored in a still image storage unit 42 (see FIG. 8) in the processor device 14. The still image storage unit 42 is a storage unit such as a hard disk or a USB (Universal Serial Bus) memory. If the processor device 14 is connectable to a network, instead of or in addition to the still image storage unit 42, the still images of the subject image may be stored in a still image storage server (not shown) connected to the network.

[0022] Note that a still image acquisition instruction may be issued using an operating device other than the still image acquisition instruction switch 12g. For example, a foot pedal may be connected to the processor device 14, and a still image acquisition instruction may be issued when the user operates the foot pedal (not shown) with his / her foot. Mode switching may also be performed by the foot pedal. In addition, a gesture recognition unit (not shown) that recognizes a user's gesture may be connected to the processor device 14, and a still image acquisition instruction may be issued when the gesture recognition unit recognizes a specific gesture performed by the user. Mode switching may also be performed using the gesture recognition unit.

[0023] Also, a gaze input unit (not shown) provided near the display 15 may be connected to the processor device 14, and when the gaze input unit recognizes that the user's gaze is within a predetermined area of ​​the display 15 for a certain period of time or more, it may issue an instruction to acquire a still image. Also, a voice recognition unit (not shown) may be connected to the processor device 14, and when the voice recognition unit recognizes a specific voice uttered by the user, it may issue an instruction to acquire a still image. Mode switching may also be performed using the voice recognition unit. Also, an operation panel (not shown) such as a touch panel may be connected to the processor device 14, and when the user performs a specific operation on the operation panel, it may issue an instruction to acquire a still image. Mode switching may also be performed using the operation panel.

[0024] As shown in FIG. 2, the tip portion 12d is equipped with an imaging optical system 21 that receives light from the subject, an illumination optical system 22 for irradiating illumination light onto the subject, a measurement light emitting section 23 that radiates measurement light used in the length measurement mode onto the subject, an opening 24 for protruding the treatment tool toward the subject, and an air and water supply nozzle 25 for supplying air and water.

[0025] A balloon 19 as a fixing member is removably attached to the insertion section 12a. The balloon 19 is a disposable type balloon, and is discarded after one or a few uses and replaced with a new one. Note that the number of uses here refers to the number of cases, and a few uses refers to 10 times or less.

[0026] The balloon 19 is formed in a generally cylindrical shape with its ends narrowed by an elastic material such as rubber. The balloon 19 has a small-diameter tip end 19a and base end 19b, and a central bulge 19c. After the insertion section 12a is inserted into the balloon 19 and the balloon 19 is positioned at a predetermined position, the balloon 19 is fixed to the insertion section 12a, for example, by fitting rubber rings 20a, 20b into the tip end 19a and base end 19b.

[0027] As shown in Fig. 3, the predetermined position where the balloon 19 is fixed to the insertion section 12a is preferably closer to the base end of the insertion section 12a than the bending section 12c, and is a position where the tip of the tip section 19a of the balloon 19 coincides with the base end of the bending section 12c. This prevents the balloon 19 from interfering with the bending operation of the bending section 12c, and prevents the bending section 12c from interfering with the inflation or deflation of the balloon 19. The inflation or deflation of the balloon 19 is controlled by a balloon control device BLC, as described later. The balloon control device BLC is preferably operated by a user interface 16.

[0028] As shown in Figure 4, when the balloon 19 is deflated, the insertion portion 12a is not fixed to the intestinal tract 26, and if measurement light is irradiated and imaging is performed with the balloon in a deflated state, the position of the tip portion 12d may move in the up, down, left, or right directions, and the measurement light may not be accurately irradiated onto the observation object that the user wishes to measure.

[0029] Therefore, as shown in Fig. 5, the balloon 19 is inflated under the control of the balloon control device BLC. Since the outer diameter of the inflated balloon 19 is formed to match the inner diameter of the intestinal tract 26, the insertion section 12a is in a state of being fixed in position within the intestinal tract 26. As a result, the insertion section 12a is fixed relative to the intestinal tract 26, so that the user can accurately irradiate the measurement light onto the observation target being measured. Note that the "fixed state" here includes a state in which the position of the insertion section 12a relative to the insertion direction is fixed, but the orientation of the tip section 12d is finely adjustable.

[0030] 6, the tip 12d of the endoscope is substantially circular, and is provided with the imaging optical system 21, the illumination optical system 22, the opening 24, and the air / water nozzle 25 along the first direction D1. Two illumination optical systems 22 are provided on both sides of the imaging optical system 21 in a second direction perpendicular to the first direction. The measurement light emitting unit 23 is provided between the imaging optical system 21 and the air / water nozzle 25 in the first direction. Therefore, since the air / water port of the air / water nozzle 25 faces the imaging optical system 21 and the measurement light emitting unit 23, both the imaging optical system 21 and the measurement light emitting unit 23 can be cleaned by air or water supply.

[0031] As shown in Fig. 7, a tip cap 27 is attached to the tip portion 12d. The tip cap 27 is provided with a tip surface 28. The tip surface 28 has a flat surface 28a, a flat surface 28b, and a guide surface 28c. The flat surface 28a is a flat surface perpendicular to the axial direction Z. The flat surface 28b is parallel to the flat surface 28a and is located closer to the tip side than the flat surface 28a in the axial direction Z. The guide surface 28c is located between the flat surfaces 28a and 28b.

[0032] The plane 28b is provided with a through hole 27a exposing the tip surface 21c of the imaging optical system 21 and a through hole 27b exposing the tip surfaces 22b of the pair of illumination optical systems 22. The tip surfaces 21c and 22b are disposed on the same plane as the plane 28.

[0033] Through holes 27c and 27d are arranged on the plane 28a. The air and water nozzle 25 is exposed from the through hole 27c. That is, the plane 28a is the mounting position of the air and water nozzle 25 in the axial direction Z. An injection tube portion 25a is formed on the tip side of the air and water nozzle 25. The injection tube portion 25a is formed in a cylindrical shape protruding from the base end of the air and water nozzle 25 in a direction bent at 90 degrees, for example, and has an injection port 25b at its tip. The injection tube portion 25a is arranged protruding from the through hole 52c on the tip side in the axial direction Z.

[0034] The ejection port 25b is disposed facing the imaging optical system 21. Thus, the air / water nozzle 25 ejects a cleaning liquid or gas, which is a fluid, onto the tip surface 21c of the imaging optical system 21 and its periphery.

[0035] When cleaning water or gas is sprayed from the air / water nozzle 25 to the imaging optical system 21, it is preferable that the flow velocity F1 of the cleaning water at the position where it reaches the imaging optical system 21, i.e., at the outer periphery of the imaging optical system 21, is 2 m / s or more, and the flow velocity F2 of the gas at the outer periphery of the imaging optical system 21 is 40 m / s or more. Note that it is preferable that the flow velocities F1 and F2 satisfy the above values ​​regardless of the orientation of the tip 12d. For example, when the air / water nozzle 25 is positioned vertically below the imaging optical system 21, the flow velocity decreases due to the effect of gravity of the cleaning water or gas, but even in this case, it is preferable that the above values ​​are satisfied.

[0036] The tip surface of the measurement light emitting part 23 exposed from the through hole 27d is disposed on the plane 28a. That is, the attachment position of the air / water nozzle 25 and the tip surface of the measurement light emitting part 23 are disposed at the same position in the axial direction Z. The measurement light emitting part 23 is disposed within the fluid injection range of the air / water nozzle 25 and between the imaging optical system 21 and the air / water nozzle 25. In this embodiment, when the tip surface 28 is viewed from the axial direction Z, the measurement light emitting part 23 is disposed within a region connecting the injection port 25b of the air / water nozzle 25 and the outer periphery of the imaging optical system 21. This makes it possible to simultaneously inject fluid from the air / water nozzle 25 to the imaging optical system 21 and to inject fluid to the measurement light emitting part 23.

[0037] The guide surface 28c is formed as a continuous surface connecting the plane 28a and the plane 28b. The guide surface 28c is an inclined surface formed flat from a position where it contacts the outer periphery of the measurement light emission unit 23 to a position where it contacts the outer periphery of the imaging optical system 21. Since the guide surface 28c is arranged in the fluid injection range of the air and water nozzle, when a fluid is injected from the air and water nozzle 25, the fluid is also injected onto the guide surface 28c. The fluid injected onto the guide surface 28c is diffused and sprayed onto the imaging optical system 21. In this case, the fluid injection range of the air and water nozzle may include the entire guide surface 28c, or may include only a part of the guide surface 28c. In this embodiment, the entire guide surface 28c is included within the area connecting the injection port 25b of the air and water nozzle 25 and the outer periphery of the imaging optical system 21.

[0038] As shown in FIG. 8, the light source device 13 includes a light source unit 30 and a light source processor 31. The light source unit 30 generates illumination light or special light for illuminating a subject. The illumination light or special light emitted from the light source unit 30 is incident on the light guide LG and is irradiated onto the subject through the illumination lens 22a. As the light source unit 30, a white light source that emits white light, or a plurality of light sources including a white light source and a light source that emits light of other colors (for example, a blue light source that emits blue light) are used as a light source of the illumination light. As the light source unit 30, a light source that emits broadband light including a narrowband blue light for emphasizing surface information such as surface blood vessels is used as a light source of the special light. The illumination light may be a white mixed color light that combines at least one of purple light, blue light, green light, and red light. In this case, it is preferable to perform optical design of the illumination optical system 22 so that the irradiation range of the green light is larger than the irradiation range of the red light.

[0039] The light source processor 31 controls the light source unit 30 based on instructions from the system control unit 41. The system control unit 41 not only gives instructions to the light source processor 31 regarding light source control, but also controls the light source 23a (see FIG. 10) of the measurement light emission unit 23. In the normal observation mode, the system control unit 41 controls to turn on the illumination light and turn off the measurement light. In the special observation mode, the system control unit 41 controls to turn on the special light and turn off the measurement light. In the length measurement mode, the system control unit 41 controls to turn on or off the illumination light or measurement light.

[0040] The illumination optical system 22 has an illumination lens 22a, and light from the light guide LG is irradiated onto the observation object via the illumination lens 22a. The imaging optical system 21 has an objective lens 21a, a zoom lens 21b, and an imaging element 32. Reflected light from the observation object is incident on the imaging element 32 via the objective lens 21a and the zoom lens 21b. As a result, a reflected image of the observation object is formed on the imaging element 32.

[0041] The zoom lens 21b has an optical zoom function that enlarges or reduces the subject by moving between the telephoto end and the wide-angle end. The optical zoom function can be switched ON and OFF by a zoom operation unit 12h (see FIG. 1) provided on the operation unit 12b of the endoscope, and when the optical zoom function is ON, the subject is enlarged or reduced at a specific magnification by further operating the zoom operation unit 12h.

[0042] The imaging element 32 is a color imaging sensor that captures a reflected image of the subject and outputs an image signal. The imaging element 32 is preferably a CCD (Charge Coupled Device) imaging sensor or a CMOS (Complementary Metal-Oxide Semiconductor) imaging sensor. The imaging element 32 used in the present invention is a color imaging sensor for obtaining a blue image, a green image, and a red image of three colors, R (red), G (green), and B (blue). The red image is an image output from red pixels in the imaging element 32 provided with a red color filter. The green image is an image output from green pixels in the imaging element 32 provided with a green color filter. The blue image is an image output from blue pixels in the imaging element 32 provided with a blue color filter. The imaging element 32 is controlled by an imaging control unit 33.

[0043] The image signal output from the imaging element 32 is transmitted to a CDS / AGC circuit 34. The CDS / AGC circuit 34 performs correlated double sampling (CDS) and automatic gain control (AGC) on the image signal, which is an analog signal. The image signal passed through the CDS / AGC circuit 34 is converted into a digital image signal by an A / D (Analog / Digital) converter 35. The A / D converted digital image signal is input to a communication I / F (Interface) 37 of the light source device 13 via a communication I / F (Interface) 36.

[0044] In the processor device 14, programs related to various processes or controls are incorporated in a program storage memory (not shown). A system control unit 41 constituted by an image control processor operates the programs incorporated in the program storage memory to realize the functions of a receiving unit 38 connected to a communication I / F (Interface) 37 of the light source device 13, a signal processing unit 39, and a display control unit 40.

[0045] The receiving unit 38 receives the image signal transmitted from the communication I / F 37 and transmits it to the signal processing unit 39. The signal processing unit 39 has a built-in memory for temporarily storing the image signal received from the receiving unit 38, and processes a group of image signals, which is a collection of the image signals stored in the memory, to generate a subject image. Note that the receiving unit 38 may send a control signal related to the light source processor 31 directly to the system control unit 41.

[0046] When the normal observation mode is set, the signal processing unit 39 performs signal allocation processing to allocate a blue image of the subject image to the B channel of the display 15, a green image of the subject image to the G channel of the display 15, and a red image of the subject image to the R channel of the display 15, thereby displaying a color subject image on the display 15. In the length measurement mode, the same signal allocation processing as in the normal observation mode is performed.

[0047] On the other hand, when the signal processing unit 39 is set to the special observation mode, the red image of the subject image is not used for display on the display 15, but the blue image of the subject image is assigned to the B channel and G channel of the display 15, and the green image of the subject image is assigned to the R channel of the display 15, thereby displaying a pseudo-colored subject image on the display 15. When the signal processing unit 39 is set to the length measurement mode, the signal processing unit 39 transmits the subject image including the irradiation position of the measurement light to the data transmission / reception unit 43. The data transmission / reception unit 43 transmits data related to the subject image to the extension processor device 17. The data transmission / reception unit 43 can receive data from the extension processor device 17. The received data can be processed by the signal processing unit 39 or the system control unit 41.

[0048] When the digital zoom function is set to ON by the user interface 16 as the zoom function, the signal processing unit 39 enlarges or reduces the subject at a specific magnification by cropping and enlarging or reducing a part of the subject image. Fig. 9(A) shows a subject image with the digital zoom function OFF, and Fig. 9(B) shows a subject image with the digital zoom function ON in which the center part of the subject image in Fig. 9(A) is cropped and enlarged. Note that when the digital zoom function is OFF, the subject is not enlarged or reduced by cropping the subject image.

[0049] The display control unit 40 displays the subject image generated by the signal processing unit 39 on the display 15. The system control unit 41 performs various controls on the endoscope 12, the light source device 13, the processor device 14, and the extended processor device 17. The system control unit 41 controls the image sensor 32 via the imaging control unit 33 provided in the endoscope 12. The imaging control unit 33 also controls the CDS / AGC circuit 34 and the A / D converter 35 in accordance with the control of the image sensor 32.

[0050] The extended processor device 17 receives data transmitted from the processor device 14 at the data transmission / reception unit 44. The signal processing unit 45 performs processing related to the length measurement mode based on the data received at the data transmission / reception unit 44. Specifically, the signal processing unit 45 performs processing to determine the size of the virtual scale from the subject image including the irradiation position of the measurement light, and superimposes and displays the determined virtual scale on the subject image. The display control unit 46 displays the subject image with the virtual scale superimposed on it on the extended display 18. The data transmission / reception unit 44 is capable of transmitting data, etc. to the processor device 14.

[0051] As shown in FIG. 10, the measurement light emitting unit 23 emits measurement light obliquely with respect to the optical axis Ax (see FIG. 13) of the imaging optical system 21. The measurement light emitting unit 23 includes a light source 23a, a diffractive optical element DOE 23b, a prism 23c, and an emitting unit 23d. The light source 23a emits light (specifically, visible light) of a color detectable by the pixels of the imaging element 32, and includes a light emitting element such as a laser light source LD (Laser Diode) or an LED (Light Emitting Diode), and a condensing lens that condenses the light emitted from the light emitting element. The light source 23a is provided on a scope electric circuit board (not shown). The scope electric circuit board is provided on the tip 12d of the endoscope, and receives power from the light source device 13 or the processor device 14 to supply power to the light source 23a. Although the light source 23a is provided at the tip 12d of the endoscope, it may be provided inside a connector for connecting the endoscope 12 and the processor device 14. Even in this case, the members of the measurement light emitting unit 23 other than the light source 23a (the diffractive optical element DOE 23b, the prism 23c, and the emitting unit 23d) are provided at the tip 12d of the endoscope.

[0052] In this embodiment, the wavelength of the light emitted by the light source 23a is, for example, red laser light (beam light color) having a wavelength of 600 nm or more and 650 nm or less, but light of other wavelength bands, for example, green light having a wavelength of 495 nm or more and 570 nm or less, may be used. The light source 23a is controlled by the system control unit 41, and emits light based on an instruction from the system control unit 41. The DOE 23b converts the light emitted from the light source into measurement light for obtaining measurement information. Note that the amount of the measurement light is adjusted based on the viewpoint of protecting the human body, eyes, and internal organs, and is preferably adjusted to an amount of light that is sufficient to cause whiteout (pixel saturation) in the observation range of the endoscope 12.

[0053] The prism 23c is an optical member for changing the traveling direction of the measurement light after conversion by the DOE 23b. The prism 23c changes the traveling direction of the measurement light so as to intersect the field of view of the imaging optical system 21 including the objective lens 21a. Details of the traveling direction of the measurement light will be described later. The measurement light Lm emitted from the prism 23c is irradiated onto the subject.

[0054] As shown in FIG. 11, the measurement light emitting unit 23 is housed in a housing portion 47 for the measurement light emitting unit provided at the tip portion 12d of the endoscope. The housing portion 47 for the measurement light emitting unit has a hole corresponding to the size of the measurement light emitting unit 23. The housing portion 47 for the measurement light emitting unit is closed by a transparent lid 48. The transparent lid 48 has a transparent plate shape, and one end face is a flat portion 48a. The transparent lid 48 is arranged such that the flat portion 48a is flush with the tip face 28 of the tip portion 12d. By providing the transparent lid 48 flush with the tip face 28, foreign matters that may block the emission of the measurement light are prevented from being caught.

[0055] As shown in FIG. 12, a prism 49 is arranged between the transparent lid 48 and the prism 23c. The prism 49 has a first contact surface 49a and a second contact surface 49b, and the first contact surface 49a is in contact with the prism 23c, and the second contact surface 49b is in contact with the transparent lid 48. The prism 49 excludes gas from between the transparent lid 48 and the prism 23c to make it airtight. By making it airtight in this way, condensation can be prevented. That is, problems such as attenuation, diffusion, convergence, and refraction of the measurement light due to condensation can be prevented from occurring.

[0056] When the refractive index of the prism 23c is "n1" and the refractive index of the prism 49 is "n2", an example where "n1 < n2" is satisfied and the light emitting surface of the prism 23c is inclined toward the optical axis Ax side has been described, but the reverse configuration may also be used. It may be set as "n1 > n2" and the light emitting surface of the prism 23c may be provided on the side opposite to the optical axis Ax. However, in this case, since there is a possibility of total reflection at the light emitting surface of the prism 23c, it is necessary to provide a limitation to the light emitting surface of the prism 23c.

[0057] Instead of configuring the prism 23c from an optical member, it may be a measurement auxiliary slit formed in the tip 12d of the endoscope. If the prism 23c is configured from an optical member, it is preferable to apply an anti-reflection coating (AR (Anti-Reflection) coating) (anti-reflection portion) to the exit surface. The reason for providing the anti-reflection coating in this manner is that if the measurement light is reflected without passing through the exit surface of the prism 23c and the proportion of the measurement light irradiated on the subject decreases, it becomes difficult for the irradiation position detection unit 61 described later to recognize the position of the spot SP formed on the subject by the measurement light.

[0058] The measurement light emitting unit 23 may be any unit capable of emitting measurement light toward the field of view of the imaging optical system 21. For example, the light source 23a may be provided in a light source device, and the light emitted from the light source 23a may be guided to the DOE 23b by an optical fiber. Alternatively, the measurement light Lm may be emitted in a direction crossing the field of view of the imaging optical system 21 by setting the light source 23a and the DOE 23b obliquely with respect to the optical axis Ax of the imaging optical system 21 without using the prism 23c.

[0059] As shown in FIG. 13, the measurement light is emitted in a traveling direction in which the optical axis of the measurement light Lm intersects with the optical axis Ax of the imaging optical system 21. If the object is observable in the range Rx of the observation distance, the positions (points where the arrows Qx, Qy, and Qz intersect with the optical axis Ax) of the spots SP formed on the object by the measurement light Lm in the imaging range (indicated by the arrows Qx, Qy, and Qz) at each point are different at the near end Px, the center vicinity Py, and the far end Pz of the range Rx. The imaging angle of view of the imaging optical system 21 is represented within the area sandwiched between two solid lines 101X, and the measurement is performed in the central area (area sandwiched between two dotted lines 102X) of this imaging angle of view, which has less aberration. The third direction D3 is a direction perpendicular to the first direction D1 and the second direction D2 (similar to FIG. 45).

[0060] As described above, by emitting the measurement light Lm with the optical axis of the measurement light Lm intersecting the optical axis Ax, the size of the subject can be measured from the movement of the spot position with respect to the change in observation distance. Then, by capturing an image of the subject illuminated with the measurement light by the image sensor 32, an image of the subject including the spot SP can be obtained. In the subject image, the position of the spot SP varies depending on the relationship between the optical axis Ax of the imaging optical system 21 and the optical axis of the measurement light Lm, and on the observation distance, but if the observation distance is close, the number of pixels showing the same actual size (for example, 5 mm) will be large, and if the observation distance is long, the number of pixels will be small.

[0061] As shown in Figure 14, the system control unit 41 includes a length measurement-compatible endoscope suitability determination unit 140, a measurement light ON / OFF switching unit 141, a length measurement image display setting ON / OFF switching unit 142, a length measurement function operation status display ON / OFF switching unit 143, a virtual scale display switching control unit 144, and a pre-switching image display setting storage unit 149.

[0062] The length measurement compatible endoscope availability determination unit 140 determines whether the endoscope 12 is a length measurement compatible endoscope when the endoscope 12 is connected to the processor device 14. When the endoscope 12 is a length measurement compatible endoscope, switching to the length measurement mode is enabled. A length measurement compatible endoscope refers to an endoscope that can emit and receive measurement light and can display a length measurement image that displays a virtual scale based on the measurement light on the extended display 18 (which may be the display 15). The length measurement compatible endoscope availability determination unit 140 includes a scope ID table (not shown) that associates the scope ID provided in the endoscope 12 with a flag indicating the presence or absence of a length measurement compatible endoscope (for example, "1" for a length measurement compatible endoscope and "0" for other endoscopes). Then, the length measurement compatible endoscope availability determination unit 140 reads out the endoscope scope ID when the endoscope 12 is connected. It is determined whether the read scope ID is a length measurement compatible endoscope by referring to the flag in the scope ID table.

[0063] The measurement light ON / OFF switching unit 141 controls the light source 23a to switch the measurement light on (ON) or off (OFF). The length measurement image display setting ON / OFF switching unit 142 enables (ON) or disables (OFF) the execution of various image display settings in the length measurement mode, such as the display settings (color tone, etc.) of the length measurement image, by the user interface 16, etc. The virtual scale display switching control unit 144 switches the virtual scale on the extended display 18 to either display (ON) or not display (OFF), or change the display mode.

[0064] When switching to the length measurement mode is enabled, the system control unit 41 performs at least one of the following by switching to the length measurement mode using the observation mode switching switch 12f: switching the measurement light ON or OFF, switching the length measurement image display setting ON or OFF, switching the length measurement function operation status display ON or OFF, switching the virtual scale display ON or OFF, or changing the display mode.

[0065] For example, it is preferable that the system control unit 41 switches the measurement light ON, the measurement image display setting ON, the measurement function operation status display ON, and the virtual scale display ON in response to a switching operation to the measurement mode. On the other hand, it is preferable that the system control unit 41 switches the measurement light OFF, the measurement image display setting OFF, the measurement function operation status display OFF, and the virtual scale display OFF in response to a switching operation from the measurement mode to another mode (normal observation mode, special observation mode).

[0066] 15, the measurement function operation status display is preferably displayed by a scale display icon 146 in the auxiliary information display area 18a of the extended display 18. Following a switching operation to the measurement mode, the scale display icon 146 is displayed, and following a switching operation from the measurement mode to another mode, the scale display icon 146 is hidden. In addition, it is preferable that a virtual scale 147 is displayed in the observation image display area 18b of the extended display 18. The display mode of the virtual scale 147 is changed by a virtual scale display switching control unit 144.

[0067] The virtual scale 147 includes a 5 mm virtual scale 147a, a 10 mm virtual scale 147b, and a 20 mm virtual scale 147c. The virtual scales 147a, 147b, and 147c each have a circular scale (shown by dotted lines) and a line scale (shown by solid lines). The "5" on the virtual scale 147a indicates a 5 mm scale, the "10" on the virtual scale 147b indicates a 10 mm scale, and the "20" on the virtual scale 147c indicates a 20 mm scale.

[0068] The display mode of the virtual scale is changed, for example, by selection from a plurality of predetermined scale patterns. For example, the plurality of scale patterns include a scale pattern combining three virtual scales 147a, 147b, and 147c having a circular scale and a line segment scale, a scale pattern combining two virtual scales 147b and 147c having a circular scale and a line segment scale, and a scale pattern combining three virtual scales 147a, 147b, and 147c having only line segments, as shown in Fig. 15. The scale pattern is represented by one or a combination of a plurality of scale sizes and a plurality of scale shapes such as a circular scale and a line segment scale.

[0069] In addition, when the critical dimension image display setting is turned ON, it is preferable to store the image display setting before switching to the critical dimension mode in the pre-switching image display setting storage unit 149. For example, if the observation mode before switching to the critical dimension mode is the normal observation mode, it is preferable to store the image display setting of the normal observation mode set by the signal processing unit 39 in the pre-switching image display setting storage unit 149. In addition, when the critical dimension image display setting is turned OFF, it is preferable to switch to the image display setting stored in the pre-switching image display setting storage unit 149. For example, if the image display setting of the normal observation mode is stored in the pre-switching image display setting storage unit 149, the signal processing unit 39 sets the image display setting of the normal observation mode stored in the pre-switching image display setting storage unit 149 in accordance with switching to the normal observation mode.

[0070] On the other hand, when the mode switching conditions are not satisfied in the switching operation to the length measurement mode, the system control unit 41 prohibits switching the measurement light to ON, the length measurement image display setting to ON, the length measurement function operation status display to ON, and the display of the virtual scale to ON. The mode switching conditions are setting conditions related to the endoscope 12, the light source device 13, the processor device 14, and the extended processor device 17, and are conditions suitable for executing the length measurement mode. It is preferable that the mode switching conditions are conditions that do not correspond to the following prohibition setting conditions. Note that, when the mode switching conditions are not satisfied, instead of prohibiting the display of the scale display icon 146, it is preferable to display (ON) the length measurement function operation status unavailable display indicating that the virtual scale 147 is not being displayed on the extended display 18. It is preferable that the length measurement function operation status unavailable display is displayed by the scale hidden icon 148 in the incidental information display area 18a.

[0071] 16, the system control unit 41 is provided with a length measurement mode control unit 50 that controls whether or not to execute the length measurement mode. When a switching operation to the length measurement mode is performed by the observation mode change switch 12f, the length measurement mode control unit 50 performs at least one of the following: a first control for prohibiting switching to the length measurement mode when the currently set setting conditions for the endoscope 12, the light source device 13, and the processor device 14 correspond to a predetermined prohibited setting condition; a second control for invalidating the setting change operation when a setting change operation of the setting conditions is performed by the user interface 16 during the length measurement mode and the setting conditions to be changed by the setting change operation correspond to the prohibited setting condition; and a third control for automatically switching from the length measurement mode to another mode when a setting change operation of the setting conditions is performed by the user interface 16 during the length measurement mode and the setting conditions to be changed by the setting change operation correspond to the prohibited setting condition.

[0072] The setting conditions for the light source device 13 include the illumination conditions of the illumination light used in the normal observation mode or the length measurement mode, the illumination conditions of the special light used in the special observation mode, or the illumination conditions of the measurement light used in the length measurement mode. The illumination conditions include, for example, the illumination light amount. The setting conditions for the endoscope 12 include the imaging conditions for imaging the subject. The imaging conditions include, for example, the shutter speed. The setting conditions for the processor device 14 include processing conditions such as image processing for the subject image. The processing conditions include, for example, color balance, brightness correction, various emphasis processes, etc. In the length measurement mode, it is preferable to optimize the position detection of the spot SP and set the setting conditions (illumination light amount, shutter speed, color balance, brightness correction, various emphasis processes) that satisfy the visibility when the user measures the dimensions.

[0073] The prohibition setting conditions include a first prohibition setting condition that prevents detection of the irradiation position of the measurement light from the subject image in the length measurement mode, and a second prohibition setting condition that prevents accurate display of a virtual scale corresponding to the observation distance in the length measurement image. The first prohibition setting condition includes, for example, a special observation mode, brightness emphasis or red emphasis on the subject image, etc. In the special observation mode, the red image used for detecting spots SP in the length measurement mode is not used for image display, so that it is difficult to detect the irradiation position of the measurement light. Note that, in the length measurement mode, it is preferable to lower the brightness of the subject image and suppress the redness, as compared with the normal observation mode or the special observation mode.

[0074] The second prohibited setting condition includes, for example, the use (ON) of a zoom function such as an optical zoom function or a digital zoom function. This is because the virtual scale displayed in the measurement image is determined according to the position of the spot SP and is not determined according to the magnification of the zoom function, and therefore, when the zoom function is ON, it is difficult to display the virtual scale in accordance with the observation distance.

[0075] For example, when the observation mode changeover switch 12f is used to switch to the length measurement mode in the special observation mode, the length measurement mode control unit 50 performs a first control to maintain the special observation mode by prohibiting switching to the length measurement mode, as shown in Fig. 17. When the first control is performed, the length measurement mode control unit 50 displays a message notifying that switching to the length measurement mode is prohibited on the extended display 18 as shown in Fig. 18 (or may emit a warning sound). Note that, instead of prohibiting switching to the length measurement mode, the length measurement mode control unit 50 may cancel the special observation mode and perform a control to switch to the length measurement mode as shown in Fig. 19.

[0076] Furthermore, in the measurement mode, when a setting change operation to turn on the zoom function is performed by operating the zoom operation unit 12h, the measurement mode control unit 50 performs a second control to disable the setting change operation to turn on the zoom function, as shown in Fig. 20. When the second control is performed, the measurement mode control unit 50 displays a message on the extended display 18 to notify that the setting change operation to turn on the zoom function has been disabled (a warning sound may be emitted), as shown in Fig. 21.

[0077] Furthermore, when a setting change operation to turn on the zoom function is performed by operating the zoom operation unit 12h during the length measurement mode, the length measurement mode control unit 50 performs a third control to cancel the length measurement mode and switch to the normal observation mode as another mode, as shown in Fig. 22. When the third control is performed, the length measurement mode control unit 50 displays a message on the extended display 18 (or may emit a warning sound) to notify that the length measurement mode has been canceled (the virtual scale is hidden) and switched to the normal observation mode, as shown in Fig. 23. Note that when the third control is performed, the setting change operation to turn on the zoom function is enabled, and the subject on the subject image is enlarged or reduced by the zoom function.

[0078] As shown in FIG. 24, the system control unit 41 may be provided with a brightness information calculation unit 53, an illumination light quantity level setting unit 54, a first light emission control table 55, and a second light emission control table 56. The brightness information calculation unit 53 calculates brightness information regarding the brightness of the subject based on an image obtained in normal observation mode or a first captured image (an image based on illumination light and measurement light) obtained in length measurement mode. The illumination light quantity level setting unit 54 sets the illumination light quantity level based on the brightness information. There are five levels of illumination light quantity levels: Level 1, Level 2, Level 3, Level 4, and Level 5. Information regarding the illumination light quantity level is sent to the light source processor 31. The light source processor 31 controls the light source unit 30 so that the illumination light quantity matches the illumination light quantity level.

[0079] The first light emission control table 55 is used for controlling the amount of light of the measurement light, and stores a first relationship between the coordinate information of the spot SP and the light amount level of the measurement light. Specifically, as shown in FIG. 25, the light amount levels Level1, Level2, Level3, Level4, and Level5 of the measurement light are respectively defined for five coordinate areas to which the coordinate information of the spot SP belongs. The system control unit 41 refers to the first light emission control table 55 to specify the light amount level corresponding to the coordinate area to which the position of the spot SP belongs. The system control unit 41 controls the light source 23a to control the amount of light of the measurement light so that the specified light amount level is achieved. Note that the user interface 16 is appropriately set to use either the first light emission control table 55 or the second light emission control table 56 to control the amount of light of the measurement light.

[0080] As shown in Fig. 26, coordinate area 1 is the area set at the bottom in the first captured image, and when the spot SP belongs to coordinate area 1, it is located at the position with the shortest observation distance. Therefore, the smallest utility level of the measurement light, Level 1, is assigned to coordinate area 1. Coordinate area 2 is an area set above coordinate area 1, and when the spot SP belongs to coordinate area 2, it is located at the position with the farther observation distance than coordinate area 1, so that Level 2, which is higher than Level 1, is assigned as the light quantity level of the measurement light. The movement direction of the spot SP changes according to the intersecting direction of the optical axis Ax of the imaging optical system 21 and the optical axis of the measurement light Lm.

[0081] Similarly, coordinate area 3 is set above coordinate area 2. When the spot SP belongs to coordinate area 3, the observation distance is farther than in coordinate area 2, so Level 3, which is higher than Level 2, is assigned as the light intensity level of the measurement light. Coordinate area 4 is set above coordinate area 3. When the spot SP belongs to coordinate area 4, the observation distance is farther than in coordinate area 3, so Level 4, which is higher than Level 3, is assigned as the light intensity level of the measurement light. Coordinate area 5 is the area set at the top. When the spot SP belongs to coordinate area 5, the observation distance is farther than in coordinate area 1 to 4, so Level 5, which is the highest, is assigned as the light intensity level of the measurement light.

[0082] The second light emission control table 56 is used for controlling the amount of measurement light, and stores coordinate information of the spot SP and a second relationship between the light amount level of the illumination light and the light amount level of the measurement light. Specifically, as shown in Fig. 27, the light amount levels of the measurement light are determined for five coordinate areas to which the coordinate information of the spot SP belongs and the light amount levels of the illumination light, Level1, Level2, Level3, Level4, and Level5. For example, if the spot SP belongs to coordinate area 1 and the light amount level of the illumination light is Level3, Level3 is assigned as the light amount level of the measurement light.

[0083] The system control unit 41 refers to the second light emission control table 56 and specifies the light intensity level of the measurement light from the coordinate area to which the position of the spot SP belongs and the light intensity level of the illumination light. The system control unit 41 controls the light source 23a to control the light intensity of the measurement light to achieve the specified light intensity level.

[0084] In the second light emission control table 56, the light intensity level of the illumination light and the light intensity level of the measurement light are set to a ratio that enables the position of the spot SP to be specified. This is because if the ratio between the light intensity of the illumination light and the light intensity of the measurement light is not appropriate, the contrast of the spot SP will be low, making it difficult to specify the position of the spot SP.

[0085] In the length measurement mode, the light source processor 31 continuously emits the illumination light used for illuminating the entire observation target, while emitting the measurement light Lm in pulses. Therefore, the frames in which light is emitted in the length measurement mode include an illumination light only emission frame FLx in which the measurement light is not emitted and the illumination light is emitted alone, and a measurement light emission frame FLy in which the illumination light and the measurement light are emitted, as shown in FIG. 28. In the length measurement mode, the position of the spot SP is detected from the first captured image obtained in the measurement light emission frame FLy, while a virtual scale is displayed for the second captured image obtained in the illumination light only emission frame FLx. Note that the solid lines shown in the parts corresponding to the illumination light or the measurement light in FIG. 28 represent the light emission state in a certain frame. The period in which the solid line corresponds to "on" indicates the period in which the illumination light or the measurement light is emitted, and the period in which the solid line corresponds to "off" indicates the period in which the illumination light or the measurement light is not emitted.

[0086] The light emission and imaging patterns in the length measurement mode are as follows. In the first pattern, a CCD (global shutter type imaging element) that exposes each pixel and reads out the electric charge at the same timing to output an image signal is used as the imaging element 32. In the first pattern, the measurement light is emitted at a specific frame interval every two frames.

[0087] In the first pattern, as shown in Fig. 29, based on the exposure to illumination light at timing T1 in the normal observation mode, when the normal observation mode and the length measurement mode are switched (when switching from timing T1 to timing T2), electric charges are read out all at once (global shutter) to obtain a second captured image N having only the illumination light component. This second captured image N is displayed on the extended display 18 at timing T2. Note that, regarding the "CCD (frame period) global shutter" in Fig. 29, the rising line 57 rising vertically indicates that the global shutter was performed when switching from timing T1 to timing T2. This also applies to the other rising lines 57.

[0088] Also, at timing T2, illumination light and measurement light are emitted. Based on the exposure of illumination light and measurement light at timing T2, simultaneous readout of charges is performed when switching from timing T2 to timing T3, thereby obtaining a first captured image N+Lm containing components of illumination light and measurement light. Based on this first captured image N+Lm, position detection of spot SP is performed. A virtual scale corresponding to the position of spot SP is displayed for the second captured image N displayed at timing T2. As a result, at timing T3, a length measurement image S displaying the virtual scale is displayed for the second captured image N at timing T2.

[0089] The second captured image N at timing T2 (first timing) is displayed on the extended display 18 not only at timing T2 but also at timing T3. That is, the second captured image at timing T2 is displayed for two consecutive frames until timing T4 (second timing) when the next second captured image is obtained (the same subject image is displayed at timings T2 and T3). At timing T3, the first captured image N+Lm is not displayed on the extended display 18. Here, in the normal observation mode, the second captured image N is displayed in a different manner for each frame, whereas in the first pattern of the length measurement mode, the same second captured image N is displayed for two consecutive frames as described above, so that the frame rate of the first pattern of the length measurement mode is substantially half that of the normal observation mode.

[0090] The same process is carried out for timing T4 and after. That is, at timings T4 and T5, the second captured image at timing T4 is displayed consecutively with respect to the length measurement image S, and at timings T6 and T7, the second captured image N at timing T6 is displayed consecutively with respect to the length measurement image S. In contrast, at timings T4, T5, T6, and T7, the first captured image N+Lm is not displayed on the extended display 18. In this way, by displaying the second captured image N that does not include the component of the measurement light in the display of the length measurement image S, although the frame rate is somewhat reduced, the visibility of the observed object that may be caused by the emission of the measurement light is not hindered.

[0091] The second pattern is a case where a CMOS (rolling shutter type image sensor) is used as the image sensor 32, which has multiple lines for capturing an image of an observation target illuminated with illumination light or measurement light, performs exposure at different exposure timings for each line, reads out charges at different readout timings for each line, and outputs an image signal. Also, in the second pattern, the measurement light is emitted at a specific frame interval, every three frames.

[0092] In the second pattern, as shown in Fig. 30, exposure to illumination light and readout of electric charges are performed for each line at timing T1, and readout of electric charges is completed (rolling shutter) when switching from the normal observation mode to the length measurement mode (when switching from timing T1 to timing T2), thereby obtaining a second captured image N having only the illumination light component. This second captured image N is displayed on the extended display 18 at timing T2. Note that, for the "CMOS (frame period) rolling shutter" in Fig. 30, the diagonal line 58 represents the timing of exposure to light and readout of electric charges, with exposure and readout of electric charges starting at line Ls and completing at line Lt.

[0093] Also, at timing T2, illumination light and measurement light are emitted. By performing a rolling shutter based on the illumination of the illumination light from timing T1 to timing T2 and the illumination of the measurement light at timing T2, a first captured image N+Lm containing components of the illumination light and measurement light is obtained at the timing switching from timing T2 to timing T3. Also, a first captured image N+Lm containing components of the illumination light and measurement light is obtained at the timing switching from timing T3 to timing T4. Based on the above first captured image N+Lm, the position of the spot SP is detected. At timings T3 and T4, the measurement light is not emitted.

[0094] A virtual scale corresponding to the position of the spot SP is displayed for the second captured image N displayed at the time T2. As a result, at the times T3 and T4, a length measurement image S with the virtual scale displayed is displayed for the second captured image N at the time T2. The second captured image N at the time T2 (first timing) is displayed on the extended display 18 not only at the time T2 but also at the times T3 and T4. That is, the second captured image at the time T2 is displayed for three consecutive frames until the time T5 (second timing) at which the next second captured image is obtained (the same subject image is displayed at the times T2, T3, and T4). In contrast, at the times T3 and T4, the first captured image N+Lm is not displayed on the extended display 18. In the second pattern of the length measurement mode, the same second captured image N2 is displayed for three consecutive frames, so that the frame rate of the second pattern of the length measurement mode is substantially 1 / 3 of that of the normal observation mode.

[0095] The same is true for timing T5 and thereafter. At timings T5, T6, and T7, the second captured image at timing T5 is displayed for the length measurement image S. In contrast, at timings T5, T6, and T7, the first captured image N+Lm is not displayed on the extended display 18. In this way, by displaying the second captured image that does not include components of the measurement light in the display of the length measurement image S, the frame rate is reduced, but the visibility of the observed object, which may be impaired by the emission of planar measurement light, is not hindered.

[0096] 31, the signal processing unit 45 of the extension processor device 17 includes a first signal processing unit 59 that detects the position of the spot SP in the captured image and a second signal processing unit 60 that sets the virtual scale according to the position of the spot SP in order to recognize the position of the spot SP and set a virtual scale. Note that the captured image includes not only a captured image obtained when the illumination light and the measurement light are always on, but also a first captured image obtained when both the illumination light and the measurement light are on when the illumination light is always on and the measurement light is on or off.

[0097] The first signal processing unit 59 includes an irradiation position detection unit 61 that detects the irradiation position of the spot SP from the captured image. The irradiation position detection unit 61 preferably acquires the coordinates of the center of gravity of the spot SP as the irradiation position of the spot SP.

[0098] The second signal processing unit 60 sets a first virtual scale as a virtual scale for measuring the size of the subject based on the irradiation position of the spot SP, and sets the scale display position of the first virtual scale. The second signal processing unit 60 sets a virtual scale corresponding to the irradiation position of the spot SP by referring to a scale table 62 that stores a virtual scale image, the display mode of which differs depending on the irradiation position of the spot SP and the scale display position, in association with the irradiation position of the spot. The virtual scale differs in size or shape, for example, depending on the irradiation position of the spot SP and the scale display position. The display of the virtual scale image will be described later. The stored contents of the scale table 62 are maintained even when the power supply of the extension processor device 17 is turned off. The scale table 62 stores the virtual scale image in association with the irradiation position, but may also store the distance to the subject corresponding to the irradiation position (the distance between the tip 12d of the endoscope 12 and the subject) in association with the virtual scale image.

[0099] In addition, since the virtual scale image is required for each irradiation position and the data volume becomes large, it is preferable to store it in the extended processor device 17 (or the processor device 14) rather than in the memory (not shown) in the endoscope 12, taking into consideration the storage capacity of the memory in the endoscope 12, startup, and processing time. In addition, the virtual scale image is created from the representative point of the virtual scale image obtained by calibration as described later. However, if the virtual scale image is created from the representative point in the length measurement mode, a loss of time occurs and the real-time nature of the processing is impaired. Therefore, after the endoscope 12 is connected to the endoscope connection section and a virtual scale image is created from the representative point and the scale table 62 is updated, the virtual scale image is not created from the representative point, but the updated scale table 62 is used to display the virtual scale image. In addition, in the second signal processing section 60, in an emergency in which it is difficult to superimpose the image, instead of the virtual scale image to be displayed superimposed on the length measurement image, a reference marker that determines the size of the virtual scale based on the relationship between the irradiation position of the spot SP and the number of pixels corresponding to the actual size of the subject is displayed on the length measurement image.

[0100] The second signal processing unit 60 also includes a table update unit 64 for updating the scale table 62 when the endoscope 12 is connected to the endoscope connection unit. The scale table 62 can be updated in this manner because the positional relationship between the optical axis of the measurement light Lm and the imaging optical system 21 varies depending on the model and serial number of the endoscope 12, and the display mode of the virtual scale image also changes accordingly. The table update unit 64 uses a representative point data table 66 that stores representative point data related to the representative point extracted from the virtual scale image in association with the irradiation position. Details of the table update unit 64 and the representative point data table 66 will be described later. Note that the representative point data table 66 may store the distance to the subject corresponding to the irradiation position (the distance between the tip 12d of the endoscope 12 and the subject) in association with the representative point data.

[0101] When a length measurement image in which a virtual scale is superimposed on a captured image is displayed on the extended display 18, the display control unit 46 controls the virtual scale so that the display mode differs depending on the irradiation position of the spot SP and the scale display position. Specifically, the display control unit 46 displays a length measurement image in which a first virtual scale is superimposed on the spot SP on the extended display 18. For example, a circular measurement marker is used as the first virtual scale. In this case, as shown in FIG. 32, when the observation distance is close to the near end Px (see FIG. 13), a virtual scale M1 indicating an actual size of 5 mm (horizontal and vertical directions of the captured image) is displayed in alignment with the center of the spot SP1 formed on the tumor tm1 of the subject.

[0102] The scale display position of the virtual scale M1 is located at the periphery of the captured image that is affected by distortion caused by the imaging optical system 21, and therefore the virtual scale M1 is elliptical in consideration of the influence of distortion, etc. Since the above virtual scale M1 almost coincides with the range of the tumor tm1, the tumor tm1 can be measured to be about 5 mm. Note that it is also possible to display only the first virtual scale on the captured image without displaying the spots.

[0103] 33, when the observation distance is close to the center Py, a virtual scale M2 indicating an actual size of 5 mm (horizontal and vertical directions of the captured image) is displayed in alignment with the center of a spot SP2 formed on a tumor tm2 of a subject. Since the scale display position of the virtual scale M2 is located in the center of the captured image which is less susceptible to distortion caused by the imaging optical system 21, the virtual scale M2 is circular and not affected by distortion.

[0104] Also, as shown in Fig. 34, a virtual scale M3 indicating an actual size of 5 mm (horizontal and vertical directions of the captured image) is displayed in alignment with the center of a spot SP3 formed on a tumor tm3 of a subject. Since the scale display position of the virtual scale M3 is located in the peripheral part of the captured image that is affected by distortion caused by the imaging optical system 21, the virtual scale M3 has an elliptical shape in accordance with the influence of the distortion, etc. As shown in Figs. 32 to 34 above, the size of the first virtual scale corresponding to the same actual size of 5 mm becomes smaller as the observation distance becomes longer. Also, the shape of the first virtual scale differs depending on the scale display position in accordance with the influence of distortion caused by the imaging optical system 21.

[0105] 32 to 34, the center of the spot SP and the center of the marker are displayed to coincide with each other, but if there is no problem in terms of measurement accuracy, the first virtual scale may be displayed at a position away from the spot SP. However, even in this case, it is preferable to display the first virtual scale near the spot. Also, instead of displaying the first virtual scale after modifying it, the first virtual scale may be displayed on the corrected captured image after correcting the distortion aberration.

[0106] 32 to 34, the first virtual scale corresponding to the actual size of the subject, which is 5 mm, is displayed, but the actual size of the subject may be set to any value (for example, 2 mm, 3 mm, 10 mm, etc.) depending on the observation target and the observation purpose. In addition, in FIG. 32 to FIG. 34, the first virtual scale is substantially circular, but as shown in FIG. 35, it may be a cross shape in which vertical and horizontal lines intersect. Also, it may be a graduated cross shape in which at least one of the vertical and horizontal lines of the cross is provided with a scale Mx. Also, the first virtual scale may be a distorted cross shape in which at least one of the vertical and horizontal lines is tilted. Also, the first virtual scale may be a circle and a cross shape in which a cross shape and a circle are combined. In addition, the first virtual scale may be a measurement point cloud type in which a plurality of measurement points EP corresponding to the actual size are combined from the spot. Also, the number of the first virtual scales may be one or more, and the color of the first virtual scale may be changed depending on the actual size.

[0107] As the first virtual scale, as shown in FIG. 36, three concentric virtual scales M4A, M4B, and M4C (each having a diameter of 2 mm, 5 mm, and 10 mm) having different sizes may be displayed on the captured image with a spot SP4 formed on the tumor tm4 at the center. Since the three concentric virtual scales are displayed in multiple numbers, the time required for switching is eliminated, and measurement is possible even when the subject has a nonlinear shape. When multiple concentric virtual scales are displayed with a spot at the center, a size and a color are not specified for each virtual scale, but multiple combinations of conditions may be prepared in advance and selected from the combinations.

[0108] In Fig. 36, all three concentric virtual scales are displayed in the same color (black), but when multiple concentric markers are displayed, multiple colored concentric markers may be used with different colors depending on the virtual scale. As shown in Fig. 37, the virtual scale M5A is displayed with a dotted line representing red, the virtual scale M5B with a solid line representing blue, and the virtual scale M5C with a dashed line representing white. By changing the color of the virtual scales in this way, the distinction is improved, and measurements can be easily performed.

[0109] As the first virtual scale, in addition to a plurality of concentric virtual scales, a plurality of distorted concentric virtual scales obtained by distorting each concentric circle may be used as shown in Fig. 38. In this case, distorted concentric virtual scales M6A, M6B, and M6C are displayed on the captured image with a spot SP5 formed on the tumor tm5 at the center.

[0110] When the endoscope 12 is connected to the endoscope connection section, the table update section 64 refers to the representative point data table 66, creates a virtual scale image corresponding to the model and / or serial number of the endoscope 12, and updates the scale table 62.

[0111] The representative point data table 66 stores representative point data relating to the representative points of the virtual scale image obtained during calibration in association with the irradiation positions of the spots SP. The representative point data table 66 is created by a calibration method described later. As shown in Fig. 39, the representative point data includes coordinate information (X coordinate, Y coordinate) of representative points RP obtained by extracting some points from an image M of a circular virtual scale, which is the virtual scale image. The representative point data stored in the representative point data table 66 is data when the positional relationship between the optical axis of the measurement light Lm and the imaging optical system 21 is a default positional relationship.

[0112] When the endoscope 12 is connected to the endoscope connection section, the table update section 64 acquires information regarding the positional relationship between the optical axis of the measurement light Lm and the imaging optical system 21, and updates the scale table 62 using the positional relationship and the representative point data table 66. Specifically, based on the difference between the positional relationship between the optical axis of the measurement auxiliary light Lm and the imaging optical system 29b in the endoscope 12 connected to the endoscope connection section and the default positional relationship, the table update section 64 calculates a difference value of the coordinate information of the representative point RP, which is obtained by shifting the coordinates of the default representative point RP by the calculated difference value, as shown in FIG. * Based on the virtual scale image M * The virtual scale image is created by using the representative point RP * It is preferable to perform an interpolation process to connect the two points. The created virtual scale image after the interpolation process is associated with the irradiation position by the table update unit 64. This completes the update of the scale table 62. In FIG. 40, the representative points RP, RP * Only a portion of the above is labeled.

[0113] In addition, the measurement light used is a light that is formed as a spot when irradiated on the subject, but other light may be used. For example, as shown in FIG. 41, a planar measurement light that is formed as a cross line 67 on the subject when irradiated on the subject may be used. In this case, a second virtual scale is generated as a virtual scale, which includes the cross line 67 and a scale 68 on the cross line 67 that is an index of the size of the subject (e.g., polyp P). When the planar measurement light is used, the irradiation position detection unit 61 detects the position of the cross line 67 (the irradiation position of the measurement light). The lower the cross line 67 is located, the closer the observation distance is, and the higher the cross line 67 is located, the farther the observation distance is. Therefore, the lower the cross line 67 is located, the larger the interval of the scale 68 is, and the higher the cross line 67 is located, the smaller the interval of the scale 68 is.

[0114] Also, the measurement light may be a planar light including at least two first characteristic lines CL, as shown in FIG. 42. When the measurement light is irradiated onto the subject, an intersection curve CC is formed according to the undulations on the subject, and a first spot SPk1 is formed on the intersection curve CC at a position corresponding to each of the two first characteristic lines CL1. The measurement light also includes a plurality of second characteristic lines CL2 different from the first characteristic line CL1 between the two first characteristic lines CL1. When the measurement light including the first characteristic line CL1 and the second characteristic line CL2 is irradiated onto the subject, a second spot SPk2 is formed at a position corresponding to each of the plurality of second characteristic lines CL2. The second spot SPk2 is smaller than the first spot SPk1, and the interval between the second spots SPk2 is small. Therefore, a specific intersection curve SCC is formed on the intersection curve by the plurality of second spots SPk2. Measurement information is calculated based on the position of the specific intersection curve SCC.

[0115] 43, the signal processing unit 45 of the extension processor device 17 has a position identification unit 69 and a measurement information processing unit 70 in order to recognize the position of the first spot SPk1 or the second spot SPk2 and calculate the measurement information. The position identification unit 69 identifies the position of the first spot SPk1 or the second spot SPk2 from the captured image. As a method of identifying the position, for example, the captured image is binarized and the center of gravity of a white part (a pixel whose signal intensity is higher than a binarization threshold) in the binarized image is identified as the position of the first spot SPk1 or the second spot SPk2.

[0116] The measurement information processing unit 70 calculates measurement information from the position of the first spot SPk1 or the second spot SPk2. The calculated measurement information is displayed on the captured image by the display control unit 46. When the measurement information is calculated based on the positions of the two first spots SPk1, it is possible to accurately calculate the measurement information even in a situation where the subject has a three-dimensional shape.

[0117] As shown in FIG. 44, the measurement information includes a first linear distance indicating a linear distance between two spots, the first spot SPk1 and the second spot SPk2. The measurement information processing unit 70 calculates the first linear distance by the following method. As shown in FIG. 45, the measurement information processing unit 70 obtains coordinates (xp1, yp1, zp1) indicating the actual size of the first spot SPk1 based on the position of the first spot SPk1. For xp1 and yp2, coordinates corresponding to the actual size are obtained from the coordinates of the position of the first spot SPk1 in the captured image. For zp1, coordinates corresponding to the actual size are obtained from the coordinates of the position of the first spot SPk1 and the coordinates of the position of the predetermined specific spot SPk. Similarly, based on the position of the second spot SPk2, coordinates (xp2, yp2, zp2) indicating the actual size of the second spot SPk2 are obtained. Also, for xp2 and yp2, coordinates corresponding to the actual size are obtained from the coordinates of the position of the second spot SPk2 in the captured image. For zp2, coordinates corresponding to the actual size are obtained from the coordinates of the position of the second spot SPk2 and the coordinates of the position of the predetermined specific spot SPk. Then, the first linear distance is calculated by the following formula). Formula) 1st straight line distance = ((xp2-xp1) 2 +(yp2-yp1) 2 +(zp2-zp1) 2 ) 0.5 The calculated first linear distance is displayed on the captured image as measurement information 71 ("20 mm" in FIG. 44). The specific spot SPk may or may not be displayed on the extended display 18.

[0118] Furthermore, the measurement light may be a plurality of spot lights arranged in a grid pattern at a predetermined interval in the vertical and horizontal directions. By capturing an image of a tumor tm or the like in the subject with the spot lights arranged in a grid pattern, an image of diffraction spots DS1 is obtained as shown in FIG. 46. The signal processing unit 45 of the extension processor device 17 measures the interval DT between the diffraction spots DS1. The interval DT corresponds to the number of pixels on the imaging surface of the imaging element 32. Note that the interval at a specific portion of the plurality of diffraction spots DS1 (for example, the interval near the center of the imaging surface) may be measured.

[0119] When the interval between the diffraction spots DS1 is measured, the direction and distance to the subject are calculated based on the measurement result. This process uses the relationship between the interval (number of pixels) between the diffraction spots DS1 and the distance to the subject. Specifically, as shown in FIG. 47, the direction (α, β) and distance (r) of the diffraction spots DS1 to be measured are calculated. When the direction and distance to the subject are calculated, two-dimensional or three-dimensional information of the subject is calculated based on the calculated direction and distance. As the two-dimensional or three-dimensional information of the subject, the shape, size, area, etc. of the subject in the two-dimensional space (XY plane in FIG. 47) or in the three-dimensional space (XYZ space in FIG. 47) can be calculated. Note that the conversion from (α, β, r) to (X, Y, Z) can be performed by the following formulas A), B), and C), and the shape, size, area, etc. can be calculated from the (X, Y, Z) coordinates of each point of the subject. Formula A)X=r×cosα×cosβ Formula B) Y=r×cosα×sinβ Formula C) Z=r×sinα

[0120] As shown in FIG. 48, the signal processing unit 45 of the extension processor device 17 includes a position identification unit 72 that identifies the position of the spot SP in the first captured image (an image based on measurement light and illumination light) in order to recognize the position of the spot and set a virtual scale, and an image processing unit 73 that processes the first captured image or the second captured image (an image based on illumination light) based on the position of the spot SP to generate a measurement image.

[0121] The position identifying unit 72 has a noise component removing unit 74 that removes noise components that hinder the identification of the position of the spot SP. When the first captured image contains a color that is different from the color of the measurement light that forms the spot SP but is close to the color of the measurement light (a measurement light approximate color), the position of the spot SP may not be identified accurately. Therefore, the noise component removing unit 74 removes the component of the measurement light approximate color as a noise component from the first captured image. The position identifying unit 72 identifies the position of the spot SP based on the first captured image from which the noise components have been removed.

[0122] The noise component removal unit 74 includes a color information conversion unit 75, a binarization processing unit 76, a mask image generation unit 77, and a removal unit 78. The flow of processing for obtaining the first captured image from which noise has been removed will be described with reference to FIG. 49. The color information conversion unit 75 converts the first captured image, which is an RGB image, into a first color information image, and converts the second captured image, which is an RGB image, into a second color information image. For example, the color information is preferably HSV (H (Hue), S (Saturation), V (Value)). Alternatively, the color information may be color differences Cr and Cb.

[0123] A binarization processing unit 76 binarizes the first color information image to generate a binarized first color information image, and binarizes the second color information image to generate a binarized second color information image. The threshold value for binarization is a binarization threshold value that includes the color of the measurement assist light. As shown in Fig. 50 and Fig. 51, the binarized first color information image includes color information 80 of the noise component in addition to color information 79 of the measurement light.

[0124] The mask image generating unit 77 removes color information of noise components from the first captured image based on the binarized first color information image and the binarized second color information image, and generates a mask image for extracting color information of the measurement light. As shown in FIG. 52, the mask image generating unit 77 specifies a noise component region 81 having noise components from the noise components contained in the binarized second captured image. It is preferable that the noise component region 81 is larger than the region occupied by the noise component color information 80. This is because, when camera shake or the like occurs, the region of the noise component color information 80 becomes larger compared to when there is no camera shake or the like. Then, as shown in FIG. 53, the mask image generating unit 77 generates a mask image in which the region of the measurement light color information 79 in the binarized first color information image is set as an extraction region from which color information is extracted, and the noise component region 81 is set as a non-extraction region from which color information is not extracted. Note that Figs. 50 to 53 are schematic diagrams for explaining the binarized first color information image, the binarized second color information image, the noise component region, and the mask image.

[0125] The removal unit 78 extracts color information from the first color information image using the mask image, thereby removing color information of noise components and obtaining a noise-removed first color information image from which color information of the measurement light is extracted. The noise-removed first color information image becomes a noise-removed first captured image by performing an RGB conversion process that returns the color information to an RGB image. The position identification unit 72 identifies the position of the spot SP based on the noise-removed first captured image. Since the noise components have been removed from the noise-removed second captured image, the position of the spot SP can be accurately identified.

[0126] The image processing unit 73 has an image selection unit 82 and a scale table 62. The image selection unit 82 selects a processing target image, which is an image to be processed based on the position of the spot SP, from the first captured image or the second captured image. The image processing unit 73 processes the image selected as the processing target image based on the position of the spot SP. The image selection unit 82 selects the processing target image based on the state related to the position of the spot SP. Note that the image selection unit 82 may select the processing target image in response to an instruction from the user. For example, the user interface 16 is used for the instruction from the user.

[0127] Specifically, when the spot SP is within a specific range during a specific period, it is considered that there is little movement of the subject or the tip 12d of the endoscope, and therefore the second captured image is selected as the image to be processed. When there is little movement like this, it is considered that even without the spot SP, it is possible to easily align the lesion part included in the subject. In addition, since the second captured image does not include the color components of the measurement light, the color reproducibility of the subject is not impaired. On the other hand, when the position of the spot SP is not within the specific range during a specific period, it is considered that there is a large movement of the subject or the tip 12d of the endoscope, and therefore the first captured image is selected as the image to be processed. When there is a large movement like this, the user operates the endoscope 12 so that the spot SP is located at the lesion part. This makes it easier to align the spot SP to the lesion part.

[0128] The image processing unit 73 generates a first virtual scale indicating the actual size of the subject as a virtual scale based on the position of the spot SP in the first captured image. The image processing unit 73 calculates the size of the virtual scale from the position of the spot SP by referring to the scale table 62 that stores the relationship between the position of the spot SP in the first captured image and the first virtual scale indicating the actual size of the subject. Then, the image processing unit 73 generates the first virtual scale corresponding to the size of the virtual scale.

[0129] As shown in FIG. 54, the signal processing unit 45 of the extended processor device 17 includes a first signal processing unit 84 that detects the position of the spot SP in the captured image and a second signal processing unit 85 that sets the virtual scale according to the position of the spot SP in order to recognize the position of the spot SP and set a virtual scale.

[0130] The first signal processing unit 84 includes a mask processing unit 86, a binarization processing unit 87, a noise component removal unit 88, and an irradiation position detection unit 89. The process of removing noise components in the first signal processing unit 84 will be described with reference to Figs. 55 to 57. The mask processing unit 86 performs mask processing on the red image, the green image, and the blue image of the captured image to extract a substantially parallelogram-shaped illumination position movable range Wx indicating the movable range of the illumination position of the measurement light on the subject. As a result, as shown in Figs. 56 and 57, a red image PRx, a green image PGx, and a blue image PBx after the mask processing from which the illumination position movable range Wx is extracted are obtained. For pixels within the illumination position movable range, noise components are removed, and the irradiation position of the spot SP is detected.

[0131] Next, the binarization processing unit 87 performs a first binarization process on pixels in the range in which the illumination position can be moved out of the red image PRx after the mask process, thereby obtaining a binarized red image PRy (binarized first spectral image). In the first binarization process, as a threshold condition for the first binarization process, pixels whose pixel value is equal to or greater than "225" are set to "1", and pixels whose pixel value is less than "225" are set to "0". This first binarization process detects the spot SP, which is a component of the measurement light. However, in the first binarization process, in addition to the first noise component N1, which is a high-luminance component of the red component of the illumination light, the second noise component N2, which is blown-out (pixel saturation) caused by the illumination light, is also detected. These first and second noise components are factors that hinder detection of the irradiation position of the spot SP. The threshold conditions refer to conditions related to the threshold that indicate the boundary between the pixel values ​​of pixels that are to be binarized to "0" and the pixel values ​​of pixels that are to be binarized to "1", as well as conditions that determine the range of pixel values ​​of pixels that are to be binarized to "0" and the range of pixel values ​​of pixels that are to be binarized to "1".

[0132] Therefore, in order to remove the first noise component, the noise component removal unit 88 performs a first difference process between the binarized red image PRy and the binarized green image PGy (binarized second spectral image) obtained by binarizing the green image PGx by the second binarization process. The first difference image PD1 obtained by the first difference process has the first noise component N1 removed. However, in the first difference image PD1, the second noise component N2 often remains and is not removed. For pixels that are "0" or less by the first difference process, the pixel value is set to "0". In the second binarization process, as a threshold condition for the second binarization process, pixels whose pixel value is within the range of "30" to "220" are set to "1", and pixels in other ranges, that is, pixels whose pixel value is less than "0" to "30" or exceeds "220" are set to "0". In addition, the first noise component is removed by the first difference process between the binarized red image and the binarized green image, but the first noise component may be removed by other first calculation processes.

[0133] In order to remove the second noise component, as shown in FIG. 57, the noise component removal unit 88 performs a second difference process between the first difference image PD1 and a binarized blue image PBy obtained by binarizing the blue image PBx by the third binarization process. The second difference image PD2 obtained by the second difference process has the second noise component removed, which was difficult to remove by the first difference process. As in the first difference process, the pixel value of pixels that become "0" or less by the second difference process is set to "0". Note that in the third binarization process, as a threshold condition for the third binarization process, pixels with a pixel value of "160" or more are set to "1", and pixels with a pixel value below "160" are set to "0". In addition, the second noise component is removed by the second difference process between the first difference image and the binarized blue image, but the second noise component may be removed by another second calculation process.

[0134] The irradiation position detection unit 89 detects the irradiation position of the spot SP from the first differential image or the second differential image. It is preferable that the irradiation position detection unit 89 acquires the coordinates of the center of gravity of the spot SP as the irradiation position of the spot SP.

[0135] The second signal processing unit 85 sets a first virtual scale indicating the actual size of the subject as a virtual scale based on the position of the spot SP. The second signal processing unit 85 refers to the scale table 62 that stores the relationship between the position of the spot SP and the first virtual scale indicating the actual size of the subject, and calculates the size of the virtual scale from the position of the spot. Then, the second signal processing unit 85 sets the first virtual scale corresponding to the size of the virtual scale.

[0136] As shown in FIG. 58, the signal processing unit 45 of the extension processor device 17 includes an irradiation area recognition unit 90 and a second signal processing unit 60. The irradiation area recognition unit 90 recognizes a measurement light irradiation area having a pattern of a specific shape from a captured image. Specifically, as shown in FIG. 59, the pattern of the specific shape includes a white central area CR1 and a peripheral area SR1 that covers the periphery of the central area and has a feature amount based on the measurement light. If the measurement light irradiation area is the above-mentioned spot SP, the pattern of the specific shape has a circular shape. In this case, the white central area CR1 is circular, and the peripheral area SR1 is ring-shaped.

[0137] FIG. 60 shows the distribution of pixel values ​​of each color image in a captured image including a red image RP, a green image GP, and a blue image BP as multiple color images. The pixel values ​​of the red image RP, the green image GP, and the blue image BP in the central region CR1 reach the maximum pixel value (for example, 255), so that the central region CR1 is white. In this case, when the measurement light is incident on the image sensor 32, as shown in FIG. 61, the measurement light is transmitted at the maximum transmittance of not only the red color filter RF of the image sensor 32 but also the green color filter GF and the blue color filter BF in the wavelength range WMB of the measurement light. On the other hand, in the peripheral region SR1, the pixel value of the red image RP is larger than the pixel value of the green image GP or the blue image BP. Therefore, the peripheral region SR1 is reddish. Note that the light source 23a emits the measurement light with a specific light amount, so that the pixel values ​​of the red image RP, the green image GP, and the blue image BP in the central region CR1 are the maximum pixel values.

[0138] The irradiation area recognition unit 90 can recognize the spot SP having the specific shape and feature amount as described above. Specifically, as shown in Fig. 62, the irradiation area recognition unit 90 preferably has a learning model 91 that recognizes the spot SP by outputting the spot SP, which is a measurement light irradiation area, in response to an input of a captured image. The learning model 91 is machine-learned by a large amount of teacher data that associates the captured image with the measurement light irradiation area that has already been recognized. As the machine learning, it is preferable to use a CNN (Convolutional Neural Network).

[0139] By using the learning model 91 to recognize the spot SP, it is possible to recognize not only a circular spot SP (see FIG. 59) consisting of a circular central region CR1 and a ring-shaped peripheral region SR1, but also a spot SP with a pattern deformed from a specific circular shape. For example, as shown in FIG. 63(A), a spot SP deformed in the vertical direction can be recognized. Also, as shown in FIG. 63(B), a spot SP deformed by missing a part of the circle can be recognized. Also, as the feature amount of the peripheral region SR1 that can be recognized by the learning model 91, in addition to red, which is the color of the measurement light, there are blue and green. Also, as the feature amount of the peripheral region SR1 that can be recognized by the learning model 91, there are the luminance, lightness, saturation, and hue of the measurement light. It is preferable to obtain the luminance, lightness, saturation, and hue of the measurement light by performing a luminance conversion process or a lightness, saturation, and hue conversion process on the peripheral region of the spot SP included in the captured image.

[0140] As shown in FIG. 64, the signal processing unit 45 of the extended processor device 17 is equipped with a position identification unit 92 that identifies the position of the spot SP in the captured image and calculates the observation distance in order to recognize the position of the spot SP, calculate the observation distance from the subject, and set a virtual scale, and an image processing unit 93 that sets various virtual scales based on the observation distance and generates a measurement image by processing the captured image using the various virtual scales.

[0141] The position identifying unit 92 has a distance calculation unit 94. The position identifying unit 92 identifies the position of a spot SP formed on the subject by the measurement light, based on a captured image in which the subject is illuminated by the illumination light and the measurement light. The distance calculation unit 94 calculates the observation distance from the position of the spot SP.

[0142] The image processing unit 93 has an image selection unit 95, a scale table 62, an offset setting unit 97, an offset distance calculation unit 98, and an offset virtual scale generation unit 99. The image selection unit 95 selects an image to be processed based on the position of the spot SP. The offset setting unit 97 sets an offset amount according to the height of the spot SP of the convex polyp 100 for the observation distance. The offset distance calculation unit 98 calculates the offset distance by adding the offset amount to the observation distance. The offset virtual scale generation unit 99 generates a virtual scale for offset based on the offset distance.

[0143] The offset will be explained below. First, the convex shape of the subject refers to a shape that protrudes from the surroundings of the subject. Therefore, as long as there is at least a part of the shape that protrudes from the surroundings, the size, width of the shape, height and / or number of the protruding parts, continuity of the height, etc., and other shapes are not important.

[0144] More specifically, for example, as shown in FIG. 65, the subject has a polyp 100 as a convex shape. The polyp 100 has a shape that protrudes from the surrounding subject. The polyp 100 has a top portion 100a and a flat portion 100b. FIG. 65 shows the polyp 100 viewed from the horizontal direction when the height direction of the polyp 100 is the vertical direction, but since the polyp 100 is a three-dimensional object, it also exists in the front direction and the back direction of the paper. The plane formed by the flat portion of the subject around the polyp 100, where the polyp 100 is formed, is the flat portion 100b of the polyp 100. The plane formed by the flat portion around the polyp 100 is the extension plane 101 of the flat portion 100b.

[0145] Next, the height of the spot SP of the convex polyp 100 will be described. In this embodiment, the height of the spot SP of the polyp 100 is the vertical distance from the spot SP of the polyp 100 to the flat part 100b of the polyp 100. More specifically, as shown in FIG. 66, the spot SP1 is formed on the top 100a of the polyp 100. Therefore, if a plane that is parallel to the extended surface 101 and passes through the top 100a of the polyp 100 is taken as a parallel surface 102, the distance between the parallel surface 102 and the extended surface 101 is the height HT1 of the spot SP of the polyp 100 (the vertical distance from the top 100a of the polyp 100 to the flat part 100b). The polyp 100 and the height HT1 (the same applies to the height HT2 described below) are shown merely for illustrative purposes, and the type, shape, size, and the like of the convex shape are not important so long as the polyp 100 protrudes from the surrounding area.

[0146] 66, the spot SP2 is formed in an area other than the top 100a of the polyp 100. That is, the spot SP2 is formed between the top 100a of the polyp 100 and the end of the polyp 100. Therefore, if a plane that is parallel to the extended plane 101 and passes through the spot SP2 of the polyp 100 is a parallel plane 103, the distance between the parallel plane 103 and the extended plane 101 is the height HT2 of the spot SP2 of the polyp 100. Therefore, the height HT2 of the spot SP2 of the polyp 100 is the vertical distance from the spot SP2 of the polyp 100 to the flat portion 100b.

[0147] The observation distance and the offset amount will be described below. As shown in FIG. 67, a spot SP1 is formed on the top 100a of the polyp 100 by the measurement light Lm. The observation distance obtained from the spot SP1 is the distance D5 between the position P1 of the tip 12d of the endoscope and the position P2 of the spot SP1. The virtual scale according to the distance D5 is a virtual scale that matches the actual measurement on the parallel surface 102. Therefore, when the spot SP1 is formed on the top 100a of the polyp 100, if a virtual scale according to the distance D5 is generated and displayed, a virtual scale that matches the actual measurement of the object on the parallel surface 102 is displayed. Therefore, a virtual scale with a scale or the like shifted to the larger side with respect to the actual measurement of the object on the extended surface 101 is displayed.

[0148] Therefore, the offset setting unit 97 sets the height HT1 of the spot SP of the polyp 100 as an offset amount for the observation distance D5. Next, the offset distance calculation unit 98 calculates the offset distance D6 by adding the height HT1 of the spot SP1 of the polyp 100, which is the offset amount, to the observation distance D5. Therefore, the offset distance calculation unit 98 calculates the offset distance D6 by the following formula OS). Note that HT1 is the distance between positions P2 and P3. Formula OS)D6=D5+HT1

[0149] Next, the virtual scale for offset generation unit 99 generates a virtual scale based on the observation distance D6 as a virtual scale for offset. More specifically, the virtual scale for offset generation unit 99 refers to the scale table 62, and uses a virtual scale when the observation distance is distance D6, and sets this as the virtual scale for offset. The virtual scale for offset indicates the actual distance or size of the subject on the extension plane 101.

[0150] The image processing unit 93 processes the generated virtual scale for offsetting to be superimposed on the captured image to generate a length measurement image. For more accurate measurement, it is preferable that the virtual scale for offsetting is superimposed so as to be displayed at the position where the spot SP is formed. Therefore, when the virtual scale for offsetting is displayed at a position away from the spot SP, it is displayed as close to the spot SP as possible. The length measurement image with the virtual scale for offsetting superimposed thereon is displayed on the extended display 18 by the display control unit 46.

[0151] As shown in Figure 68, for the virtual scale, the number of pixels of the width of the lines that make up the virtual scale may be set to be larger as the observation distance becomes smaller, and the number of pixels of the width of the lines that make up the virtual scale may be set to be smaller as the observation distance becomes larger.

[0152] In the example shown in Fig. 68(A), when the observation distance is 21 mm or more at the far end Px of the range Rx, the width W11 of the lines constituting the virtual scale M11 is set to 1 pixel, which is the smallest setting value. In the example shown in Fig. 68(B), when the observation distance is 13 mm to 20 mm between the far end Pz and the vicinity of the center Py of the range Rx, the width W12 of the lines constituting the virtual scale M12 is set to 2 pixels. In the example shown in Fig. 68(C), when the observation distance is 8 mm to 12 mm at the vicinity of the center Py of the range Rx, the width W13 of the lines constituting the virtual scale M13 is set to 3 pixels, which is the intermediate setting value.

[0153] In the example shown in Fig. 68(D), when the observation distance is between the center Py and the near end Px of the range Rx and is 4 to 7 mm, the width W14 of the lines constituting the virtual scale M14 is set to 4 pixels. In the example shown in Fig. 68(E), when the observation distance is 3 mm or less at the near end Px of the range Rx, the width W15 of the lines constituting the virtual scale M15 is set to 5 pixels, which is the largest setting value.

[0154] As described above, the lines constituting the virtual scales M11-M15 are changed according to the observation distance, making it easier for the doctor, who is the user, to measure the exact dimensions of the subject. Furthermore, the widths W11-W15 of the lines constituting the virtual scales M11-M15 are set to values ​​inversely proportional to the observation distance, making it possible to recognize the magnitude of the dimensional error from the line width. For example, taking into account the recognized error, if the tumor tm is located inside the lines constituting the virtual scales M11-M15, it can be seen that it is definitely smaller than the set actual size (within 5 mm in the example shown in FIG. 68).

[0155] Also, as shown in Fig. 69, a concentric virtual scale M2 consisting of three concentric circles of different sizes may be set based on the position of one spot SP. The three concentric circles M21, M22, and M23 constituting the virtual scale M2 represent actual sizes of, for example, "5 mm," "10 mm," and "20 mm." For convenience of illustration, the concentric circles M21, M22, and M23 constituting the virtual scale M2 are cross-hatched on the lines constituting the circular shape, but in reality, each line is filled with one color.

[0156] The width W22 of the concentric circle M22 located immediately outside the innermost concentric circle M21 is set to be larger than the width W21 of the concentric circle M21, and the width W23 of the outermost concentric circle M23 is set to be larger than the width W22 of the concentric circle M22. In the example shown in Fig. 69, the width W23 is set to a value that is √2 times the width W22 and twice the width W21.

[0157] It is set to a value inversely proportional to the observation distance while maintaining the ratio of widths W21 to W23 (in the example shown in FIG. 69, the ratio is W21:W22:W23=1:√2:2). This makes it possible to recognize the magnitude of the dimensional error from the line width. For example, taking into account the recognized error, if the tumor tm is inside the line that constitutes the concentric circle M22 and outside the line that constitutes the concentric circle M23, it can be known with certainty that it is within the set range of actual size (in the example shown in FIG. 69, 10 mm or more and 20 mm or less).

[0158] 70, the lines constituting the virtual scale M3 may be provided with a gradation in which the density gradually decreases from the center in the width direction of the line toward the outside. Then, as in the above-described embodiment, the width of the line with the gradation is changed according to the observation distance.

[0159] As shown in Fig. 71, the lines constituting the virtual scales M41 to M43 are dashed lines, and the gaps constituting the dashed lines are set to a value inversely proportional to the observation distance. Fig. 71 shows circular virtual scales M41, M42, and M43 when images are captured at the far end Pz, the vicinity of the center Py, and the near end Px in the observation distance range Rx.

[0160] In the example shown in FIG. 71(A), in the case of the far end Pz in the range Rx, the gap G1 of the dashed lines constituting the virtual scale M41 is set to the smallest setting value. Note that in the example shown in FIG. 71(A), the virtual scale M41 is constituted by dashed lines, but only in the case of the far end Pz, the gap G1 may be set to 0, that is, the virtual scale M41 may be constituted by solid lines. In the example shown in FIG. 71(B), in the case of the vicinity Py of the center in the range Rx, the gap G2 of the dashed lines constituting the virtual scale M42 is set to the intermediate setting value. In the example shown in FIG. 71(C), in the case of the near end Px in the range Rx, the gap G3 of the dashed lines constituting the virtual scale M43 is set to the largest setting value.

[0161] As described above, the gaps G1 to G3 between the dashed lines constituting the virtual scales M41 to M43 are set to values ​​that are inversely proportional to the observation distance, so that it is possible to recognize the magnitude of the dimensional error from the gaps between the dashed lines.

[0162] The number of lines constituting the virtual scale is the same regardless of the observation distance. As shown in Fig. 72, the number of lines constituting the virtual scales M51 to M53 is changed according to the observation distance. Note that Fig. 72 shows the virtual scales M51, M52, and M53 when images are captured at the far end Pz, the vicinity of the center Py, and the near end Px in the observation distance range Rx.

[0163] In the example shown in FIG. 72(A), in the case of the far end Pz in the range Rx, the virtual scale M51 is composed of three lines, that is, three concentric circles of different sizes. The three concentric circles represent actual sizes of, for example, "5 mm", "10 mm", and "20 mm". In the example shown in FIG. 72(B), in the case of the center vicinity Py in the range Rx, the virtual scale M52 is composed of two lines, that is, two concentric circles of different sizes. The two concentric circles represent actual sizes of, for example, "5 mm" and "10 mm". In the example shown in FIG. 72(C), in the case of the near end Px in the range Rx, the virtual scale M53 is composed of one line, that is, one circle. The one circle represents an actual size of, for example, "5 mm". As described above, the number of lines constituting the virtual scales M51 to M53 is set to a value proportional to the observation distance, so that it is possible to recognize the magnitude of the dimensional error from the number of lines.

[0164] As shown in FIG. 73, the signal processing unit 45 of the extension processor device 17 includes a position specifying unit 92 including a distance calculation unit 94, and an image processing unit 104. The image processing unit 104 includes an image selection unit 95, a scale table 62, a virtual scale setting unit 105, a virtual scale switching reception unit 106, and a length measurement image creation unit 107. The virtual scale setting unit 105 sets a virtual scale that represents the actual size of the observation target on the subject according to the position of the spot SP and has a scale with an end portion as a base point. The virtual scale switching reception unit 106 receives an instruction to switch and set a plurality of virtual scales. The length measurement image creation unit 107 creates a length measurement image by superimposing the virtual scale set by the virtual scale setting unit 105 on the captured image so that the position of the spot SP and the base point of the scale of the virtual scale overlap.

[0165] The functions of the virtual scale setting unit 105 and the length measurement image creation unit 107 will be described below. As shown in Fig. 74, a captured image 109 in which a subject including a polyp 108 is illuminated is input to the signal processing unit 45. For example, since the polyp 108 has a spherical three-dimensional shape, the captured image 109 includes the polyp 108, a spot SP, and possibly a shadow 110.

[0166] The position identifying unit 92 identifies the position of the spot SP based on the captured image 109 input to the signal processing unit 45. The virtual scale setting unit 105 refers to the scale table 62 and sets a virtual scale that indicates the actual size of the observation target corresponding to the position of the spot SP and has a scale with an end portion as a base point. The end portion is a portion, a starting point, or an end point, etc., that is closer to the outer portion than the central portion in the shape of the virtual scale.

[0167] As shown in FIG. 75, the length measurement image creation unit 107 creates a captured image 109 in which the virtual scale 111 set by the virtual scale setting unit 105 is superimposed on the captured image 109 so that the position of the spot SP and the base point of the scale of the virtual scale 111 overlap. For more accurate measurement, it is preferable to superimpose the virtual scale 111 so as to display it at the position of the spot SP. Therefore, even if it is displayed at a position away from the spot SP, it is preferable to display it as close to the spot SP as possible. The virtual scale 111 is a straight line segment, and has scales at the start and end points of the line segment that are perpendicular to the straight line segment. When the virtual scale 111 is a line segment or the like and has a start and end point, the start and / or end points themselves may be scales, and in this case, for example, there may be no scale in the shape of a line segment perpendicular to the straight line segment. The virtual scale 111 may also have the number "10" near the base end of the scale. This number "10" is the scale label 111a of the virtual scale 111, and is added so that it can be easily recognized that the line segment of the virtual scale 111 is 10 mm in actual size. Hereinafter, the numbers on the virtual scale have the same meaning. The numerical value of the scale label 111a can be changed by settings, and the virtual scale 111 may not display the scale label 111a itself.

[0168] Various types of virtual scales are used depending on the settings, for example, a line segment having a straight line shape or a combination of line segments, a circle having a straight line shape or a combination of circles, or a combination of a line segment and a circle.

[0169] 76, for example, the captured image 113 includes a virtual scale 112 whose shape is a combination of straight line segments. The virtual scale 112 has a shape in which straight line segments are combined into an L shape, with the line segments extending upward and toward the paper surface from a corner of the L shape as a base point, and has scale marks at the end points of each of the base points. Similarly to the virtual scale 111, the virtual scale 112 has a numerical value of "10" as a scale label 112a near the base point of the scale.

[0170] As shown in FIG. 77, for example, the captured image 114 includes a virtual scale 115 whose shape is a combination of a straight line segment and a circle. The virtual scale 115 has a shape that combines a circle and a line segment that is the diameter of the circle, and the intersections of the line segment and the circle are the scales. The scale 116 may be provided at the point where the line segment is halved or at the center of the circle. Similarly to the virtual scale 111 or the virtual scale 112, the virtual scale 115 has the number "10" as the scale label 116a near the base point of the scale. The scale label 116b represents half of the scale label 116a.

[0171] As shown in Figure 78, the virtual scale can take various other shapes, such as virtual scale 117 including scale label 117a (Figure 78(A)) in which a line segment extends from a base point to the left on the paper, virtual scale 118 including scale label 118a (Figure 78(B)) in which a line segment extends from a base point downward on the paper, or virtual scale 119 including scale label 119a (Figure 78(C)) in which a line segment extends from a base point diagonally upward to the right on the paper.

[0172] As shown in Fig. 79, the signal processing unit 45 of the extension processor device 17 includes a position identifying unit 92, a reference scale setting unit 120, a measurement value scale generating unit 121, and a length measurement image generating unit 122. The reference scale setting unit 120 sets a reference scale indicating the actual size of the subject based on the position of the spot SP. The measurement value scale generating unit 121 generates a measurement value scale indicating the measurement value obtained by measuring the measurement portion of the region of interest based on the set reference scale. Note that the reference scale and the measurement value scale are virtual ones displayed on the captured image, and therefore correspond to virtual scales.

[0173] The attention area is an area included in the subject that the user should pay attention to. The attention area is, for example, a polyp, etc., and is an area where measurement is highly likely to be required. The measurement portion is a portion of the attention area where the length, etc. is measured. For example, if the attention area is a reddened area, the measurement portion is the longest part of the reddened area, etc., and if the attention area is circular, the measurement portion is the diameter part of the attention area, etc.

[0174] The length measurement image generating unit 122 creates a length measurement image by superimposing a measurement value scale on the captured image. The measurement value scale is superimposed on the captured image in a state aligned with the measurement portion of the region of interest. The length measurement image is displayed on the extended display 18.

[0175] As shown in FIG. 80, the reference scale setting unit 120 includes a reference scale table 121a. The reference scale table 121a is correspondence information that associates the position of the spot SP with measurement information corresponding to the actual size of the subject. In the length measurement mode, a captured image 114 of a subject including a polyp 123 to be observed is input to the signal processing unit 45. As shown in FIG. 81, in the captured image 124, the polyp 123 has a three-dimensional shape, for example, like overlapping spheres. For example, a spot SP is formed at an end portion on the polyp 123. Based on the captured image 124, the position specifying unit 92 specifies the position of the spot SP. The reference scale setting unit 120 refers to the reference scale table 121a and sets a reference scale 131 indicating the actual size of the subject corresponding to the specified position of the spot SP.

[0176] The reference scale 131 is, for example, a line segment having a number of pixels corresponding to 20 mm in the actual size, as well as a numerical value and a unit indicating the actual size. The reference scale 131 is not usually displayed on the extended display 18, but when the reference scale 131 is displayed on the extended display 18, it is displayed like the captured image 124.

[0177] As shown in Fig. 82, the measurement value scale generating unit 121 includes an attention area extracting unit 125, a measurement portion determining unit 126, a measurement content receiving unit 127, and a measurement value calculating unit 128. As shown in Fig. 83, the attention area extracting unit 125 extracts a hatched area as an attention area 129 as in the captured image 124. Next, as shown in Fig. 84, when a preset reference is a reference for measuring a portion of the attention area in the horizontal direction with the spot SP as the base point, the measurement portion determining unit 126 extracts a horizontal edge position 130 with the spot SP as the base point as in the captured image 124. The portion between the spot SP and the horizontal edge position 130 becomes the measurement portion.

[0178] For example, when the actual size of the reference scale is L0, the number of pixels of the reference scale 131 in the captured image 124 is Aa, the number of pixels of the measurement portion when the reference scale 131 is superimposed on the attention area 129 in the captured image 124 is Ba, and the actual size of the measurement value scale 132 is L1, the measurement value calculation unit 128 generates the measurement value scale 132 to satisfy the following formula (K1). Formula (K1) L1=L0×Ba / Aa

[0179] As shown in FIG. 85, the measurement value calculation unit 128 calculates the actual size of the measurement value scale 132 to be 13 mm, as in the captured image 124d, based on the number of pixels Aa corresponding to the reference scale 131 shown in the captured image 124a and the number of pixels Bb corresponding to the measurement portion between the spot SP and the horizontal edge position 130 shown in the captured image 124b. For example, when Ba / Aa is 0.7 and the actual size of the reference scale 131 is 20 mm, the measurement value calculation unit 128 calculates the actual size of the measurement value scale 132 to be 13 mm, as in the captured image 124d.

[0180] The length measurement image generating section 122 generates a length measurement image 133 by superimposing a measurement value scale 132 on the captured image 124. For example, as shown in Fig. 86, the measurement value scale 132 is superimposed on the captured image 124 by a graphic such as an arrow having the shape of a straight line segment. The length measurement image 133 may include a numerical value of the actual size of the measurement value scale 132. The numerical value of the actual size of the measurement value scale 132 may be superimposed on the captured image 124 apart from the graphic such as the arrow.

[0181] It is possible to select from a plurality of types of the measurement value scale 132. The measurement content receiving unit 127 receives the setting of the content of the measurement value scale and sends the content to the measurement value scale generating unit 121, and the measurement value scale generating unit 122 generates the measurement image 133 using the measurement value scale 132 generated by the measurement value scale generating unit 121 based on the content.

[0182] It is preferable that the attention area extraction unit 125 extracts the attention area using a trained model trained from captured images acquired in the past. As the trained model, various models suitable for image recognition by machine learning can be used. For the purpose of recognizing the attention area on the image, a model using a neural network can be preferably used. When training these models, the models are trained using a captured image having information on the attention area as training data. Examples of the information on the attention area include the presence or absence of the attention area, the position or range of the attention area, etc. It is also possible to train using a captured image having no information on the attention area depending on the model.

[0183] In addition, it is preferable that the measurement portion determination unit 126 also determines the measurement portion using a trained model trained by a captured image acquired in the past. The models and the like used for the trained model are similar to those used for the attention area extraction unit, but when these models are trained, they are trained by a captured image having information of the measurement portion. The information of the measurement portion includes a measurement value and its measurement portion. Depending on the model, it may be trained by a captured image having no information of the measurement portion. The trained model used by the attention area extraction unit 125 and the trained model used by the measurement portion determination unit 126 may be the same. In the case of the purpose of extracting the measurement portion, it is also possible to extract the measurement portion without extracting the attention area from the captured image 124 by using one trained model.

[0184] In the second signal processing unit 60, the scale table 62 for displaying the virtual scale deformed according to the position of the spot SP is updated from the representative point data table 66 storing the irradiation position of the measurement light and the representative point of the virtual scale (see Figs. 39 and 40), but the scale table 62 may be created by other methods. For example, from an image obtained when a square lattice chart is imaged, a distorted lattice area QN that encloses a circular virtual scale centered on the spot SP is obtained as shown in Fig. 87. The distorted lattice area QN is distorted more by the distortion aberration of the imaging optical system 21 as it moves away from the center of the screen. The distorted lattice area QN is transformed into a square lattice area SQ as shown in Fig. 88 by an affine transformation matrix. In the square lattice area SQ, the coordinates of a point indicating the circular virtual scale are calculated. Then, the coordinates of the point of the virtual scale in the square lattice area SQ are transformed into a distorted circular virtual scale distorted by the imaging optical system 21 by the inverse matrix of the affine transformation matrix. The coordinates of this distorted circular virtual scale and the position of the spot SP are stored in a scale table 62 in association with each other.

[0185] In addition, in consideration of the distortion aberration of the imaging optical system 21, the display mode of the virtual scale may be changed between the area where the measurement by the virtual scale is effective and the other area. Specifically, as shown in Fig. 89(a), when the spot SP is outside the range of the effective measurement area (near end Px side), and as shown in Fig. 89(c), when the spot SP is outside the range of the effective measurement area (far end Pz side), the measurement of the tumor tm by the virtual scale is not effective, so that cross-shaped virtual scales MN and MF are displayed. On the other hand, as shown in Fig. 89(b), when the spot SP is in the area where the measurement by the circular virtual scale M is effective, a circular virtual scale M is displayed.

[0186] The line type of the virtual scale may be changed depending on whether the spot SP is within or outside the effective measurement area. In this case, it is preferable to display the movement trajectory MT of the spot SP as shown in FIG. 90 so that the change in the line type of the virtual scale can be seen. As shown in FIG. 90(a), when the spot SP is outside the effective measurement area (near end Px side), and as shown in FIG. 90(c), when the spot SP is outside the effective measurement area (far end Pz side), measurement of the tumor tm by the virtual scale is not effective, so the circular virtual scales MpN and MpF are displayed with dotted lines. On the other hand, as shown in FIG. 90(b), when the spot SP is in an area where measurement by the circular virtual scale Mp is effective, the circular virtual scale Mp is displayed with a solid line. Note that the line type of the virtual scale is changed between dotted and solid depending on whether the spot SP is within or outside the effective measurement area, but may be different colors. For example, when the spot SP is outside the effective measurement area, the line type of the virtual scale is blue, and when the spot SP is within the effective measurement area, the line type of the virtual scale is white.

[0187] Details of still image acquisition in the length measurement mode will be described. When a still image acquisition instruction is not given, the system control unit 41 controls the light source device 13 to emit illumination light and measurement light. As shown in FIG. 91, when a still image acquisition instruction switch 12g is operated to give a still image acquisition instruction, at a first timing including the still image acquisition instruction, the illumination light is turned on, while the measurement light is turned off. At a second timing and a third timing after the first timing, the illumination light is kept on, and the measurement light is turned on again. Note that the second timing and the third timing are the same, but may be different.

[0188] At the first timing, a second captured image is obtained by capturing an image of an object illuminated with the measurement light. At the second timing and the third timing, a first captured image is obtained by capturing an image of an object illuminated with the illumination light and the measurement light. Then, as shown in Figs. 91 and 92, the system control unit 41 stores a still image of the first captured image and a still image of the second captured image as saved images to be saved in the still image saving unit 42. In addition, the signal processing unit 45 of the extended processor device 17 acquires a still image of a third captured image in which a virtual scale Mxm set according to the position of the spot SP is displayed for the first captured image. The still image of the third captured image is sent to the processor device 14 and saved in the still image saving unit 42. In addition, for a certain period of time after the still image is saved, in order to inform the user that the still image has been recorded, the display control unit 46 displays the second captured image and the third captured image on the extended display 18 as shown in Fig. 93. It is preferable that at least two of the first captured image, the second captured image, and the third captured image are saved in the still image saving unit 42 by one still image acquisition instruction. For example, it is preferable to store two captured images, the second captured image and the third captured image. As described above, the third captured image corresponds to an image for storage to be stored in the still image storage unit 42 among the length measurement images on which the virtual scale is superimposed.

[0189] As shown in Fig. 94, the second timing or the third timing may be earlier than the first timing. In this case, in the length measurement mode, it is necessary to store several frames of the first captured image corresponding to the second timing or the third timing in a temporary storage unit (not shown) of the processor device 14. When a still image acquisition instruction is given, the first captured image stored in the temporary storage unit is stored in the still image storage unit 42 as the first captured image at the second timing, and a third captured image obtained by adding a virtual scale to the first captured image stored in the temporary storage unit is stored in the still image storage unit 42.

[0190] 95, the second timing and the third timing may be different timings. In this case, the first captured image obtained at the second timing is stored in the still image storage section 42 in the same manner as described above. Moreover, the first captured image obtained at the third timing is converted into a third captured image by adding a virtual scale before being stored in the still image storage section 42, and then stored in the still image storage section 42.

[0191] As shown in FIG. 96, the signal processing unit 45 of the extended processor device 17 may be provided with a lesion recognition unit 135, a diagnosis information acquisition unit 136, and a learning unit 137 in addition to the first signal processing unit 59 and the second signal processing unit 60. The lesion recognition unit 135 performs image processing on the first captured image (an image based on the illumination light and the measurement light) and performs a recognition process. The recognition process performed by the lesion recognition unit 135 is a detection process for detecting a region of interest such as a lesion from the first captured image. It is preferable to use a machine-learned learning model for the recognition process. That is, in response to an input of the first captured image to the learning model, a detection result of the region of interest is output from the learning model. It is preferable that the learning model is a machine-learned learning model such as a Convolutional Neural Network (CNN). The recognition process performed by the lesion recognition unit 135 may be a discrimination process for discriminating the degree of progression of the lesion from the lesion recognized from the first captured image. Furthermore, the lesion recognition unit 135 may perform recognition processing by performing image processing on the second captured image (an image based only on illumination light).

[0192] The diagnostic information acquisition unit 136 acquires diagnostic information related to the first captured image or the second captured image from the diagnostic information management device 138. As the diagnostic information, a medical record of the patient who is the subject of the examination is acquired. The medical record is information that records the progress of medical treatment or examination for a patient, and includes, for example, the patient's name, sex, and age, disease name, main symptoms, prescription or treatment contents, medical history, etc. The information on the lesion recognized and processed by the lesion recognition unit 135 and the diagnostic information related to the first captured image or the second captured image acquired by the diagnostic information acquisition unit 136 are stored in the still image storage unit 42 in association with the first captured image or the second captured image as auxiliary data of the data set DS.

[0193] The learning unit 137 performs machine learning using the first captured image or the second captured image stored in the still image storage unit 42 and the associated data (data set) associated with the first and second captured images. Specifically, the learning unit 137 performs machine learning on the learning model of the lesion recognition unit 135. The second captured image is preferable as a candidate for teacher data for machine learning. The second captured image is an image obtained by a still image acquisition instruction during measurement of a tumor tm or the like, and is therefore an image that is likely to include a region of interest to be observed. In addition, since the second captured image is a normal endoscopic image to which measurement light is not irradiated, it is highly useful as teacher data for machine learning. In addition, since information on the lesion and diagnostic information are also attached as the associated data, the user does not need to input them when performing machine learning. The more the second captured image is accumulated as a candidate for teacher data and machine learning is performed, the more the accuracy of the recognition process in the lesion recognition unit 135 is improved. When the first captured image is used as a candidate for teacher data for machine learning, the first captured image may be used as it is, but it is more preferable to use a portion other than the region irradiated with the measurement light as a candidate for teacher data.

[0194] A calibration method for creating a representative point data table 66 using the calibration device 200 shown in Fig. 97 will be described below. The calibration device 200 includes a calibration display 201, a movement mechanism 202, a calibration display control unit 204, a calibration image acquisition unit 206, and a calibration unit 208. The calibration display control unit 204, the calibration image acquisition unit 206, and the calibration unit 208 are provided in a calibration image processing device 210. The calibration image processing device 210 is electrically connected to the processor device 14, the calibration display 201, and the movement mechanism 202.

[0195] The movement mechanism 202 has a holder (not shown) that holds the tip 12d of the endoscope 12 facing the calibration display 201, and changes the distance Z between the tip 12d of the endoscope 10 and the calibration display 201 by moving the holder at a specific interval. Every time the movement mechanism 202 changes the distance Z, the calibration display control unit 204 displays, on the calibration display 201, an image of a virtual scale in a first display mode that is not affected by the imaging optical system 21 with respect to the irradiation position of the measurement light. The virtual scale image in the first display mode does not take into account the influence of distortion and the like caused by the imaging optical system 21, and is therefore not displayed with a size, shape, or the like that corresponds to the scale display position when displayed on the extended display 18.

[0196] The calibration image acquisition unit 206 acquires a calibration image obtained by imaging the virtual scale in the first display mode displayed on the calibration display 201 by the endoscope 12. The calibration image is acquired by the endoscope 12 capturing an image every time the distance Z is changed, that is, every time the virtual scale in the first display mode is displayed. For example, when the virtual scale in the first display mode is displayed n times, n calibration images are acquired.

[0197] The calibration image includes an image of the virtual scale in a second display mode, which is affected by the imaging optical system 21 with respect to the irradiation position of the measurement light. The image of the virtual scale in the second display mode takes into consideration the influence of distortion and the like caused by the imaging optical system 21, and is therefore displayed with a size, shape, or the like corresponding to the scale display position.

[0198] The calibration unit 208 performs calibration of the display of the virtual scale on the extended display 18, based on the calibration image acquired by the calibration image acquisition unit 206. Specifically, the calibration unit 208 performs a representative point extraction process to extract a representative point from the image of the virtual scale in the second display mode included in the calibration image, and a table creation process to create a representative point data table by associating representative point data on the representative point with the irradiation position at the timing when the calibration image was acquired. The created representative point data table is sent to the extended processor device 17 and stored in the representative point data table 66.

[0199] 98, inspection system 300 is used to inspect the accuracy of a scale, such as whether a virtual scale has a predetermined shape. Inspection system 300 includes a test chart 302, a display 15, and a movement mechanism unit 304. Display 15 is shared with endoscope system 10, but a separate display for inspection of accuracy may be provided.

[0200] As shown in FIG. 99, the test chart 302 has a chart body 305, and the chart body 305 is provided with an inspection area section 306 having an inspection area of ​​a specific shape, and an inspection reference position 308 which is a reference for aligning the irradiation position of the measurement light during accuracy inspection. The inspection area section 306 has three circular inspection areas 306a, 306b, and 306c as the inspection area of ​​a specific shape. These three inspection areas 306a, 306b, and 306c are provided concentrically with the inspection reference position 308 as the center. The inspection areas 306a, 306b, and 306c are obtained by illuminating the chart body 205 with measurement light (for example, spot SP) from the endoscope 12 used for confirmation inspection of a 5 mm virtual scale (indicating a diameter of "5 mm"), a 10 mm virtual scale (indicating a diameter of "10 mm"), and a 20 mm virtual scale (indicating a diameter of "20 mm"), respectively, and capturing an image.

[0201] As shown in FIG. 100, the inspection image is displayed on the display 15. In addition to an inspection area portion 306 and an inspection reference position 308, the inspection image displays a virtual scale M corresponding to the irradiation position of the measurement light (position of the spot SP). During accuracy inspection, the moving mechanism portion 304 moves the test chart 302 so that the irradiation position of the measurement light (position of the spot SP) matches the inspection reference position. When the irradiation position of the measurement light matches the inspection reference position, the user determines whether the virtual scale M is displayed properly.

[0202] For example, in the inspection image, when the irradiation position of the measurement light is at the inspection reference position 308, if the virtual scale M is within the inspection area 306, the user determines that the virtual scale M is displayed properly. In contrast, as shown in Fig. 101, when even a part of the virtual scale M does not fall within the inspection area 306, such as when the virtual scale M protrudes even partially from the inspection area 306, the user determines that the 5 mm virtual scale M is not displayed properly.

[0203] The scale table 62 may be created as follows. The relationship between the spot position and the size of the virtual scale can be obtained by imaging a chart on which a pattern of the actual size is regularly formed. For example, a spot-shaped measurement light is emitted toward the chart, and an image of a graph paper-like chart with the same lines as the actual size (5 mm) or finer lines (for example, 1 mm) is captured while changing the observation distance to change the spot position, and the relationship between the spot position (pixel coordinates on the imaging surface of the image sensor 32) and the number of pixels corresponding to the actual size (how many pixels represent 5 mm, which is the actual size) is obtained.

[0204] As shown in FIG. 102, (x1, y1) is the pixel position in the X and Y directions of the spot SP4 on the imaging surface of the imaging element 32 (the upper left is the origin of the coordinate system). The number of pixels in the X direction corresponding to an actual size of 5 mm at the position (x1, y1) of the spot SP4 is Lx1, and the number of pixels in the Y direction is Ly1. Such measurements are repeated while changing the observation distance. FIG. 103 shows a state in which the same 5 mm ruled chart as in FIG. 102 is imaged, but the shooting distance is closer to the far end than in the state of FIG. 102, and the interval between the rules is narrower. In the state of FIG. 103, the number of pixels in the X direction corresponding to an actual size of 5 mm at the position (x2, y2) of the spot SP5 on the imaging surface of the imaging element 32 is Lx2, and the number of pixels in the Y direction is Ly2. Then, while changing the observation distance, measurements such as those in FIGS. 102 and 103 are repeated, and the results are plotted. Note that FIGS. 102 and 103 are displayed without taking into consideration the distortion aberration of the imaging optical system 21.

[0205] Fig. 104 shows the relationship between the X coordinate of the spot position and Lx (the number of pixels in the X direction of the first virtual scale), and Fig. 105 shows the relationship between the Y coordinate of the spot position and Lx. From the relationship in Fig. 104, Lx can be expressed as Lx = g1(x) as a function of the X direction position, and from the relationship in Fig. 105, Ly can be expressed as Ly = g2(y) as a function of the Y direction position. g1 and g2 can be found from the above-mentioned plot results, for example, by the least squares method.

[0206] Since the X and Y coordinates of the spot correspond one-to-one and essentially the same result (the same number of pixels for the same spot position) is obtained using either function g1 or g2, either function may be used when calculating the size of the first virtual scale, and the function g1 or g2 that has the higher sensitivity to changes in the number of pixels relative to changes in position may be selected. Also, if the values ​​of g1 and g2 differ greatly, it may be determined that "the spot position could not be recognized."

[0207] Fig. 106 shows the relationship between the X coordinate of the spot position and Ly (number of pixels in the Y direction), and Fig. 107 shows the relationship between the Y coordinate of the spot position and Ly. From the relationship in Fig. 106, Ly is expressed as Ly=h1(x) as the coordinate of the X direction position, and from the relationship in Fig. 108, Ly is expressed as Ly=h2(y) as the coordinate of the Y direction position. For Ly, either function h1 or h2 may be used, as in the case of Lx.

[0208] The functions g1, g2, h1, and h2 obtained as described above are stored in the scale table 62 in the form of a lookup table. Note that the functions g1 and g2 may be stored in the scale table 62 in the form of a function.

[0209] As for the measurement light, a stripe pattern light ZPL may be used that is formed as a stripe pattern light on the subject as shown in FIG. 108 when irradiated to the subject (see, for example, JP 2016-198304 A). The stripe pattern light ZPL is obtained by irradiating a liquid crystal shutter (not shown) with variable transmittance with a specific laser light, and is formed from two different vertical stripe patterns in which a region (transmitting region) that transmits the specific laser light and a region (non-transmitting region) that does not transmit the specific laser light are periodically repeated in the horizontal direction by the liquid crystal shutter. When using the stripe pattern light as the measurement light, the period of the stripe pattern light changes depending on the distance from the subject, so the period or phase of the stripe pattern light is shifted by the liquid crystal shutter and irradiated multiple times, and the three-dimensional shape of the subject is measured based on multiple images obtained by shifting the period or phase.

[0210] For example, a stripe pattern light of phase X, a stripe pattern light of phase Y, and a stripe pattern light of phase Z are alternately irradiated onto the subject. The stripe pattern light of phases X, Y, and Z has a vertical stripe pattern phase shifted by 120° (2π / 3) each. In this case, the three-dimensional shape of the subject is measured using three types of images obtained based on each stripe pattern light. For example, as shown in FIG. 109, it is preferable to irradiate the subject with the stripe pattern light of phase X, the stripe pattern light of phase Y, and the stripe pattern light of phase Z by switching between them every frame (or every few frames). It is preferable to constantly irradiate the subject with illumination light.

[0211] As for the measurement light, a measurement light LPL having a grid pattern may be used that is formed as a grid pattern as shown in FIG. 110 when the measurement light LPL is irradiated onto the subject (see, for example, Japanese Patent Application Laid-Open No. 2017-217215). In this case, the three-dimensional shape of the subject is measured based on the deformation state of the grid pattern when the measurement light LPL having the grid pattern is irradiated onto the subject, so that it is required to accurately detect the grid pattern. Therefore, the measurement light LPL having the grid pattern is not a perfect grid, but is slightly deformed from the grid pattern, such as by being wavy, so as to improve the detection accuracy of the grid pattern. In addition, the grid pattern is provided with a code S indicating that the end points of the horizontal line segments on the left and right are continuous. When detecting the grid pattern, not only the pattern but also the code S is detected to improve the detection accuracy of the pattern. In addition, the grid pattern may be a pattern in which vertical and horizontal lines are regularly arranged, or a pattern in which multiple spots are vertically and horizontally arranged in a grid pattern.

[0212] When the measurement light LPL of the grid pattern is used as the measurement light, the illumination light and the measurement light LPL of the grid pattern may be constantly irradiated to the subject during the length measurement mode, or, as shown in FIG. 111, the illumination light may be constantly irradiated to the subject, while the measurement light LPL of the grid pattern may be intermittently irradiated to the subject by repeatedly turning on and off (or dimming) the measurement light LPL of the grid pattern every frame (or every few frames). In this case, in the frame in which the measurement light LPL of the grid pattern is turned on, the measurement of the three-dimensional shape is performed based on the measurement light LPL of the grid pattern. Then, it is preferable to superimpose the measurement result of the three-dimensional shape on the image obtained in the frame in which only the illumination light is irradiated.

[0213] As for the measurement light, three-dimensional planar light TPL represented by reticle lines on the subject image may be used as shown in Fig. 112 (see, for example, JP2017-508529A). In this case, the tip 12d is moved so that the three-dimensional planar light TPL is aligned with the measurement subject. Then, when the three-dimensional planar light TPL intersects with the measurement subject, the distance of the intersection curve CC between the three-dimensional planar light TPL and the subject is calculated by processing based on manual operation of a user interface or the like or by automatic processing.

[0214] When the three-dimensional planar light TPL is used as the measurement light, the illumination light and the three-dimensional planar light TPL may be constantly irradiated onto the subject during the length measurement mode, or, as shown in FIG. 113, while the illumination light is constantly irradiated onto the subject, the three-dimensional planar light TPL may be intermittently irradiated onto the subject by repeatedly turning it on and off (or dimming) every frame (or every few frames).

[0215] In the above embodiment, the hardware structure of the processing units that execute various processes, such as the receiving unit 38, the signal processing unit 39, the display control unit 40, the system control unit 41, the still image storage unit 42, the data transmission / reception unit 43, the data transmission / reception unit 44, the signal processing unit 45, and the display control unit 46 (including various control units or processing units provided in these control units, for example, the length measurement mode control unit 50, the first signal processing unit 59, etc.), is various processors as shown below. The various processors include a CPU (Central Processing Unit), which is a general-purpose processor that executes software (programs) and functions as various processing units, a programmable logic device (PLD), which is a processor whose circuit configuration can be changed after manufacture, such as an FPGA (Field Programmable Gate Array), and a dedicated electric circuit, which is a processor having a circuit configuration designed exclusively for executing various processes.

[0216] One processing unit may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same type or different types (for example, multiple FPGAs, or a combination of a CPU and an FPGA). Also, multiple processing units may be configured with one processor. As an example of configuring multiple processing units with one processor, first, as represented by a computer such as a client or server, one processor is configured with a combination of one or more CPUs and software, and this processor functions as multiple processing units. Second, as represented by a system on chip (SoC), there is a form in which a processor is used that realizes the functions of the entire system including multiple processing units with one IC (Integrated Circuit) chip. In this way, the various processing units are configured using one or more of the above various processors as a hardware structure.

[0217] Furthermore, the hardware structure of these various processors is, more specifically, an electric circuit (circuitry) in the form of a combination of circuit elements such as semiconductor elements, and the hardware structure of the memory unit is a storage device such as a hard disk drive (HDD) or a solid state drive (SSD). [Explanation of symbols]

[0218] 10 Endoscope System 12 Endoscopy 12a Insertion part 12b Operation section 12c Curved section 12d Tip 12f Observation mode switch 12g Still image capture switch 12h Zoom operation section 13 Light source device 14 Processor unit 15 Display 16 User Interface 17 Extended Processor Unit 18 Extended Display 18a Additional information display area 18b Observation image display area 19 Balloon 19a Tip 19b Proximal end 19c bulge 20a, 20b Ring 21 Imaging optical system 21a Objective lens 21b Zoom lens 21c Tip surface 22 Illumination optical system 22a Lighting lens 22b Tip surface 23 Measurement light output part 23a light source 23b DOE 23c Prism 24 Aperture 25 Air and water supply nozzle 25a Injection cylinder part 25b Nozzle 26 Intestinal Tract 27 Tip cap 27a, 27b, 27c, 27d through holes 28 Tip surface 28a, 28b plane 30 Light source section 31 Light source processor 32 Image sensor 33 Imaging control section 34 CDS / AGC circuit 35 A / D converter 36 Communication I / F 37 Communication I / F 38 Receiving section 39 Signal Processing Section 40 Display control unit 41 System control section 42 Still image storage section 43 Data transmission and reception unit 44 Data transmission and reception unit 45 Signal Processing Section 46 Display control section 47 Storage section for measurement light emission section 48 Transparent lid 49 Prism 50 Measurement mode control section 53 Brightness information calculation unit 54 Illumination light intensity level setting section 55 First light emission control table 56 Second light emission control table 57 Rising Line 58 Diagonal Line 59 First signal processing section 60 Second signal processing section 61 Irradiation position detection unit 62 Scale Table 64 Table Update Section 66 Representative point data table 67 Crossing Lines 68 scales 69 Location identification part 70 Measurement information processing section 71 Measurement Information 72 Location identification part 73 Image Processing Department 74 Noise component removal section 75 Color Information Conversion Unit 76 Binarization processing section 77 Mask image generation unit 78 Removal section 79 Color information of measurement light 80 Color information of noise components 81 Noise component area 82 Image selection section 83 Scale Table 84 First signal processing section 85 Second signal processing section 86 Mask processing section 87 Binarization processing section 88 Noise component removal section 89 Irradiation position detection unit 90 Irradiation area recognition unit 91 Learning Model 92 Location identification part 93 Image Processing Department 94 Distance Calculation Unit 95 Image Selection Section 97 Offset setting section 98 Offset distance calculation section 99 Offset virtual scale generator 100 Polyps 100a top 100b flat area 101 Extension surface 101X Solid line 102, 103 Parallel surfaces 102X Dotted Line 104 Image Processing Department 105 Virtual scale setting section 106 Virtual scale switching reception unit 107 Measurement image creation section 108 Polyps 109 Captured Images 110 Shadow 111, 112 Virtual scale 111a Scale label 113, 114 Captured images 115 Virtual Scale 116 scales 116a Scale label 116b Scale label 118 Virtual Scale 118a Scale label 119 Virtual Scale 119a Scale label 120 Reference scale setting section 121 Measurement value scale generator 121a Reference scale table 122 Measurement image generation unit 123 Polyps 124 Captured images 125 Attention area extraction part 126 Measurement part determination unit 127 Measurement Content Reception Department 128 Measurement value calculation section 129 Areas of Interest 130 Horizontal edge position 131 Reference Scale 132 Measurement scale 133 Measurement Image 135 Lesion Recognition Department 136 Diagnostic information acquisition unit 137 Learning Department 138 Diagnostic information management device 140 Length measurement compatible endoscope suitability determination unit 141 Measurement light ON / OFF switching section 142 Measurement image display setting ON / OFF switch 143 Measurement function operation status display ON / OFF switch 144 Virtual scale display switching control section 146 Scale display icon 147 Virtual Scale 147a, 147b, 147c Virtual Scale 148 Scale hidden icon 149 Previous image display setting storage section 200 Calibration Device 201 Calibration Display 202 Moving mechanism 204 Calibration display control unit 206 Calibration Image Acquisition Unit 208 Calibration section 210 Calib Image Processing Device 300 Inspection System 302 Test Chart 304 Moving mechanism section 305 Chart Body 306 Inspection area 306a, 306b, 306c Inspection area 308 Inspection reference position Aa, Ba Pixel count Ax optical axis BLC Balloon Control Device BF Blue color filter CL1 First characteristic line CL2 Second characteristic line CC intersection curve CR1 White center region D1 1st direction D2 2nd direction D3 Third direction D5 distance D6 Offset distance DS1 Diffraction Spot DT Interval EP measurement point G1, G2, G3 Gap GF Green color filter HT1, HT2 Height LG Light Guide Ls, Lt line Lm measurement light Lx1, Lx2 Number of pixels in the X direction Ly1, Ly2 Y-direction pixel count LPL Lattice pattern measurement light M Circular virtual scale M1, M2, M3 Cross-shaped virtual scale M11, M12, M13, M14, M15 Virtual Scale M21, M22, M23 concentric circles M41, M42, M43 Virtual Scale M4A, M4B, M4C, M5A, M5B, M5C Concentric markers M5A, M5B, M5C Virtual Scale M6A, M6B, M6C Distorted concentric virtual scale MN, MF Virtual scale of crosshairs MpN, Mp, MpF Circular virtual scale MT movement trajectory Mx scale Mxm Virtual Scale N1 First noise component N2 Second noise component P polyp P1, P2, P3 position Px near end Py Center Pz far end RP, PRx Red image PRy Binarized red image GP, PGx green image PGy binary green image BP, PBx blue image PBy Binarized Blue Image PD1 First difference image Qx, Qy, Qz arrows QN distorted grid region RP, RP * Representative point RF Red Color Filter SCC specific intersection curve SP Spot SP1, SP2, SP3, SP4, SP5 Spot SPk1 1st spot SPk2 2nd Spot SQ square lattice area SR1 Surrounding Area tm, tm1, tm2, tm3, tm4, tm5 tumors TPL 3D plane light W11, W12, W13, W14, W15 width W21, W22, W23 width Wx Irradiation position movable range WMB Measurement light wavelength range ZPL striped pattern light

Claims

1. An endoscope and a processor device to which the endoscope is connected; a processor; The processor, When an endoscope is connected to the processor device, it is determined whether or not the endoscope is a length measurement compatible endoscope; When the endoscope is the length measurement-compatible endoscope, enabling switching to a length measurement mode in which measurement light is irradiated from the endoscope and a virtual scale based on the measurement light is displayed on a display; An endoscope system that performs control such that, when an observation mode is set to a special observation mode, a special light used to highlight a specific area of ​​an observation object is turned on and the measurement light is turned off.

2. The processor, 2. The endoscope system according to claim 1, wherein when a switching operation to the length measurement mode is performed in a state in which the special observation mode is set, switching to the length measurement mode is prohibited.

3. The processor, 2. The endoscope system according to claim 1, wherein when a switching operation to the length measurement mode is performed while the special observation mode is set, the special observation mode is released and the mode is switched to the length measurement mode.

4. The processor, 2. The endoscope system according to claim 1, wherein in the special observation mode, a pseudo-color image in which colors of an object image are assigned to different color channels is displayed on the display.

5. The endoscope system according to claim 1 , wherein, when the length measurement mode is set, the endoscope irradiates an illumination light different from the special light and the measurement light.

6. The processor, The endoscope system according to claim 1 , wherein a scale display icon is displayed on the display in response to an operation for switching to the length measurement mode.

7. The processor, The endoscope system according to claim 1 , wherein, when a mode switching condition is not satisfied during switching to the length measurement mode, a length measurement function unavailable display indicating that the virtual scale is not being displayed is turned ON.

8. The endoscope system according to claim 7 , wherein the mode switching conditions are set conditions relating to the endoscope, the processor device, and the processor, and are conditions used for executing a length measurement mode.

9. 2. The endoscope system according to claim 1, wherein in the length measurement mode, the display mode of the virtual scale is changed by selecting from a plurality of scale patterns.

10. The processor, The endoscopic system of claim 1, wherein when switching to the length measurement mode is enabled, a switching operation to the length measurement mode performs at least any of the following: switching the measurement light ON or OFF, switching a length measurement image display setting related to the length measurement image displaying the virtual scale ON or OFF, switching a length measurement function operation status display indicating that the virtual scale is being displayed on the display ON or OFF, and switching the display of the virtual scale ON, OFF, or changing the display mode.

11. The processor, The endoscope system according to claim 10 , wherein the operation of switching to the length measurement mode switches the measurement light ON, the length measurement image display setting ON, the length measurement function operation status display ON, and the display of the virtual scale ON.

12. The processor, The endoscope system according to claim 11, wherein when a mode switching condition is not satisfied during a switching operation to the length measurement mode, the measurement light, the length measurement image display setting, the length measurement function operation status display, and the virtual scale display are prohibited from being switched on.

13. The endoscope system according to claim 12, wherein, instead of prohibiting the length measurement function operation status display from being switched ON, a length measurement function operation status disabled display, which indicates that the virtual scale is not being displayed, is switched ON.

14. The processor, The endoscope system according to claim 11 , wherein, when the length measurement image display setting is turned ON, the image display setting before switching to the length measurement mode is stored.

15. The endoscope system according to claim 10, wherein the display mode of the virtual scale is changed by selecting from a plurality of scale patterns.

16. The processor, The endoscope system according to claim 10, wherein a switching operation from the length measurement mode to another mode switches the measurement light to OFF, the length measurement image display setting to OFF, the length measurement function operation status display to OFF, and the display of the virtual scale to OFF.

17. The processor, 17. The endoscope system according to claim 10, wherein when the length measurement image display setting is turned off, the image display setting is switched to an image display setting saved before switching to the length measurement mode.

18. An operation method for an endoscope system including an endoscope, a processor device to which the endoscope is connected, and a processor, comprising: The processor, When an endoscope is connected to the processor device, it is determined whether or not the endoscope is a length measurement compatible endoscope; When the endoscope is the length measurement-compatible endoscope, enabling switching to a length measurement mode in which measurement light is irradiated from the endoscope and a virtual scale based on the measurement light is displayed on a display; A method for operating an endoscope system, which, when an observation mode is set to a special observation mode, turns on a special light used to highlight a specific area of ​​an observation object and turns off the measurement light,

Images