Endoscopic system and its operating method

Abstract

To provide an endoscope system that can determine whether a length measurement mode can be executed depending on connection of an endoscope and a method of operating the endoscope system.SOLUTION: A processor determines whether an endoscope 12 is a length measurement-compatible endoscope in a case where the endoscope 12 is connected to a processor device 14, and enables switching to a length measurement mode where the endoscope 12 emits measurement light and causes a display to display a virtual scale based on the measurement light in a case where the endoscope 12 is the length measurement-compatible endoscope. In a case where an observation mode is set as a special observation mode, special light is turned on, which is used to enhance a specific region of an object to be observed, while the measurement light is turned off.SELECTED DRAWING: Figure 14

Description

[Technical Field]

[0001] The present invention relates to an endoscope system that displays a virtual scale for measuring the size of an object, and a method for operating the same. [Background technology]

[0002] In endoscopic systems comprising a light source device, an endoscope, and a processor device, the distance to the subject or the size of the subject is acquired. For example, in Patent Document 1, illumination light and measurement light are irradiated onto the subject, and the irradiation of the measurement light causes a measurement light irradiation area, such as a spot light, to appear on the subject. Then, a virtual scale for measuring the size of the subject is displayed on the image corresponding to the position of the spot light. [Prior art documents] [Patent Documents]

[0003] [Patent Document 1] International Publication No. 2018 / 051680 [Overview of the Initiative] [Problems that the invention aims to solve]

[0004] As described in Patent Document 1, when displaying a virtual scale using measurement light, a length-measuring endoscope capable of emitting measurement light is required. In an endoscope system, it is necessary to determine whether a length-measuring endoscope is connected in order to execute the length-measuring mode that displays the virtual scale.

[0005] The present invention aims to provide an endoscope system and a method for operating the same that can determine whether or not a length measurement mode can be performed by connecting an endoscope. [Means for solving the problem]

[0006] The endoscope system of the present invention comprises an endoscope, a processor device to which the endoscope is connected, and a processor. When the endoscope is connected to the processor device, the processor determines whether the endoscope is a length-measuring endoscope. If the endoscope is a length-measuring endoscope, it activates a switch to a length-measuring mode in which measurement light is emitted from the endoscope and a virtual scale based on the measurement light is displayed on the display. If the observation mode is set to a special observation mode, it controls the system to turn on special light used to highlight a specific area of ​​the observation target and turn off the measurement light.

[0007] It is preferable that the processor prohibits switching to the measurement mode when a switching operation to the measurement mode is performed while the special observation mode is set. It is also preferable that the processor cancels the special observation mode and switches to the measurement mode when a switching operation to the measurement mode is performed while the special observation mode is set.

[0008] In special observation mode, the processor preferably displays a pseudo-color image on the display, in which the colors of the subject image are assigned to different color channels. When set to length measurement mode, it is preferable that illumination light and measurement light different from special light are emitted from the endoscope. In accordance with the operation to switch to length measurement mode, the processor preferably displays a scale display icon on the display.

[0009] When switching to the measurement mode, the processor preferably turns on an indicator showing that the measurement function is not operational, if the mode switching conditions are not met. The mode switching conditions are preferably the settings conditions for the endoscope, processor device, and processor used to execute the measurement mode. In the measurement mode, it is preferable that the display pattern of the virtual scale is changed by selecting from a plurality of scale patterns.

[0010] When switching to measurement mode is enabled, the processor preferably performs at least one of the following actions upon switching to measurement mode: switching the measurement light ON or OFF, switching the measurement image display setting ON or OFF for the measurement image that displays the virtual scale, switching the measurement function operation status display ON or OFF to indicate that the virtual scale is being displayed on the display, and switching the display of the virtual scale ON, OFF, or changing the display mode.

[0011] It is preferable that the processor switches the measurement light ON, the measurement image display setting ON, the measurement function operation status display ON, and the virtual scale display ON when switching to measurement mode. It is preferable that the processor prohibits switching the measurement light ON, the measurement image display setting ON, the measurement function operation status display ON, and the virtual scale display ON if the conditions for switching to measurement mode are not met during the operation to switch to measurement mode. Instead of prohibiting switching the measurement function operation status display ON, it is preferable to turn ON a measurement function operation status unavailable display that indicates the virtual scale is not being displayed.

[0012] When the processor turns on the measurement image display setting, it is preferable to save the image display setting from before switching to measurement mode. When the display mode of the virtual scale is changed, it is preferable to select from among multiple scale patterns. When switching from measurement mode to another mode, it is preferable for the processor to turn off the measurement light, the measurement image display setting, the measurement function operation status display, and the virtual scale display. When the processor turns off the measurement image display setting, it is preferable to switch to the image display setting saved before switching to measurement mode.

[0013] In an operating method of an endoscope system including an endoscope, a processor device to which the endoscope is connected, and a processor, when the endoscope is connected to the processor device, the processor determines whether the endoscope is a length measurement-compatible endoscope. When the endoscope is a length measurement-compatible endoscope, switching to a length measurement mode in which measurement light is irradiated from the endoscope and a virtual scale based on the measurement light is displayed on a display is enabled. When the observation mode is set to a special observation mode, special light used to emphasize a specific area among the observation targets is turned on and the measurement light is turned off.

Effect of the Invention

[0014] According to the present invention, it is possible to determine whether the length measurement mode can be executed by connecting the endoscope.

Brief Description of the Drawings

[0015] [Figure 1] It is a schematic diagram of an endoscope system. [Figure 2] It is a perspective view of a balloon. [Figure 3] It is a front view of a balloon. [Figure 4] It is an explanatory diagram showing a balloon in a contracted state in the intestinal tract. [Figure 5] It is an explanatory diagram showing a balloon in an inflated state in the intestinal tract. [Figure 6] It is a front view of the tip of an endoscope. [Figure 7] It is a perspective view of the tip of an endoscope. [Figure 8] It is a block diagram showing the functions of an endoscope system. [Figure 9] It is an image diagram showing the state (A) where the digital zoom function is OFF and the state (B) where it is ON. [Figure 10] It is a schematic diagram showing a measurement light emitting part. [Figure 11] It is a cross-sectional view of the tip of an endoscope having a measurement light emitting part. [Figure 12] It is a plan view showing a transparent cover. [Figure 13]This is a schematic diagram showing the direction of propagation of the measurement light. [Figure 14] This is a block diagram showing the functions of the system control unit. [Figure 15] This image shows icons indicating whether the scale is displayed or hidden. [Figure 16] This is a block diagram showing the functions of the system control unit. [Figure 17] This is an explanatory diagram showing the first control. [Figure 18] This image shows the message displayed when the first control is performed. [Figure 19] This is an explanatory diagram showing how to deactivate the special observation mode and switch to the length measurement mode. [Figure 20] This is an explanatory diagram showing the second control. [Figure 21] This image shows the message displayed when the second control is performed. [Figure 22] This is an explanatory diagram showing the third control. [Figure 23] This image shows the message displayed when the third control is performed. [Figure 24] This is a block diagram showing the functions of the system control unit. [Figure 25] This is an explanatory diagram showing the first light emission control table. [Figure 26] This is an explanatory diagram showing coordinate areas 1 to 5. [Figure 27] This is an explanatory diagram showing the second light emission control table. [Figure 28] This is an explanatory diagram showing the light emission control in length measurement mode. [Figure 29] This is an explanatory diagram showing the first pattern of light emission and imaging control in length measurement mode. [Figure 30] This is an explanatory diagram showing the second pattern of light emission and imaging control in length measurement mode. [Figure 31] This is a block diagram showing the functions of the signal processing unit. [Figure 32] This image shows the virtual scale displayed when the near end of the Px range is reached. [Figure 33] This image shows the virtual scale displayed when using Py near the center. [Figure 34] This image shows the virtual scale displayed when the far end Pz is at its limit. [Figure 35] This is an explanatory diagram showing the virtual scale of various shapes. [Figure 36] This is an explanatory diagram showing virtual scales of different sizes. [Figure 37] This is an explanatory diagram showing different virtual scales represented by different colors. [Figure 38] This is an explanatory diagram showing a virtual scale in the shape of distorted concentric circles. [Figure 39] This is an explanatory diagram showing representative point data. [Figure 40] This is an explanatory diagram showing the processing of the table update section. [Figure 41] This image shows a virtual scale displayed when a planar measurement light is irradiated. [Figure 42] This is an explanatory diagram showing planar light containing two first feature lines. [Figure 43] This is an explanatory diagram showing the functions of the signal processing unit. [Figure 44] This image shows the image displayed when a planar light containing two first feature lines is shone onto the surface. [Figure 45] This is an explanatory diagram showing the direction of propagation of planar light when planar light containing two first characteristic lines is irradiated. [Figure 46] This is an explanatory diagram showing diffraction spots. [Figure 47] This is an explanatory diagram illustrating the method for calculating 2D and 3D information of an object when using diffraction spots. [Figure 48] This is a block diagram showing the functions of the signal processing unit. [Figure 49] This is an explanatory diagram showing the process for obtaining the first image after noise reduction. [Figure 50] This is an explanatory diagram showing a binarized first-color information image. [Figure 51]This is an explanatory diagram showing a binarized second-color information image. [Figure 52] This is an explanatory diagram showing a binarized second-color information image. [Figure 53] This is an explanatory diagram showing a binarized first-color information image. [Figure 54] This is a block diagram showing the functions of the signal processing unit. [Figure 55] This is an explanatory diagram showing the process for obtaining the first difference image or the second difference image. [Figure 56] This is an explanatory diagram illustrating the first differential processing. [Figure 57] This is an explanatory diagram illustrating the second differential processing. [Figure 58] This is a block diagram showing the functions of the signal processing unit. [Figure 59] This is a schematic diagram showing a spot including the white central region and surrounding regions. [Figure 60] This graph shows the distribution of pixel values ​​for various images in an captured image. [Figure 61] This graph shows the relationship between the transmission distribution of each color filter and the wavelength range of the measured light. [Figure 62] This is a block diagram showing the functions of the irradiation area recognition unit. [Figure 63] This is an explanatory diagram showing an example of a spot pattern that has deformed from a circular shape. [Figure 64] This is a block diagram showing the functions of the signal processing unit. [Figure 65] This is a schematic diagram showing a convex polyp. [Figure 66] This is an explanatory diagram showing the height of the spot. [Figure 67] This is an explanatory diagram regarding the calculation of the offset distance D6. [Figure 68] These are schematic diagrams of virtual scales, each with a different line width. [Figure 69] This is a schematic diagram showing a concentric virtual scale. [Figure 70] This is a schematic diagram of a virtual scale with gradients applied to the lines. [Figure 71]The gaps between the dashed lines represent schematic diagrams of different virtual scales. [Figure 72] These are schematic diagrams of virtual scales, each with a different number of lines. [Figure 73] This is a block diagram showing the functions of the signal processing unit. [Figure 74] This image shows the spot illuminated around the periphery of the polyp. [Figure 75] This is an explanatory diagram showing a virtual scale where the base end is aligned with the spot position. [Figure 76] This is an explanatory diagram showing two virtual scales with the base aligned to the spot position. [Figure 77] This is an explanatory diagram showing a virtual scale of a circle and line segments with the base aligned to the spot position. [Figure 78] This is an explanatory diagram showing a virtual scale where the base end is aligned with the spot position. [Figure 79] This is a block diagram showing the functions of the signal processing unit. [Figure 80] This is a block diagram showing the functions of the reference scale setting unit. [Figure 81] This image shows a virtual scale superimposed on a polyp. [Figure 82] This is a block diagram showing the functions of the measurement value scale generation unit. [Figure 83] This is an image diagram showing the region of interest. [Figure 84] This is an explanatory diagram showing the measurement section. [Figure 85] This is an image diagram related to the measurement scale. [Figure 86] This image shows a measurement scale superimposed on a polyp. [Figure 87] This is an explanatory diagram showing the distorted grid region. [Figure 88] This is an explanatory diagram showing a square lattice region. [Figure 89] This is an explanatory diagram illustrating an example of displaying virtual scales with different shapes inside and outside the effective measurement area. [Figure 90]This is an explanatory diagram illustrating an example of displaying virtual scales of different line types within and outside the effective measurement area. [Figure 91] This is an explanatory diagram regarding still image acquisition in length measurement mode. [Figure 92] This is an explanatory diagram showing the first to third captured images. [Figure 93] This image shows the second and third captured images displayed when acquiring still images. [Figure 94] This is an explanatory diagram regarding still image acquisition in length measurement mode. [Figure 95] This is an explanatory diagram regarding still image acquisition in length measurement mode. [Figure 96] This is a block diagram showing the functions of the signal processing unit. [Figure 97] This is a block diagram showing the functions of a calibration device. [Figure 98] This is a schematic diagram of the inspection system. [Figure 99] This is a plan view showing the test chart. [Figure 100] This image shows the inspection reference position, spot, and virtual scale displayed on the inspection image. [Figure 101] This image shows the inspection reference position, spot, and virtual scale displayed on the inspection image. [Figure 102] This is a chart image with 5mm grid lines. [Figure 103] This is a chart image with 5mm grid lines (captured at a point further away than Figure 102). [Figure 104] This is an explanatory diagram showing the pixel position of the spot in the X direction. [Figure 105] This is an explanatory diagram showing the pixel position of the spot in the Y direction. [Figure 106] This is an explanatory diagram showing the pixel position of the spot in the X direction. [Figure 107] This is an explanatory diagram showing the pixel position of the spot in the Y direction. [Figure 108] This is an explanatory diagram showing a striped pattern of light ZPL. [Figure 109]This is an explanatory diagram showing the emission pattern of ZPL light, which has a striped pattern of phases X, Y, and Z. [Figure 110] This is an explanatory diagram showing a LPL (Large-Scale Photon Array) with a grid-like pattern for measurement. [Figure 111] This is an explanatory diagram showing an emission pattern in which measurement light in a grid pattern is intermittently irradiated. [Figure 112] This is an explanatory diagram showing a 3D planar optical TPL. [Figure 113] This is an explanatory diagram showing an emission pattern when intermittently irradiated with 3D planar light TPL. [Modes for carrying out the invention]

[0016] As shown in Figure 1, the endoscope system 10 includes an endoscope 12, a light source device 13, a processor device 14, a display 15, a user interface 16, an extended processor device 17, and an extended display 18. The endoscope 12 is optically connected to the light source device 13 and electrically connected to the processor device 14. The endoscope 12 has an insertion section 12a that is inserted into the body of the object to be observed, an operating section 12b provided at the base end of the insertion section 12a, and a bending section 12c and a tip section 12d provided at the tip end of the insertion section 12a. The bending section 12c bends when the operating section 12b is operated. The tip section 12d is directed in a desired direction by the bending movement of the bending section 12c.

[0017] Furthermore, the control unit 12b is equipped with an observation mode switch 12f used for switching observation modes, a still image acquisition instruction switch 12g used for instructing the acquisition of still images of the object being observed, and a zoom control unit 12h used for operating the zoom lens 21b.

[0018] The processor unit 14 is electrically connected to the display 15 and the user interface 16. The display 15 outputs and displays images or information of the object being observed, processed by the processor unit 14. The user interface 16 has a keyboard, mouse, touchpad, microphone, etc., and has the function of accepting input operations such as function settings. The expansion processor unit 17 is electrically connected to the processor unit 14. The expansion display 18 outputs and displays images or information processed by the expansion processor unit 17.

[0019] The endoscope 12 has a normal observation mode, a special observation mode, and a length measurement mode, which can be switched using the observation mode switch 12f. The normal observation mode is a mode in which the object to be observed is illuminated by illumination light. The special observation mode is a mode in which the object to be observed is illuminated by special light different from the illumination light. In length measurement mode, the object to be observed is illuminated by either illumination light or measurement light, and a virtual scale used for measuring the size of the object to be observed is displayed on the subject image obtained by imaging the object to be observed. Subject images without the superimposed virtual scale are displayed on the display 15, while subject images with the superimposed virtual scale are displayed on the extended display 18.

[0020] The illumination light is used to provide brightness to the entire object being observed, allowing for observation of the entire object. The special light is used to highlight a specific area of ​​the object being observed. The measurement light is used to display a virtual scale. In this embodiment, the virtual scale displayed on the image is described, but a real scale may be provided inside the actual lumen, allowing the real scale to be confirmed through the image. In this case, the real scale may be inserted through the forceps channel of the endoscope 12, and the real scale may protrude from the tip 12d.

[0021] When the user operates the still image acquisition instruction switch 12g, the display 15 screen freezes, and an alert sound (e.g., "beep") is emitted to indicate that a still image is being acquired. The still image of the subject obtained before and after the operation of the still image acquisition instruction switch 12g is then saved to the still image storage unit 42 (see Figure 8) in the processor device 14. The still image storage unit 42 is a storage unit such as a hard disk or USB (Universal Serial Bus) memory. If the processor device 14 can connect to a network, the still image of the subject may be saved to a network-connected still image storage server (not shown) instead of or in addition to the still image storage unit 42.

[0022] Furthermore, the still image acquisition command may be issued using operating devices other than the still image acquisition command switch 12g. For example, a foot pedal may be connected to the processor device 14, and the still image acquisition command may be issued when the user operates the foot pedal (not shown) with their foot. Mode switching may also be performed using the foot pedal. In addition, a gesture recognition unit (not shown) that recognizes user gestures may be connected to the processor device 14, and the still image acquisition command may be issued when the gesture recognition unit recognizes a specific gesture performed by the user. Mode switching may also be performed using the gesture recognition unit.

[0023] Alternatively, an eye-tracking input unit (not shown) located near the display 15 may be connected to the processor device 14, and the eye-tracking input unit may issue a still image acquisition command when it recognizes that the user's gaze has been within a predetermined area of ​​the display 15 for a certain period of time or longer. Alternatively, a voice recognition unit (not shown) may be connected to the processor device 14, and the voice recognition unit may issue a still image acquisition command when it recognizes a specific voice spoken by the user. Mode switching may also be performed using the voice recognition unit. Furthermore, an operation panel (not shown), such as a touch panel, may be connected to the processor device 14, and the system may issue a still image acquisition command when the user performs a specific operation on the operation panel. Mode switching may also be performed using the operation panel.

[0024] As shown in Figure 2, the tip portion 12d is equipped with an imaging optical system 21 for receiving light from the subject, an illumination optical system 22 for irradiating the subject with illumination light, a measurement light emission unit 23 for emitting measurement light used in length measurement mode towards the subject, an opening 24 for extending the treatment instrument toward the subject, and an air and water supply nozzle 25 for supplying air and water.

[0025] A balloon 19 is detachably attached to the insertion section 12a as a fixing member. The balloon 19 is a disposable type balloon and is discarded after one or a few uses and replaced with a new one. The number of uses referred to here is the number of cases, and a few uses means 10 times or less.

[0026] The balloon 19 is formed in a roughly cylindrical shape with its ends narrowed by an elastic material such as rubber. The balloon 19 has a small-diameter tip portion 19a and base portion 19b, and a central bulge portion 19c. After inserting the insertion portion 12a into the balloon 19 and positioning it in a predetermined location, the balloon 19 is fixed to the insertion portion 12a by, for example, fitting rubber rings 20a and 20b onto the tip portion 19a and base portion 19b.

[0027] As shown in Figure 3, the predetermined position where the balloon 19 is fixed to the insertion portion 12a is preferably on the base end side of the insertion portion 12a rather than the curved portion 12c, and the tip of the tip portion 19a of the balloon 19 coincides with the base end of the curved portion 12c. This ensures that the balloon 19 does not obstruct the bending motion of the curved portion 12c, and that the curved portion 12c does not obstruct the inflation or deflation of the balloon 19. The inflation or deflation of the balloon 19 is controlled by the balloon control device BLC, as will be described later. The balloon control device BLC is preferably operated by the user interface 16.

[0028] As shown in Figure 4, when the balloon 19 is deflated, the insertion portion 12a is not fixed to the intestinal tract 26. If measurement light is irradiated and imaging is performed with the balloon deflated, the position of the tip portion 12d may move in the up, down, left, and right directions, making it impossible to accurately irradiate the observation target that the user wants to measure with measurement light.

[0029] Therefore, as shown in Figure 5, the balloon 19 is inflated by the control of the balloon control device BLC. When the balloon 19 is inflated, its outer diameter is formed to match the inner diameter of the intestinal tract 26, so the insertion portion 12a is fixed in position within the intestinal tract 26. As a result, the insertion portion 12a is fixed to the intestinal tract 26, allowing the user to accurately irradiate the object being measured with measurement light. Note that "fixed state" here includes a state in which the position of the insertion portion 12a with respect to the insertion direction is fixed, but the orientation of the tip portion 12d can be finely adjusted.

[0030] As shown in Figure 6, the tip 12d of the endoscope is approximately circular, and along the first direction D1, an imaging optical system 21, an illumination optical system 22, an aperture 24, and an air / water supply nozzle 25 are provided. On both sides of the imaging optical system 21, two illumination optical systems 22 are provided in a second direction perpendicular to the first direction. The measurement light emission unit 23 is provided between the imaging optical system 21 and the air / water supply nozzle 25 with respect to the first direction. Therefore, since the air / water supply port of the air / water supply nozzle 25 is directed toward the imaging optical system 21 and the measurement light emission unit 23, both the imaging optical system 21 and the measurement light emission unit 23 can be cleaned by air or water supply.

[0031] As shown in Figure 7, a tip cap 27 is attached to the tip portion 12d. The tip cap 27 is provided with a tip surface 28. The tip surface 28 has a plane 28a, a plane 28b, and a guide surface 28c. Plane 28a is a plane perpendicular to the axial direction Z. Plane 28b is parallel to plane 28a and is located closer to the tip than plane 28a in the axial direction Z. The guide surface 28c is positioned between planes 28a and 28b.

[0032] The plane 28b is provided with a through-hole 27a that exposes the front end surface 21c of the imaging optical system 21, and a through-hole 27b that exposes the front end surfaces 22b of a pair of illumination optical systems 22. The front end surfaces 21c and 22b are arranged on the same plane as the plane 28.

[0033] Through holes 27c and 27d are located on the plane 28a. The air supply and water supply nozzle 25 is exposed through the through hole 27c. That is, the plane 28a is the mounting position of the air supply and water supply nozzle 25 in the axial direction Z. A spray cylinder portion 25a is formed on the tip side of the air supply and water supply nozzle 25. The spray cylinder portion 25a is formed in a cylindrical shape that protrudes from the base end of the air supply and water supply nozzle 25 in a direction that bends, for example, 90 degrees, and has a spray port 25b at its tip. The spray cylinder portion 25a is positioned to protrude from the through hole 52c towards the tip side in the axial direction Z.

[0034] The nozzle 25b is positioned facing the imaging optical system 21. As a result, the air / water supply nozzle 25 sprays a fluid, such as cleaning liquid or gas, onto the front end surface 21c and its surrounding area of ​​the imaging optical system 21.

[0035] When cleaning water or gas is sprayed from the air / water supply nozzle 25 to the imaging optical system 21, it is preferable that the flow velocity F1 of the cleaning water at the position where it reaches the imaging optical system 21, i.e., at the outer edge of the imaging optical system 21, is 2 m / s or more, and the flow velocity F2 of the gas at the outer edge of the imaging optical system 21 is 40 m / s or more. It is preferable that the flow velocities F1 and F2 satisfy the above values ​​regardless of the orientation of the tip portion 12d. For example, if the air / water supply nozzle 25 is positioned vertically below the imaging optical system 21, the flow velocity will decrease due to the influence of gravity on the cleaning water or gas, but even in this case, it is preferable that the above values ​​are satisfied.

[0036] The tip surface of the measurement light emission unit 23, exposed from the through hole 27d, is positioned on the plane 28a. That is, the mounting position of the air / water supply nozzle 25 and the tip surface of the measurement light emission unit 23 are located at the same position in the axial direction Z. The measurement light emission unit 23 is positioned within the fluid injection range of the air / water supply nozzle 25 and between the imaging optical system 21 and the air / water supply nozzle 25. In this embodiment, when the tip surface 28 is viewed from the axial direction Z, the measurement light emission unit 23 is positioned in the region connecting the injection port 25b of the air / water supply nozzle 25 and the outer edge of the imaging optical system 21. This allows fluid to be injected into the measurement light emission unit 23 simultaneously when fluid is injected from the air / water supply nozzle 25 to the imaging optical system 21.

[0037] The guide surface 28c is formed as a continuous surface connecting plane 28a and plane 28b. The guide surface 28c is a flat inclined surface that extends from a position in contact with the outer edge of the measurement light emission unit 23 to a position in contact with the outer edge of the imaging optical system 21. Since the guide surface 28c is located within the fluid injection range of the air and water supply nozzle, when fluid is injected from the air and water supply nozzle 25, fluid is also injected onto the guide surface 28c. The fluid injected onto the guide surface 28c diffuses and is blown onto the imaging optical system 21. In this case, the entire guide surface 28c may be included in the fluid injection range of the air and water supply nozzle, or only a part of the guide surface 28c may be included. In this embodiment, the entire guide surface 28c is included within the region connecting the injection port 25b of the air and water supply nozzle 25 and the outer edge of the imaging optical system 21.

[0038] As shown in Figure 8, the light source device 13 comprises a light source unit 30 and a light source processor 31. The light source unit 30 generates illumination light or special light for illuminating a subject. The illumination light or special light emitted from the light source unit 30 is incident on the light guide LG and irradiates the subject through the illumination lens 22a. As the light source unit 30, a white light source that emits white light, or a plurality of light sources including a white light source and light sources that emit light of other colors (for example, a blue light source that emits blue light) can be used as the light source for illumination light. Furthermore, as the light source unit 30, a light source that emits broadband light including blue narrowband light for emphasizing surface information such as surface blood vessels can be used as the light source for special light. The illumination light may also be a white mixed color light combining at least one of violet light, blue light, green light, or red light. In this case, it is preferable to design the optical system 22 such that the illumination range of green light is larger than the illumination range of red light.

[0039] The light source processor 31 controls the light source unit 30 based on instructions from the system control unit 41. In addition to giving instructions to the light source processor 31 regarding light source control, the system control unit 41 also controls the light source 23a (see Figure 10) of the measurement light emission unit 23. In normal observation mode, the system control unit 41 controls the illumination light to turn on and the measurement light to turn off. In special observation mode, it controls the special light to turn on and the measurement light to turn off. In length measurement mode, the system control unit 41 controls the illumination light or the measurement light to turn on or off.

[0040] The illumination optical system 22 has an illumination lens 22a, through which light from the light guide LG is irradiated onto the object to be observed. The imaging optical system 21 has an objective lens 21a, a zoom lens 21b, and an image sensor 32. Reflected light from the object to be observed enters the image sensor 32 through the objective lens 21a and the zoom lens 21b. As a result, a reflected image of the object to be observed is formed on the image sensor 32.

[0041] The zoom lens 21b has an optical zoom function that magnifies or reduces the subject by moving between the telephoto end and the wide-angle end. The optical zoom function can be switched ON and OFF by the zoom control unit 12h (see Figure 1) provided on the endoscope's control unit 12b. When the optical zoom function is ON, operating the zoom control unit 12h further magnifies or reduces the subject at a specific magnification.

[0042] The image sensor 32 is a color imaging sensor that captures a reflective image of a subject and outputs an image signal. Preferably, this image sensor 32 is a CCD (Charge Coupled Device) imaging sensor or a CMOS (Complementary Metal-Oxide Semiconductor) imaging sensor. The image sensor 32 used in this invention is a color imaging sensor for obtaining blue, green, and red images using three colors: R (red), G (green), and B (blue). The red image is output from a red pixel in the image sensor 32 that is equipped with a red color filter. The green image is output from a green pixel in the image sensor 32 that is equipped with a green color filter. The blue image is output from a blue pixel in the image sensor 32 that is equipped with a blue color filter. The image sensor 32 is controlled by the imaging control unit 33.

[0043] The image signal output from the image sensor 32 is transmitted to the CDS / AGC circuit 34. The CDS / AGC circuit 34 performs correlated double sampling (CDS) and automatic gain control (AGC) on the analog image signal. The image signal that has passed through the CDS / AGC circuit 34 is converted into a digital image signal by an A / D converter (A / D (Analog / Digital) converter) 35. The A / D converted digital image signal is input to the communication interface 37 of the light source device 13 via the communication interface 36.

[0044] The processor unit 14 has programs related to various processing or control stored in a program storage memory (not shown). The system control unit 41, which is composed of an image control processor, operates the programs stored in the program storage memory to realize the functions of the receiving unit 38 connected to the communication interface 37 of the light source unit 13, the signal processing unit 39, and the display control unit 40.

[0045] The receiving unit 38 receives the image signal transmitted from the communication interface 37 and transmits it to the signal processing unit 39. The signal processing unit 39 has a built-in memory for temporarily storing the image signal received from the receiving unit 38, and processes the image signal group, which is a collection of image signals stored in the memory, to generate an image of the subject. The receiving unit 38 may also send control signals related to the light source processor 31 directly to the system control unit 41.

[0046] In the signal processing unit 39, when set to normal observation mode, the blue image of the subject image is assigned to the B channel of the display 15, the green image of the subject image to the G channel of the display 15, and the red image of the subject image to the R channel of the display 15, thereby displaying a color image of the subject on the display 15. The same signal assignment processing is performed in the length measurement mode as in the normal observation mode.

[0047] On the other hand, when the special observation mode is set, the signal processing unit 39 does not use the red image of the subject image for display on the display 15. Instead, it assigns the blue image of the subject image to the B and G channels of the display 15, and the green image of the subject image to the R channel of the display 15, thereby displaying a pseudo-color subject image on the display 15. Also, when the length measurement mode is set, the signal processing unit 39 transmits the subject image, including the irradiation position of the measurement light, to the data transmission / reception unit 43. The data transmission / reception unit 43 transmits data related to the subject image to the expansion processor device 17. The data transmission / reception unit 43 is also capable of receiving data from the expansion processor device 17. The received data can be processed by the signal processing unit 39 or the system control unit 41.

[0048] The signal processing unit 39, as a zoom function, enlarges or reduces the subject at a specific magnification by cropping and enlarging or reducing a portion of the subject image when the digital zoom function is set to ON by the user interface 16. Figure 9(A) shows a subject image with the digital zoom function OFF, and Figure 9(B) shows a subject image with the digital zoom function ON, obtained by cropping and enlarging the central portion of the subject image in Figure 9(A). Note that when the digital zoom function is OFF, no enlargement or reduction of the subject by cropping the subject image is performed.

[0049] The display control unit 40 displays the subject image generated by the signal processing unit 39 on the display 15. The system control unit 41 performs various controls on the endoscope 12, light source device 13, processor device 14, and expansion processor device 17. The system control unit 41 controls the image sensor 32 via the image control unit 33 provided in the endoscope 12. In conjunction with the control of the image sensor 32, the image control unit 33 also controls the CDS / AGC circuit 34 and the A / D converter 35.

[0050] The extended processor unit 17 receives data transmitted from the processor unit 14 via the data transmission / reception unit 44. The signal processing unit 45 performs processing related to the length measurement mode based on the data received by the data transmission / reception unit 44. Specifically, it determines the size of the virtual scale from the subject image including the irradiation position of the measurement light, and overlays the determined virtual scale onto the subject image. The display control unit 46 displays the subject image with the overlaid virtual scale on the extended display 18. The data transmission / reception unit 44 is also capable of transmitting data to the processor unit 14.

[0051] As shown in Figure 10, the measurement light emission unit 23 emits measurement light obliquely to the optical axis Ax (see Figure 13) of the imaging optical system 21. The measurement light emission unit 23 comprises a light source 23a, a diffractive optical element (DOE) 23b, a prism 23c, and an emission unit 23d. The light source 23a emits light of a color detectable by the pixels of the image sensor 32 (specifically, visible light), and includes a light-emitting element such as a laser diode (LD) or light-emitting diode (LED), and a focusing lens that focuses the light emitted from this light-emitting element. The light source 23a is provided on a scope electronic circuit board (not shown). The scope electronic circuit board is provided at the tip 12d of the endoscope and receives power from the light source device 13 or processor device 14 to supply power to the light source 23a. Although the light source 23a is provided at the tip 12d of the endoscope, it may also be provided inside the connector that connects the endoscope 12 and the processor device 14. Even in this case, the components of the measurement light emission unit 23 other than the light source 23a (diffractive optical element DOE 23b, prism 23c, and emission unit 23d) are provided at the tip 12d of the endoscope.

[0052] In this embodiment, the wavelength of light emitted by the light source 23a is, for example, red laser light (beam color) with a wavelength of 600 nm to 650 nm, but light in other wavelength ranges, such as green light with a wavelength of 495 nm to 570 nm, may also be used. The light source 23a is controlled by the system control unit 41 and emits light based on instructions from the system control unit 41. The DOE 23b converts the light emitted from the light source into measurement light for obtaining measurement information. The measurement light is adjusted in light intensity based on the viewpoint of protecting the human body, eyes, and internal organs, and is preferably adjusted to a light intensity that causes sufficient overexposure (pixel saturation) within the observation range of the endoscope 12.

[0053] The prism 23c is an optical member for changing the traveling direction of the measurement light after conversion by the DOE 23b. The prism 23c changes the traveling direction of the measurement light so as to intersect the field of view of the imaging optical system 21 including the objective lens 21a. Details of the traveling direction of the measurement light will be described later. The measurement light Lm emitted from the prism 23c is irradiated onto the subject.

[0054] As shown in FIG. 11, the measurement light emitting unit 23 is housed in a housing portion 47 for the measurement light emitting unit provided at the distal end portion 12d of the endoscope. The housing portion 47 for the measurement light emitting unit has a hole portion corresponding to the size of the measurement light emitting unit 23. The housing portion 47 for the measurement light emitting unit is closed by a transparent lid 48. The transparent lid 48 has a transparent plate shape, and one end surface is a flat portion 48a. The transparent lid 48 is arranged such that the flat portion 48a is flush with the distal end surface 28 of the distal end portion 12d. By providing the transparent lid 48 flush with the distal end surface 28, foreign matters and the like that may block the emission of the measurement light are prevented from being caught.

[0055] As shown in FIG. 12, a prism 49 is arranged between the transparent lid 48 and the prism 23c. The prism 49 has a first contact surface 49a and a second contact surface 49b, and the first contact surface 49a is in contact with the prism 23c, and the second contact surface 49b is in contact with the transparent lid 48. The prism 49 excludes gas from between the transparent lid 48 and the prism 23c to make it airtight. By making it airtight in this way, condensation can be prevented. That is, it is possible to prevent problems such as attenuation, diffusion, convergence, and refraction of the measurement light due to condensation.

[0056] When the refractive index of the prism 23c is "n1" and the refractive index of the prism 49 is "n2", an example where "n1 < n2" is satisfied and the light emitting surface of the prism 23c is inclined toward the optical axis Ax has been described, but the reverse configuration may also be used. It may be set as "n1 > n2" and the light emitting surface of the prism 23c may be provided on the side opposite to the optical axis Ax. However, in this case, since there is a possibility of total reflection at the light emitting surface of the prism 23c, it is necessary to provide a limitation to the light emitting surface of the prism 23c.

[0057] Alternatively, instead of constructing the prism 23c from an optical material, it may be a measurement-assisting slit formed at the tip 12d of the endoscope. Furthermore, if the prism 23c is constructed from an optical material, it is preferable to apply an anti-reflective coating (AR (Anti-Reflection) coating) (anti-reflective section) to the emission surface. This anti-reflective coating is provided because if the measurement light is reflected without passing through the emission surface of the prism 23c, and the proportion of measurement light irradiated onto the subject decreases, the irradiation position detection unit 61, described later, will have difficulty recognizing the position of the spot SP formed on the subject by the measurement light.

[0058] The measurement light emission unit 23 only needs to be capable of emitting measurement light toward the field of view of the imaging optical system 21. For example, the light source 23a may be provided in a light source device, and the light emitted from the light source 23a may be guided to the DOE 23b by an optical fiber. Alternatively, without using a prism 23c, the orientation of the light source 23a and DOE 23b may be set diagonally with respect to the optical axis Ax of the imaging optical system 21, thereby emitting measurement light Lm in a direction that crosses the field of view of the imaging optical system 21.

[0059] As shown in Figure 13, the direction of propagation of the measurement light is such that the optical axis of the measurement light Lm intersects with the optical axis Ax of the imaging optical system 21. Assuming that observation is possible within the observation distance range Rx, it can be seen that the position of the spot SP formed on the subject by the measurement light Lm within the imaging range (indicated by arrows Qx, Qy, and Qz) differs at the near end Px, the central part Py, and the far end Pz of the range Rx (the point where each arrow Qx, Qy, and Qz intersects the optical axis Ax). The field of view of the imaging optical system 21 is represented within the region bounded by the two solid lines 101X, and measurements are performed in the central region of this field of view where aberrations are minimal (the region bounded by the two dotted lines 102X). Furthermore, the third direction D3 is a direction orthogonal to the first direction D1 and the second direction D2 (the same applies to Figure 45).

[0060] As described above, by emitting the measurement light Lm with its optical axis intersecting the optical axis Ax, the size of the subject can be measured from the movement of the spot position in response to changes in observation distance. Then, by imaging the subject illuminated by the measurement light with the image sensor 32, a subject image including the spot SP is obtained. In the subject image, the position of the spot SP varies depending on the relationship between the optical axis Ax of the imaging optical system 21 and the optical axis of the measurement light Lm, as well as the observation distance. When the observation distance is short, there will be more pixels indicating the same actual size (e.g., 5 mm), and when the observation distance is long, there will be fewer pixels.

[0061] As shown in Figure 14, the system control unit 41 includes a length-measuring endoscope feasibility determination unit 140, a measurement light ON / OFF switching unit 141, a length-measuring image display setting ON / OFF switching unit 142, a length-measuring function operation status display ON / OFF switching unit 143, a virtual scale display switching control unit 144, and a pre-switch image display setting storage unit 149.

[0062] The length-measuring endoscope compatibility determination unit 140 determines whether the endoscope 12 is a length-measuring endoscope when the endoscope 12 is connected to the processor device 14. If the endoscope 12 is a length-measuring endoscope, switching to length-measuring mode is enabled. A length-measuring endoscope is an endoscope that is capable of irradiating and receiving measurement light, and can display a length-measuring image that displays a virtual scale based on the measurement light on the extended display 18 (or display 15). The length-measuring endoscope compatibility determination unit 140 includes a scope ID table (not shown) that associates the scope ID provided on the endoscope 12 with a flag indicating whether or not it is a length-measuring endoscope (for example, "1" for a length-measuring endoscope and "0" for other endoscopes). When the endoscope 12 is connected, the length-measuring endoscope compatibility determination unit 140 reads the endoscope scope ID. The read scope ID is then used to determine whether or not it is a length-measuring endoscope by referring to the flag in the scope ID table.

[0063] The measurement light ON / OFF switching unit 141 controls the light source 23a to switch the measurement light on (ON) or off (OFF). The length measurement image display setting ON / OFF switching unit 142 enables (ON) or disables (OFF) the execution of various image display settings in length measurement mode, such as display settings (color tone, etc.) for the length measurement image, via the user interface 16, etc. The virtual scale display switching control unit 144 switches the display of the virtual scale on the extended display 18 to either display (ON), hide (OFF), or change the display mode.

[0064] When switching to the length measurement mode is enabled, the system control unit 41 performs at least one of the following actions when the observation mode switch 12f is used to switch to the length measurement mode: switching the measurement light ON or OFF, switching the length measurement image display setting ON or OFF, switching the length measurement function operation status display ON or OFF, switching the virtual scale display ON or OFF, or switching the display mode.

[0065] For example, it is preferable that the system control unit 41 switches the measurement light ON, the measurement image display setting ON, the measurement function operation status display ON, and the virtual scale display ON when switching to the measurement mode. On the other hand, it is preferable that the measurement light OFF, the measurement image display setting OFF, the measurement function operation status display OFF, and the virtual scale display OFF when switching from the measurement mode to another mode (normal observation mode, special observation mode).

[0066] As shown in Figure 15, the measurement function operation status is preferably displayed as a scale display icon 146 in the supplementary information display area 18a of the extended display 18. The scale display icon 146 is displayed when switching to measurement mode, and disappears when switching from measurement mode to another mode. The virtual scale 147 is preferably displayed in the observation image display area 18b of the extended display 18. The display mode of the virtual scale 147 is changed by the virtual scale display switching control unit 144.

[0067] Virtual scale 147 includes virtual scales 147a (5mm), 147b (10mm), and 147c (20mm). Virtual scales 147a, 147b, and 147c each have a circular scale (displayed as a dotted line) and a line segment scale (displayed as a solid line). The "5" in virtual scale 147a indicates a 5mm scale, the "10" in virtual scale 147b indicates a 10mm scale, and the "20" in virtual scale 147c indicates a 20mm scale.

[0068] The display mode of the virtual scale can be changed, for example, by selecting from a predetermined set of scale patterns. For example, as shown in Figure 15, the scale patterns include a combination of three virtual scales 147a, 147b, and 147c that have both a circular scale and a line segment scale, a combination of two virtual scales 147b and 147c that have both a circular scale and a line segment scale, and a combination of three virtual scales 147a, 147b, and 147c that have only line segments. The scale pattern is represented by one or more combinations of multiple scale sizes and multiple scale shapes, such as circular scales and line segment scales.

[0069] Furthermore, when turning on the length measurement image display setting, it is preferable to save the image display setting from before switching to the length measurement mode in the pre-switch image display setting storage unit 149. For example, if the observation mode before switching to the length measurement mode is the normal observation mode, it is preferable to save the image display setting for the normal observation mode set in the signal processing unit 39 in the pre-switch image display setting storage unit 149. Also, when turning off the length measurement image display setting, it is preferable to switch to the image display setting saved in the pre-switch image display setting storage unit 149. For example, if the image display setting for the normal observation mode is saved in the pre-switch image display setting storage unit 149, the signal processing unit 39 will set the image display setting for the normal observation mode saved in the pre-switch image display setting storage unit 149 in accordance with the switch to the normal observation mode.

[0070] On the other hand, when switching to the length measurement mode, the system control unit 41 prohibits switching the measurement light ON, the length measurement image display setting ON, the length measurement function operation status display ON, and the virtual scale display ON if the mode switching conditions are not met. The mode switching conditions are setting conditions for the endoscope 12, light source device 13, processor device 14, and extended processor device 17 that are suitable for executing the length measurement mode. Preferably, the mode switching conditions are conditions that do not fall under the prohibited setting conditions below. If the mode switching conditions are not met, instead of prohibiting the display of the scale display icon 146, it is preferable to display (ON) the length measurement function operation status display, which indicates that the virtual scale 147 is not being displayed on the extended display 18. Preferably, the length measurement function operation status display is displayed in the supplementary information display area 18a as the scale not displayed icon 148.

[0071] As shown in Figure 16, the system control unit 41 is provided with a length measurement mode control unit 50 that controls whether or not the length measurement mode can be executed. The length measurement mode control unit 50 performs at least one of the following: a first control that prohibits switching to the length measurement mode when the observation mode switching operation is performed by the observation mode switching switch 12f and the currently set setting conditions for the endoscope 12, light source device 13, and processor device 14 fall under predetermined prohibited setting conditions; a second control that disables the setting change operation when the setting conditions to be changed by the user interface 16 are in length measurement mode and the setting conditions to be changed by the setting change operation fall under prohibited setting conditions; or a third control that automatically switches from the length measurement mode to another mode when the setting conditions to be changed by the user interface 16 are in length measurement mode and the setting conditions to be changed by the setting change operation fall under prohibited setting conditions.

[0072] The setting conditions for the light source device 13 include the illumination conditions for the illumination light used in the normal observation mode or length measurement mode, the illumination conditions for the special light used in the special observation mode, or the illumination conditions for the measurement light used in the length measurement mode. Illumination conditions include, for example, illumination light intensity. The setting conditions for the endoscope 12 include imaging conditions for imaging the subject. Imaging conditions include, for example, shutter speed. The setting conditions for the processor device 14 include processing conditions such as image processing for the subject image. Processing conditions include, for example, color balance, brightness correction, and various enhancement processing. In length measurement mode, it is preferable to optimize the position detection of the spot SP and set the setting conditions (illumination light intensity, shutter speed, color balance, brightness correction, various enhancement processing) to satisfy the user's visibility when measuring dimensions.

[0073] The prohibited setting conditions include a first prohibited setting condition that prevents the detection of the irradiation position of the measurement light from the subject image in the length measurement mode, and a second prohibited setting condition that prevents the accurate display of a virtual scale corresponding to the observation distance in the length measurement image. Examples of the first prohibited setting conditions include special observation mode, brightness enhancement or red enhancement for the subject image. In special observation mode, the red image used for detecting spot SP etc. in length measurement mode is not used for image display, making it difficult to detect the irradiation position of the measurement light. In length measurement mode, it is preferable to lower the brightness of the subject image and suppress the redness compared to normal observation mode or special observation mode.

[0074] Furthermore, the second prohibited setting condition includes, for example, the use (ON) of zoom functions such as optical zoom or digital zoom. This is because the virtual scale displayed in the measurement image is determined according to the position of the spot SP, and not according to the magnification of the zoom function. Therefore, when the zoom function is ON, it is difficult to display the virtual scale in accordance with the observation distance.

[0075] For example, if the device is set to special observation mode and the observation mode switch 12f is used to switch to measurement mode, the measurement mode control unit 50 performs a first control to prohibit switching to measurement mode and maintain the special observation mode state, as shown in Figure 17. When the first control is performed, the measurement mode control unit 50 displays a message on the extended display 18 indicating that switching to measurement mode has been prohibited, as shown in Figure 18 (it may also emit a warning sound). Alternatively, instead of prohibiting switching to measurement mode, the measurement mode control unit 50 may perform a control to deactivate the special observation mode and switch to measurement mode, as shown in Figure 19.

[0076] Furthermore, if a setting change operation to turn on the zoom function is performed by the zoom operation unit 12h while in measurement mode, the measurement mode control unit 50 performs a second control to disable the setting change operation to turn on the zoom function, as shown in Figure 20. When the second control is performed, the measurement mode control unit 50 displays a message on the extended display 18 indicating that the setting change operation to turn on the zoom function has been disabled, as shown in Figure 21 (it may also emit a warning sound).

[0077] Furthermore, if the zoom function is turned ON by the zoom operation unit 12h while in measurement mode, the measurement mode control unit 50 performs a third control, as shown in Figure 22, to cancel the measurement mode and switch to another mode, the normal observation mode. When the third control is performed, the measurement mode control unit 50 displays a message on the extended display 18 (or emits a warning sound) indicating that the measurement mode has been canceled (virtual scale is hidden) and the system has switched to the normal observation mode, as shown in Figure 23. When the third control is performed, the setting change operation to turn on the zoom function is activated, and the subject on the subject image is enlarged or reduced using the zoom function.

[0078] As shown in Figure 24, the system control unit 41 may include a brightness information calculation unit 53, an illumination light intensity level setting unit 54, a first light emission control table 55, and a second light emission control table 56. The brightness information calculation unit 53 calculates brightness information related to the brightness of the subject based on an image obtained in normal observation mode or a first captured image (an image based on illumination light and measurement light) obtained in length measurement mode. The illumination light intensity level setting unit 54 sets the light intensity level of the illumination light based on the brightness information. There are five levels for the illumination light intensity: Level 1, Level 2, Level 3, Level 4, and Level 5. Information regarding the illumination light intensity level is sent to the light source processor 31. The light source processor 31 controls the light source unit 30 so that the illumination light intensity matches the illumination light intensity level.

[0079] The first light emission control table 55 is used to control the light intensity of the measurement light and stores the first relationship between the coordinate information of the spot SP and the light intensity level of the measurement light. Specifically, as shown in Figure 25, the light intensity levels Level 1, Level 2, Level 3, Level 4, and Level 5 of the measurement light are defined for each of the five coordinate areas to which the coordinate information of the spot SP belongs. The system control unit 41 refers to the first light emission control table 55 to identify the light intensity level corresponding to the coordinate area to which the position of the spot SP belongs. The system control unit 41 controls the light source 23a to control the light intensity of the measurement light so that it reaches the identified light intensity level. Note that whether to use the first light emission control table 55 or the second light emission control table 56 to control the light intensity of the measurement light is set as appropriate by operating the user interface 16.

[0080] As shown in Figure 26, coordinate area 1 is the area set at the bottom in the first captured image, and when spot SP is located in coordinate area 1, it is at the closest observation distance. Therefore, the lowest utility level of the measurement light, Level 1, is assigned to coordinate area 1. Coordinate area 2 is an area set above coordinate area 1, and when spot SP is located in coordinate area 2, the observation distance is farther than in the case of coordinate area 1, so a light intensity level higher than Level 1, Level 2, is assigned to the measurement light. The direction of movement of spot SP changes according to the intersection direction of the optical axis Ax of the imaging optical system 21 and the optical axis Lm of the measurement light.

[0081] Similarly, coordinate area 3 is located above coordinate area 2. When spot SP is located in coordinate area 3, the observation distance is greater than in coordinate area 2, so a light intensity level of Level 3, which is higher than Level 2, is assigned to the measurement light. Coordinate area 4 is also located above coordinate area 3. When spot SP is located in coordinate area 4, the observation distance is greater than in coordinate area 3, so a light intensity level of Level 4, which is higher than Level 3, is assigned to the measurement light. Coordinate area 5 is the area set at the very top. When spot SP is located in coordinate area 5, the observation distance is the farthest from the other coordinate areas 1-4, so the highest light intensity level of Level 5 is assigned to the measurement light.

[0082] The second light emission control table 56 is used to control the light intensity of the measurement light and stores the coordinate information of the spot SP and the second relationship between the light intensity level of the illumination light and the light intensity level of the measurement light. Specifically, as shown in Figure 27, the light intensity level of the measurement light is defined for each of the five coordinate areas to which the coordinate information of the spot SP belongs and the light intensity levels of the illumination light Level 1, Level 2, Level 3, Level 4, and Level 5. For example, if the spot SP belongs to coordinate area 1 and the light intensity level of the illumination light is Level 3, then Level 3 is assigned as the light intensity level of the measurement light.

[0083] The system control unit 41 refers to the second light emission control table 56 to determine the light intensity level of the measurement light from the coordinate area to which the position of the spot SP belongs and the light intensity level of the illumination light. The system control unit 41 controls the light source 23a to control the light intensity of the measurement light so that it reaches the determined light intensity level.

[0084] In the second light emission control table 56, the light intensity level of the illumination light and the light intensity level of the measurement light are set to a ratio that allows the position of the spot SP to be identified. This is because if the ratio between the light intensity of the illumination light and the light intensity of the measurement light is not appropriate, the contrast of the spot SP will be low, making it difficult to identify the position of the spot SP.

[0085] In length measurement mode, the light source processor 31 continuously emits illumination light used for overall illumination of the object being observed, while pulsed emission of measurement light Lm. Therefore, as shown in Figure 28, the frames that emit light in length measurement mode include illumination light-only emission frames FLx, which emit illumination light alone without emitting measurement light, and measurement light emission frames FLy, which emit both illumination light and measurement light. In length measurement mode, the position of spot SP is detected from the first captured image obtained in the measurement light emission frame FLy, while a virtual scale is displayed on the second captured image obtained in the illumination light-only emission frame FLx. The solid lines shown in Figure 28 in the parts corresponding to illumination light or measurement light represent the emission state in a given frame. The period when the solid line is in the part corresponding to "on" indicates the period when illumination light or measurement light is emitted, and the period when the solid line is in the part corresponding to "off" indicates the period when illumination light or measurement light is not emitted.

[0086] The emission and imaging patterns in the length measurement mode are as follows. The first pattern is when a CCD (global shutter type image sensor) is used as the image sensor 32, which outputs an image signal by performing exposure and charge readout at the same timing for each pixel. In the first pattern, the measurement light is emitted every two frames, with a specific frame interval.

[0087] In the first pattern, as shown in Figure 29, based on the exposure of illumination light at timing T1 in normal observation mode, a simultaneous readout of the charge (global shutter) is performed when switching between normal observation mode and length measurement mode (when switching from timing T1 to timing T2), thereby obtaining a second image N containing only the illumination light component. This second image N is displayed on the extended display 18 at timing T2. Regarding the "CCD (frame period) global shutter" in Figure 29, the rising line 57 that rises vertically at the time of switching from timing T1 to timing T2 indicates that the global shutter was performed. This is also true for other rising lines 57.

[0088] Furthermore, at timing T2, illumination light and measurement light are emitted. Based on the exposure of illumination light and measurement light at timing T2, a simultaneous readout of the charge is performed when switching from timing T2 to timing T3, thereby obtaining a first image N+Lm containing components of illumination light and measurement light. Based on this first image N+Lm, the position of spot SP is detected. A virtual scale corresponding to the position of spot SP is displayed on the second image N that was displayed at timing T2. As a result, at timing T3, a measurement image S with the virtual scale displayed is displayed on the second image N from timing T2.

[0089] Furthermore, the second captured image N at timing T2 (first timing) will be displayed on the extended display 18 not only at timing T2 but also at timing T3. That is, the second captured image at timing T2 will be displayed for two consecutive frames until timing T4 (second timing) when the next second captured image is obtained (the same subject image will be displayed at timings T2 and T3). At timing T3, the first captured image N+Lm will not be displayed on the extended display 18. Here, in normal observation mode, the second captured image N is changed and displayed every frame, whereas in the first pattern of length measurement mode, as described above, the same second captured image N is displayed for two consecutive frames, so the frame rate of the first pattern of length measurement mode is effectively half that of normal observation mode.

[0090] The same procedure is followed for timings T4 and beyond. Specifically, at timings T4 and T5, the second image captured at timing T4 is displayed consecutively with respect to the measurement image S, and at timings T6 and T7, the second image captured at timing T6, N, is displayed consecutively with respect to the measurement image S. In contrast, at timings T4, T5, T6, and T7, the first image captured N+Lm is not displayed on the extended display 18. In this way, by displaying the second image captured N, which does not contain the measurement light component, with the measurement image S, the frame rate is slightly reduced, but the visibility of the observed object, which may be affected by the emission of measurement light, is eliminated.

[0091] The second pattern involves using a CMOS (rolling shutter type image sensor) as the image sensor 32, which has multiple lines for imaging an object illuminated by illumination light or measurement light, and performs exposure at different exposure timings for each line, and reads out the charge at different readout timings for each line to output an image signal. In the second pattern, the measurement light is emitted at specific frame intervals, every three frames.

[0092] In the second pattern, as shown in Figure 30, exposure to illumination light and reading out the charge are performed line by line at timing T1, and the reading out of the charge is completed when switching from the normal observation mode to the length measurement mode (when switching from timing T1 to timing T2) (rolling shutter), thereby obtaining a second image N containing only the illumination light component. This second image N is displayed on the extended display 18 at timing T2. Regarding "CMOS (frame period) rolling shutter" in Figure 30, the diagonal line 58 represents the timing of light exposure and charge reading, with line Ls indicating the start of exposure and charge reading, and line Lt indicating the completion of exposure and charge reading.

[0093] Furthermore, at timing T2, illumination light and measurement light are emitted. Based on the illumination of the illumination light from timing T1 to timing T2, and the illumination of the measurement light at timing T2, a rolling shutter is performed, and at the timing of the switch from timing T2 to timing T3, a first image N+Lm containing components of illumination light and measurement light is obtained. Similarly, at the timing of the switch from timing T3 to timing T4, a first image N+Lm containing components of illumination light and measurement light is obtained. Based on the above first image N+Lm, the position of spot SP is detected. At timings T3 and T4, no measurement light is emitted.

[0094] A virtual scale corresponding to the position of spot SP is displayed on the second captured image N, which was displayed at timing T2. As a result, at timings T3 and T4, the measured length image S with the virtual scale displayed on the second captured image N at timing T2 is displayed. Note that the second captured image N at timing T2 (first timing) is displayed on the extended display 18 not only at timing T2, but also at timings T3 and T4. That is, the second captured image at timing T2 is displayed for 3 consecutive frames until timing T5 (second timing) when the next second captured image is obtained (the same subject image is displayed at timings T2, T3, and T4). In contrast, at timings T3 and T4, the first captured image N+Lm is not displayed on the extended display 18. Note that in the second pattern of the measured length mode, by displaying the same second captured image N2 for 3 consecutive frames, the frame rate of the second pattern of the measured length mode is effectively 1 / 3 of that of the normal observation mode.

[0095] The same procedure is followed for timings T5 and later. At timings T5, T6, and T7, the second image captured at timing T5 is displayed relative to the measurement image S. In contrast, at timings T5, T6, and T7, the first image captured N+Lm is not displayed on the extended display 18. By displaying the second image captured, which does not contain the measurement light component, in relation to the measurement image S, the frame rate is reduced, but the visibility of the observed object, which may be affected by the emission of planar measurement light, is eliminated.

[0096] As shown in Figure 31, the signal processing unit 45 of the expansion processor unit 17 includes a first signal processing unit 59 that detects the position of the spot SP in the captured image, and a second signal processing unit 60 that sets the virtual scale according to the position of the spot SP, in order to recognize the position of the spot SP and set the virtual scale. The captured image includes not only the captured image obtained when the illumination light and measurement light are constantly on, but also the first captured image obtained when both the illumination light and the measurement light are on, in cases where the illumination light is constantly on while the measurement light is on or off.

[0097] The first signal processing unit 59 includes an irradiation position detection unit 61 that detects the irradiation position of the spot SP from the captured image. Preferably, the irradiation position detection unit 61 obtains the centroid coordinates of the spot SP as the irradiation position of the spot SP.

[0098] The second signal processing unit 60 sets a first virtual scale as a virtual scale for measuring the size of a subject based on the irradiation position of the spot SP, and sets the scale display position of the first virtual scale. The second signal processing unit 60 sets a virtual scale corresponding to the irradiation position of the spot SP by referring to a scale table 62 that stores virtual scale images, whose display manner differs depending on the irradiation position of the spot SP and the scale display position, in association with the irradiation position of the spot. The virtual scale differs in size or shape, for example, depending on the irradiation position of the spot SP and the scale display position. The display of the virtual scale image will be described later. Furthermore, the contents of the scale table 62 are maintained even when the power of the expansion processor device 17 is turned OFF. Note that the scale table 62 stores virtual scale images in association with the irradiation position, but it may also store virtual scale images in association with the distance to the subject corresponding to the irradiation position (the distance between the tip 12d of the endoscope 12 and the subject).

[0099] Furthermore, since a virtual scale image is required for each irradiation position and the data size is large, it is preferable to store it in the expansion processor unit 17 (or processor unit 14) rather than in the memory (not shown) of the endoscope 12, considering the memory capacity that can be stored in the endoscope 12, startup time, and processing time. Also, as will be described later, the virtual scale image is created from representative points of the virtual scale image obtained by calibration, but if the virtual scale image is created from representative points in the length measurement mode, a loss of time occurs and the real-time processing is impaired. For this reason, after the endoscope 12 is connected to the endoscope connection unit and the scale table 62 is updated by creating a virtual scale image from representative points once, the virtual scale image is not created from representative points again, but the updated scale table 62 is used to display the virtual scale image. In addition, in the second signal processing unit 60, in emergencies when it is difficult to display superimposed images, a reference marker that determines the size of the virtual scale based on the relationship between the irradiation position of the spot SP and the number of pixels corresponding to the actual size of the subject is displayed on the length measurement image instead of the virtual scale image superimposed on the length measurement image.

[0100] Furthermore, the second signal processing unit 60 includes a table update unit 64 for updating the scale table 62 when the endoscope 12 is connected to the endoscope connection unit. The reason for enabling the updating of the scale table 62 in this way is that the positional relationship between the optical axis of the measurement light Lm and the imaging optical system 21 differs depending on the model and serial number of the endoscope 12, and accordingly, the display manner of the virtual scale image also changes. The table update unit 64 uses a representative point data table 66 that stores representative point data related to representative points extracted from the virtual scale image in association with the irradiation position. Details of the table update unit 64 and the representative point data table 66 will be described later. Note that the representative point data table 66 may also store the representative point data in association with the distance to the subject corresponding to the irradiation position (the distance between the tip 12d of the endoscope 12 and the subject).

[0101] When the display control unit 46 displays a length-measuring image with a virtual scale superimposed on the captured image on the extended display 18, it controls the display mode of the virtual scale according to the irradiation position of the spot SP and the scale display position. Specifically, the display control unit 46 displays a length-measuring image with a first virtual scale superimposed on the spot SP on the extended display 18. For example, a circular measurement marker is used as the first virtual scale. In this case, as shown in Figure 32, when the observation distance is close to the near end Px (see Figure 13), a virtual scale M1 indicating an actual size of 5 mm (horizontal and vertical directions of the captured image) is displayed aligned with the center of the spot SP1 formed on the tumor tm1 of the subject.

[0102] The virtual scale M1 is located in the peripheral area of ​​the captured image, which is affected by distortion from the imaging optical system 21. Therefore, the virtual scale M1 is elliptical in shape to accommodate the effects of distortion. Since the virtual scale M1 roughly coincides with the area of ​​tumor tm1, tumor tm1 can be measured to be approximately 5 mm. Alternatively, the captured image may be displayed without showing the spot, and only the first virtual scale may be displayed.

[0103] Furthermore, as shown in Figure 33, when the observation distance is close to the center Py, a virtual scale M2 representing the actual size of 5 mm (horizontal and vertical directions of the captured image) is displayed, aligned with the center of the spot SP2 formed on the tumor tm2 of the subject. Since the scale display position of the virtual scale M2 is located in the center of the captured image, which is less affected by distortion due to the imaging optical system 21, the virtual scale M2 is circular and unaffected by distortion.

[0104] Furthermore, as shown in Figure 34, a virtual scale M3 representing the actual size of 5 mm (horizontal and vertical directions of the captured image) is displayed aligned with the center of the spot SP3 formed on the tumor tm3 of the subject. Since the display position of the virtual scale M3 is located in the peripheral part of the captured image, which is affected by distortion caused by the imaging optical system 21, the virtual scale M3 is elliptical in accordance with the effects of distortion. As shown in Figures 32 to 34 above, the size of the first virtual scale corresponding to the same actual size of 5 mm decreases as the observation distance increases. Also, the shape of the first virtual scale differs depending on the scale display position in accordance with the effects of distortion caused by the imaging optical system 21.

[0105] In Figures 32 to 34, the center of the spot SP and the center of the marker are shown aligned. However, if this does not pose a problem in terms of measurement accuracy, the first virtual scale may be displayed at a position away from the spot SP. However, even in this case, it is preferable to display the first virtual scale near the spot. Furthermore, instead of displaying a distorted first virtual scale, the distortion aberration of the captured image may be corrected, and the first virtual scale in its undeformed state may be displayed on the corrected captured image.

[0106] Furthermore, while Figures 32 to 34 show a first virtual scale corresponding to the actual size of the subject (5 mm), the actual size of the subject may be set to any value (e.g., 2 mm, 3 mm, 10 mm, etc.) depending on the object and purpose of observation. Also, while Figures 32 to 34 show the first virtual scale as a roughly circular shape, it may also be a cross shape with intersecting vertical and horizontal lines, as shown in Figure 35. Alternatively, a graduated cross shape may be used, with a scale Mx added to at least one of the vertical and horizontal lines of the cross shape. Furthermore, the first virtual scale may be a distorted cross shape with at least one of the vertical or horizontal lines tilted. Additionally, the first virtual scale may be a combination of a cross shape and a circle, resulting in a circular and cross shape. Moreover, the first virtual scale may be a measurement point cloud type, combining multiple measurement points EP corresponding to the actual size from a spot. Furthermore, there may be one or more first virtual scales, and the color of the first virtual scale may be changed according to the actual size.

[0107] Furthermore, as a first virtual scale, as shown in Figure 36, three concentric virtual scales of different sizes M4A, M4B, and M4C (with diameters of 2 mm, 5 mm, and 10 mm, respectively) may be displayed on the acquired image with the spot SP4 formed on the tumor tm4 as the center. Displaying these three concentric virtual scales eliminates the need to switch between multiple virtual scales and allows measurement even when the subject has a non-linear shape. When displaying multiple concentric virtual scales with the spot as the center, instead of specifying the size and color for each virtual scale, multiple combinations of conditions may be prepared in advance and selected from among these combinations.

[0108] In Figure 36, the three concentric virtual scales are all displayed in the same color (black). However, when displaying multiple concentric markers, multiple colored concentric markers can be used, with the color varying depending on the virtual scale. As shown in Figure 37, virtual scale M5A is represented by a dotted line in red, virtual scale M5B by a solid line in blue, and virtual scale M5C by a dashed line in white. Changing the color of the virtual scales in this way improves their identifiability and makes measurement easier.

[0109] Furthermore, as the first virtual scale, in addition to multiple concentric virtual scales, multiple distorted concentric virtual scales may be used, as shown in Figure 38, by distorting each concentric circle. In this case, the distorted concentric virtual scales M6A, M6B, and M6C are displayed in the acquired image centered on the spot SP5 formed in tumor tm5.

[0110] When the endoscope 12 is connected to the endoscope connection unit, the table update unit 64 refers to the representative point data table 66 to create a virtual scale image corresponding to the model and / or serial number of the endoscope 12 and updates the scale table 62.

[0111] The representative point data table 66 stores representative point data relating to representative points in the virtual scale image obtained during calibration, along with the irradiation position of the spot SP. The representative point data table 66 is created by the calibration method described later. As shown in Figure 39, the representative point data includes coordinate information (X coordinate, Y coordinate) of representative points RP extracted from the circular virtual scale image M, which is the virtual scale image. The representative point data stored in the representative point data table 66 is the data when the positional relationship between the optical axis of the measurement light Lm and the imaging optical system 21 is the default positional relationship.

[0112] When the endoscope 12 is connected to the endoscope connection unit, the table update unit 64 acquires information regarding the positional relationship between the optical axis of the measurement light Lm and the imaging optical system 21, and updates the scale table 62 using the positional relationship and the representative point data table 66. Specifically, it calculates the difference value of the coordinate information of the representative point RP from the difference between the positional relationship between the optical axis of the measurement auxiliary light Lm and the imaging optical system 29b in the endoscope 12 connected to the endoscope connection unit and the default positional relationship. Then, as shown in Figure 40, the table update unit 64 shifts the coordinates of the default representative point RP by the calculated difference value to create a representative point RP. * Based on this, virtual scale image M * To create a virtual scale image, use the representative point RP. * It is preferable to perform interpolation processing to connect the gaps. The resulting virtual scale image after interpolation processing is associated with the irradiation position by the table update unit 64. This completes the update of the scale table 62. In Figure 40, representative points RP, RP * Only a portion of it is labeled with a symbol.

[0113] Regarding the measurement light, the light that is formed as a spot when irradiated onto the subject is used, but other types of light may also be used. For example, when irradiated onto the subject, a planar measurement light that is formed as an intersecting line 67 on the subject may be used, as shown in Figure 41. In this case, a second virtual scale is generated as a virtual scale, consisting of the intersecting line 67 and a scale 68 on the intersecting line 67 that serves as an indicator of the size of the subject (e.g., polyp P). When a planar measurement light is used, the irradiation position detection unit 61 detects the position of the intersecting line 67 (the irradiation position of the measurement light). The lower the intersecting line 67 is located, the closer the observation distance is, and the higher the intersecting line 67 is located, the farther the observation distance is. Therefore, the lower the intersecting line 67 is located, the larger the interval of the scale 68, and the higher the intersecting line 67 is located, the smaller the interval of the scale 68.

[0114] Furthermore, the measurement light may consist of planar light containing at least two first feature lines CL, as shown in Figure 42. When the measurement light is shone on the subject, an intersection curve CC is formed according to the undulations on the subject, and first spots SPk1 are formed on the intersection curve CC at positions corresponding to the two first feature lines CL1. The measurement light also contains multiple second feature lines CL2, which are different from the first feature lines CL1, between the two first feature lines CL1. When the measurement light containing the first feature lines CL1 and the second feature lines CL2 is shone on the subject, second spots SPk2 are formed at positions corresponding to the multiple second feature lines CL2. The second spots SPk2 are smaller than the first spots SPk1, and the spacing between the second spots SPk2 is small. Therefore, a specific intersection curve SCC is formed on the intersection curve by multiple second spots SPk2. Measurement information is calculated based on the position of the specific intersection curve SCC.

[0115] As shown in Figure 43, the signal processing unit 45 of the extended processor device 17 includes a position identification unit 69 and a measurement information processing unit 70 for recognizing the position of the first spot SPk1 or the second spot SPk2 and calculating measurement information. The position identification unit 69 identifies the position of the first spot SPk1 or the second spot SPk2 from the captured image. As a method of identifying the position, for example, the captured image is binarized, and the centroid of the white portion (pixels whose signal intensity is higher than the binarization threshold) in the binarized image is identified as the position of the first spot SPk1 or the second spot SPk2.

[0116] The measurement information processing unit 70 calculates measurement information from the position of the first spot SPk1 or the second spot SPk2. The calculated measurement information is displayed on the captured image by the display control unit 46. When the measurement information is calculated based on the positions of the two first spots SPk1, accurate measurement information can be calculated even when the subject has a three-dimensional shape.

[0117] The measurement information includes a first straight-line distance, which indicates the straight-line distance between two spots, SPk1 and SPk2, as shown in Figure 44. The measurement information processing unit 70 calculates the first straight-line distance in the following way. As shown in Figure 45, the measurement information processing unit 70 determines coordinates (xp1, yp1, zp1) that indicate the actual size at the first spot SPk1 based on its position. xp1 and yp2 are obtained from the coordinates of the position of the first spot SPk1 in the captured image, corresponding to the actual size. zp1 is obtained from the coordinates of the position of the first spot SPk1 and the coordinates of a predetermined specific spot SPk, corresponding to the actual size. Similarly, coordinates (xp2, yp2, zp2) that indicate the actual size at the second spot SPk2 are determined based on its position. xp2 and yp2 are obtained from the coordinates of the position of the second spot SPk2 in the captured image, corresponding to the actual size. zp2 obtains coordinates corresponding to the actual size from the coordinates of the second spot SPk2 and the coordinates of a predetermined specific spot SPk. Then, the first straight-line distance is calculated using the following formula. Formula) 1st straight line distance = ((xp2-xp1) 2 +(yp2-yp1) 2 +(zp2-zp1) 2 ) 0.5 The calculated first straight-line distance is displayed on the captured image as measurement information 71 (shown as "20mm" in Figure 44). The specific spot SPk may or may not be displayed on the extended display 18.

[0118] Furthermore, for the measurement light, multiple spot lights arranged in a grid pattern at predetermined intervals in the vertical and horizontal directions may be used. By imaging tumors tm, etc., within the subject with the grid-arranged spot lights, an image of diffraction spots DS1 is obtained, as shown in Figure 46. The signal processing unit 45 of the extended processor device 17 measures the interval DT of the diffraction spots DS1. The interval DT corresponds to the number of pixels on the imaging plane of the image sensor 32. Note that the interval in a specific part of the multiple diffraction spots DS1 (for example, the interval near the center of the imaging plane) may be measured.

[0119] Once the spacing of the diffraction spots DS1 is measured, the direction and distance to the subject are calculated based on the measurement results. This process uses the relationship between the spacing (number of pixels) of the diffraction spots DS1 and the distance to the subject. Specifically, as shown in Figure 47, the direction (α, β) and distance (r) of the diffraction spots DS1 of the object to be measured are calculated. Once the direction and distance to the subject are calculated, 2D or 3D information of the subject is calculated based on the calculated direction and distance. As 2D or 3D information of the subject, the shape, size, area, etc., in the 2D space (XY plane in Figure 47) or 3D space (XYZ space in Figure 47) of the subject can be calculated. Note that the conversion from (α, β, r) to (X, Y, Z) can be performed using the following equations A), B), and C), and the shape, size, area, etc., can be calculated from the (X, Y, Z) coordinates of each point of the subject. Formula A)X=r×cosα×cosβ Formula B) Y=r×cosα×sinβ Formula C) Z=r×sinα

[0120] As shown in Figure 48, the signal processing unit 45 of the extended processor device 17 includes a position identification unit 72 that identifies the position of spot SP in the first captured image (image based on measurement light and illumination light) in order to recognize the position of the spot and set the virtual scale, and an image processing unit 73 that processes the first captured image or the second captured image (image based on illumination light) based on the position of spot SP to generate a length measurement image.

[0121] The position identification unit 72 includes a noise component removal unit 74 that removes noise components that interfere with the identification of the position of the spot SP. If the first captured image contains a color that is close to the color of the measurement light, even though it is different from the color of the measurement light that forms the spot SP (measurement light approximate color), the position of the spot SP may not be able to be accurately identified. Therefore, the noise component removal unit 74 removes the measurement light approximate color component as a noise component from the first captured image. The position identification unit 72 identifies the position of the spot SP based on the noise-removed first captured image from which the noise components have been removed.

[0122] The noise component removal unit 74 includes a color information conversion unit 75, a binarization processing unit 76, a mask image generation unit 77, and a removal unit 78. The processing flow for obtaining a noise-removed first captured image will be explained using Figure 49. The color information conversion unit 75 converts the first captured image, which is an RGB image, into a first color information image, and converts the second captured image, which is an RGB image, into a second color information image. Preferably, the color information is HSV (H (Hue), S (Saturation), V (Value)). In addition, color difference Cr and Cb may be used as color information.

[0123] The binarization processing unit 76 binarizes the first color information image to obtain a binarized first color information image, and binarizes the second color information image to obtain a binarized second color information image. The threshold for binarization is a binarization threshold that includes the color of the measurement auxiliary light. As shown in Figures 50 and 51, the binarized first color information image includes not only the color information 79 of the measurement light but also the color information 80 of the noise component.

[0124] The mask image generation unit 77 generates a mask image based on the binarized first color information image and the binarized second color information image to remove color information of noise components from the first captured image and to extract color information of the measured light. As shown in Figure 52, the mask image generation unit 77 identifies a region 81 containing noise components from the noise components included in the binarized second captured image. It is preferable that the region 81 of noise components be larger than the region occupied by the color information 80 of the noise components. This is because, in the case of camera shake or the like, the region of color information 80 of the noise components becomes larger compared to the case without camera shake or the like. Then, as shown in Figure 53, the mask image generation unit 77 generates a mask image in which the region of the binarized first color information image containing the color information 79 of the measured light is designated as an extraction region for extracting color information, and the region 81 of noise components is designated as a non-extraction region for not extracting color information. Figures 50 to 53 are schematic representations illustrating the binarized first-color information image, the binarized second-color information image, the noise component region, and the mask image.

[0125] The removal unit 78 extracts color information from the first color information image using a mask image, thereby removing the color information of the noise component and obtaining a noise-reduced first color information image in which the color information of the measured light has been extracted. The noise-reduced first color information image is converted back to an RGB image by performing an RGB conversion process, resulting in a noise-reduced first image. The position identification unit 72 identifies the position of the spot SP based on the noise-reduced first image. Since the noise component has been removed from the noise-reduced second image, the position of the spot SP can be accurately identified.

[0126] The image processing unit 73 includes an image selection unit 82 and a scaling table 62. The image selection unit 82 selects an image to be processed from either the first or second captured image, which is the image to be processed based on the position of the spot SP. The image processing unit 73 performs processing on the image selected as the image to be processed based on the position of the spot SP. The image selection unit 82 selects the image to be processed based on the state of the position of the spot SP. The image selection unit 82 may also be configured to select the image to be processed based on instructions from the user. For example, the user interface 16 can be used for user instructions.

[0127] Specifically, if the spot SP is within a specific range during a specific period, it is assumed that there is little movement in the subject or the tip 12d of the endoscope, so the second image is selected as the image to be processed. In such cases of minimal movement, it is considered that the spot SP can be easily aligned to the lesion within the subject even without it. Furthermore, since the second image does not contain the color component of the measurement light, the color reproduction of the subject is not impaired. On the other hand, if the position of the spot SP is not within a specific range during a specific period, it is assumed that there is significant movement in the subject or the tip 12d of the endoscope, so the first image is selected as the image to be processed. In such cases of significant movement, the user operates the endoscope 12 so that the spot SP is positioned on the lesion. This makes it easier to align the spot SP to the lesion.

[0128] The image processing unit 73 generates a first virtual scale representing the actual size of the subject, based on the position of the spot SP in the first captured image. The image processing unit 73 calculates the size of the virtual scale from the position of the spot SP by referring to a scale table 62 that stores the relationship between the position of the spot SP in the first captured image and the first virtual scale representing the actual size of the subject. Then, the image processing unit 73 generates a first virtual scale corresponding to the size of the virtual scale.

[0129] As shown in Figure 54, the signal processing unit 45 of the expansion processor device 17 includes a first signal processing unit 84 that detects the position of the spot SP in the captured image and a second signal processing unit 85 that sets the virtual scale according to the position of the spot SP, in order to recognize the position of the spot SP and set the virtual scale.

[0130] The first signal processing unit 84 comprises a mask processing unit 86, a binarization processing unit 87, a noise component removal unit 88, and an illumination position detection unit 89. The process of removing noise components in the first signal processing unit 84 will be explained using Figures 55 to 57. The mask processing unit 86 performs a masking process on the red, green, and blue images of the captured image to extract a roughly parallelogram-shaped illumination position movable range Wx that indicates the movable range of the illumination position of the measurement light on the subject. As a result, as shown in Figures 56 and 57, the masked red image PRx, green image PGx, and blue image PBx are obtained from which the illumination position movable range Wx has been extracted. Noise components are removed from pixels within the illumination position movable range, and the illumination position of the spot SP is detected.

[0131] Next, the binarization processing unit 87 applies a first binarization process to the pixels within the movable illumination position range of the red image PRx after masking, thereby obtaining a binarized red image PRy (binarized first spectral image). In the first binarization process, the threshold condition for the first binarization process is set to "1" for pixels with a pixel value of "225" or higher, and to "0" for pixels with a pixel value less than "225". This first binarization process detects the spot SP, which is a component of the measured light. However, in the first binarization process, in addition to the first noise component N1, which is the high-luminance component of the red component of the illumination light, a second noise component N2, which is overexposure (pixel saturation) caused by the illumination light, is also detected. These first and second noise components are factors that hinder the detection of the illumination position of the spot SP. The threshold condition refers to conditions related to the threshold that indicates the boundary between the pixel values ​​of pixels that are set to "0" by binarization and the pixel values ​​of pixels that are set to "1" by binarization, as well as conditions that define the range of pixel values ​​of pixels that are set to "0" by binarization and the range of pixel values ​​of pixels that are set to "1" by binarization.

[0132] Therefore, in order to remove the first noise component, the noise component removal unit 88 performs a first difference processing between the binarized red image PRy and the binarized green image PGy (binarized second spectral image) obtained by binarizing the green image PGx in the second binarization process. The first difference image PD1 obtained by the first difference processing has the first noise component N1 removed. However, in the first difference image PD1, the second noise component N2 often remains and is not removed. For pixels that become "0" or less after the first difference processing, the pixel value is set to "0". In the second binarization process, as a threshold condition for the second binarization process, pixels with a pixel value in the range of "30" to "220" are set to "1", and pixels in other ranges, i.e., from "0" to less than "30" or greater than "220", are set to "0". In addition, although the first noise component is removed by the first difference processing between the binarized red image and the binarized green image, the first noise component may also be removed by other first calculation processing.

[0133] Furthermore, in order to remove the second noise component, as shown in Figure 57, the noise component removal unit 88 performs a second difference process between the first difference image PD1 and the binarized blue image PBy obtained by binarizing the blue image PBx in the third binarization process. The second difference image PD2 obtained by the second difference process has the second noise component, which was difficult to remove by the first difference process, removed. Similar to the first difference process, pixels that become "0" or less after the second difference process have their pixel values ​​set to "0". In the third binarization process, the threshold condition for the third binarization process is set to "1" for pixels with a pixel value of "160" or more, and to "0" for pixels with a pixel value less than "160". In addition, the second noise component is removed by the second difference process between the first difference image and the binarized blue image, but the second noise component may also be removed by other second calculation processes.

[0134] The irradiation position detection unit 89 detects the irradiation position of the spot SP from the first difference image or the second difference image. Preferably, the irradiation position detection unit 89 obtains the centroid coordinates of the spot SP as the irradiation position of the spot SP.

[0135] The second signal processing unit 85 sets a first virtual scale, which represents the actual size of the subject, as a virtual scale, based on the position of the spot SP. The second signal processing unit 85 calculates the size of the virtual scale from the position of the spot by referring to a scale table 62 that stores the relationship between the position of the spot SP and the first virtual scale representing the actual size of the subject. Then, the second signal processing unit 85 sets a first virtual scale that corresponds to the size of the virtual scale.

[0136] As shown in Figure 58, the signal processing unit 45 of the extended processor device 17 comprises an illumination area recognition unit 90 and a second signal processing unit 60. The illumination area recognition unit 90 recognizes a measurement light illumination area having a specific shaped pattern from the captured image. Specifically, as shown in Figure 59, the specific shaped pattern includes a white central area CR1 and a peripheral area SR1 that surrounds the central area and has feature quantities based on the measurement light. If the measurement light illumination area is the spot SP described above, the specific shaped pattern has a circular shape. In this case, the white central area CR1 is circular, and the peripheral area SR1 is ring-shaped.

[0137] Figure 60 shows the distribution of pixel values ​​for each color image in an captured image that includes multiple color images: a red image RP, a green image GP, and a blue image BP. In the central region CR1, the pixel values ​​of the red image RP, green image GP, and blue image BP have reached their maximum pixel values ​​(e.g., 255), making the central region CR1 appear white. In this case, when measurement light is incident on the image sensor 32, as shown in Figure 61, in the wavelength range WMB of the measurement light, the image sensor 32 transmits the measurement light not only through its red color filter RF, but also through the maximum transmittance of its green color filter GF and blue color filter BF. On the other hand, in the peripheral region SR1, the pixel values ​​of the red image RP are greater than those of the green image GP or blue image BP. Therefore, the peripheral region SR1 appears reddish. The light source 23a emits measurement light at a specific intensity, thereby setting the pixel values ​​of the red image RP, green image GP, and blue image BP in the central region CR1 to their maximum pixel values.

[0138] The illumination area recognition unit 90 is capable of recognizing spot SP having the specific shape and features described above. Specifically, as shown in Figure 62, it is preferable that the illumination area recognition unit 90 has a learning model 91 that recognizes spot SP by outputting spot SP, which is the measurement light illumination area, in response to the input of an captured image. The learning model 91 is machine-learned using a large amount of training data that associates the captured image with already recognized measurement light illumination areas. It is preferable to use a CNN (Convolutional Neural Network) for machine learning.

[0139] By using the learning model 91 to recognize spot SPs, it is possible to recognize not only circular spot SPs (see Figure 59) composed of a circular central region CR1 and a ring-shaped peripheral region SR1, but also spot SPs with specific shapes, such as patterns deformed from a circular shape. For example, as shown in Figure 63(A), spot SPs deformed in the vertical direction can also be recognized. Furthermore, as shown in Figure 63(B), spot SPs that are deformed with a part of the circular shape missing can also be recognized. In addition, the features of the peripheral region SR1 that can be recognized by the learning model 91 include red, which is the color of the measured light, as well as blue and green. Furthermore, the features of the peripheral region SR1 that can be recognized by the learning model 91 include the luminance, brightness, saturation, and hue of the measured light. It is preferable to obtain the luminance, brightness, saturation, and hue of the measured light by performing a luminance conversion process or a brightness, saturation, and hue conversion process on the peripheral region of the spot SP included in the captured image.

[0140] As shown in Figure 64, the signal processing unit 45 of the extended processor device 17 includes a position identification unit 92 that identifies the position of the spot SP in the captured image and calculates the observation distance in order to recognize the position of the spot SP, calculate the observation distance to the subject, and set a virtual scale, and an image processing unit 93 that sets various virtual scales based on the observation distance and generates a measured length image by processing the captured image using the various virtual scales.

[0141] The position identification unit 92 includes a distance calculation unit 94. Based on the captured image in which the subject is illuminated by illumination light and measurement light, the position identification unit 92 identifies the position of the spot SP formed on the subject by the measurement light. The distance calculation unit 94 determines the observation distance from the position of the spot SP.

[0142] The image processing unit 93 includes an image selection unit 95, a scale table 62, an offset setting unit 97, an offset distance calculation unit 98, and an offset virtual scale generation unit 99. The image selection unit 95 selects the image to be processed based on the position of the spot SP. The offset setting unit 97 sets an offset amount corresponding to the height of the spot SP of the convex polyp 100 relative to the observation distance. The offset distance calculation unit 98 calculates the offset distance by adding the offset amount to the observation distance. The offset virtual scale generation unit 99 generates a virtual scale for offset based on the offset distance.

[0143] The following explains the concept of offset. First, a convex shape of a subject refers to a shape that protrudes from its surroundings. Therefore, it is sufficient if at least a part of the shape protrudes from its surroundings; other aspects of the shape, such as size, width, height and / or number of protruding parts, and continuity of height, are not considered.

[0144] More specifically, as shown in Figure 65, for example, the subject has a polyp 100 as a convex shape. The polyp 100 has a shape that protrudes from the surrounding subject. The polyp 100 has a top portion 100a and a flat portion 100b. Figure 65 shows the polyp 100 as viewed from the horizontal direction when the height direction of the polyp 100 is the vertical direction, but since the polyp 100 is three-dimensional, it also exists in the direction towards the front of the paper and the direction towards the depth of the paper. The plane formed by the flat portion of the subject surrounding the polyp 100, where the polyp 100 is formed, is defined as the flat portion 100b of the polyp 100. The plane formed by the flat portion of the subject surrounding the polyp 100 is the extension surface 101 of the flat portion 100b.

[0145] Next, the height of the spot SP of the convex polyp 100 will be explained. In this embodiment, the height of the spot SP of the polyp 100 is the vertical distance from the spot SP of the polyp 100 to the flat portion 100b of the polyp 100. More specifically, as shown in Figure 66, the spot SP1 is formed on the top 100a of the polyp 100. Therefore, if the plane that is parallel to the extension plane 101 and passes through the top 100a of the polyp 100 is defined as the parallel plane 102, then the distance between the parallel plane 102 and the extension plane 101 becomes the height HT1 of the spot SP of the polyp 100 (the vertical distance from the top 100a to the flat portion 100b of the polyp 100). Note that polyp 100 and height HT1 (and height HT2 below) are shown schematically, and the type, shape, and size of the convex shape are not specified, as long as it is a part that protrudes from the surrounding area.

[0146] Furthermore, in Figure 66, spot SP2 is formed in a region of polyp 100 that is not the top 100a. That is, it is formed between the top 100a and the end of polyp 100. Therefore, if we define the parallel plane 103 as the plane that is parallel to the extension plane 101 and passes through spot SP2 of polyp 100, then the distance between the parallel plane 103 and the extension plane 101 is the height HT2 of spot SP2 of polyp 100. Thus, the height HT2 of spot SP2 of polyp 100 is the vertical distance from spot SP2 to the flat portion 100b of polyp 100.

[0147] The observation distance and offset amount are explained below. As shown in Figure 67, the measurement light Lm causes spot SP1 to form on the top 100a of polyp 100. The observation distance obtained from spot SP1 is the distance D5 between the position P1 of the tip 12d of the endoscope and the position P2 of spot SP1. The virtual scale corresponding to distance D5 is a virtual scale that matches the actual measurement on the parallel plane 102. Therefore, when spot SP1 is formed on the top 100a of polyp 100, when a virtual scale corresponding to distance D5 is generated and displayed, a virtual scale that matches the actual measurement of the subject on the parallel plane 102 is displayed. Consequently, a virtual scale is displayed that is shifted to a larger scale relative to the actual measurement value of the subject on the extension plane 101.

[0148] Therefore, the offset setting unit 97 sets the height HT1 of the spot SP of polyp 100 as the offset amount for the observation distance D5. Next, the offset distance calculation unit 98 calculates the offset distance D6 by adding the height HT1 of the spot SP1 of polyp 100, which is the offset amount, to the observation distance D5. Thus, the offset distance calculation unit 98 calculates the offset distance D6 using the following formula OS). Note that HT1 is the distance between positions P2 and P3. Formula OS)D6=D5+HT1

[0149] Next, the offset virtual scale generation unit 99 generates a virtual scale based on the observation distance D6 as the offset virtual scale. More specifically, the offset virtual scale generation unit 99 refers to the scale table 62 and uses the virtual scale when the observation distance is distance D6, and uses this as the offset virtual scale. The offset virtual scale will represent the actual distance or size of the object on the extended plane 101.

[0150] The image processing unit 93 generates a measured length image by superimposing the generated virtual scale for offset onto the captured image. For more accurate measurement, it is preferable to superimpose the virtual scale for offset so that it is displayed at the position where the spot SP is formed. Therefore, if it is to be displayed at a position far from the spot SP, it should be displayed as close to the spot SP as possible. The measured length image with the superimposed virtual scale for offset is displayed on the extended display 18 by the display control unit 46.

[0151] As shown in Figure 68, the number of pixels for the width of the lines constituting the virtual scale may be set to be larger as the observation distance decreases, and smaller as the observation distance increases.

[0152] In the example shown in Figure 68(A), when the observation distance is 21 mm or more at the far end Px in range Rx, the line width W11 that constitutes the virtual scale M11 is set to the smallest setting value of 1 pixel. In the example shown in Figure 68(B), when the observation distance is 13 mm to 20 mm between the far end Pz and the central Py in range Rx, the line width W12 that constitutes the virtual scale M12 is set to 2 pixels. In the example shown in Figure 68(C), when the observation distance is 8 to 12 mm at the central Py in range Rx, the line width W13 that constitutes the virtual scale M13 is set to the intermediate value of 3 pixels.

[0153] In the example shown in Figure 68(D), when the observation distance is 4-7 mm and the observation distance is between the central Py and the nearest Px in the range Rx, the line width W14 that constitutes the virtual scale M14 is set to 4 pixels. In the example shown in Figure 68(E), when the observation distance is 3 mm or less at the nearest Px in the range Rx, the line width W15 that constitutes the virtual scale M15 is set to the largest possible value, 5 pixels.

[0154] As described above, the lines that make up the virtual scale M11-M15 are changed according to the observation distance, making it easier for the user (a physician) to measure the accurate dimensions of the subject. Furthermore, the widths W11-W15 of the lines that make up the virtual scale M11-M15 are set to values ​​inversely proportional to the observation distance, so it is possible to recognize the magnitude of the dimensional error from the width of the lines. For example, if the tumor tm is located inside the line that makes up the virtual scale M11-M15, taking the recognized error into account, it can be seen that it is definitely smaller than the set actual size (within 5 mm in the example shown in Figure 68).

[0155] Furthermore, as shown in Figure 69, a concentric virtual scale M2 may be set based on the position of one spot SP, consisting of three concentric circles of different sizes. The three concentric circles M21, M22, and M23 that make up the virtual scale M2 represent actual sizes such as "5mm", "10mm", and "20mm". Also, for illustrative purposes, the lines that make up the circular shape of the concentric circles M21, M22, and M23 that make up the virtual scale M2 are cross-hatched, but in reality, each line is filled with a single color.

[0156] The width W22 of the concentric circle M22, which is one position outside the innermost concentric circle M21, is set to be larger than the width W21 of concentric circle M21, and the width W23 of the outermost concentric circle M23 is set to be larger than the width W22 of concentric circle M22. In the example shown in Figure 69, the width W23 is set to a value that is √2 times the width W22 and twice the width W21.

[0157] The ratio of widths W21 to W23 (in the example shown in Figure 69, the ratio is W21:W22:W23 = 1:√2:2) is maintained while setting a value inversely proportional to the observation distance. This makes it possible to recognize the magnitude of the dimensional error from the line width. For example, if the tumor tm is inside the lines that make up the concentric circle M22 and outside the lines that make up the concentric circle M23, taking the recognized error into account, it can be reliably determined that it is within the set actual size range (in the example shown in Figure 69, between 10 mm and 20 mm).

[0158] As shown in Figure 70, a gradient may be applied to the lines constituting the virtual scale M3, where the density gradually decreases from the center of the line in the width direction outwards. Then, as in the embodiment described above, the width of the gradient lines is changed according to the observation distance.

[0159] As shown in Figure 71, the lines constituting the virtual scales M41 to M43 are represented by dashed lines, and the gaps constituting the dashed lines are set to a value inversely proportional to the observation distance. In Figure 71, the circular virtual scales M41, M42, and M43 are shown when images are taken at the far end Pz, near the center Py, and near end Px within the observation distance range Rx.

[0160] In the example shown in Figure 71(A), for the far end Pz in the range Rx, the gap G1 of the dashed lines constituting the virtual scale M41 is set to the smallest value. Note that in the example shown in Figure 71(A), the virtual scale M41 is composed of dashed lines, but only for the far end Pz, the gap G1 may be set to 0, i.e., the virtual scale M41 may be composed of solid lines. In the example shown in Figure 71(B), for the central Py in the range Rx, the gap G2 of the dashed lines constituting the virtual scale M42 is set to the midpoint value. In the example shown in Figure 71(C), for the near end Px in the range Rx, the gap G3 of the dashed lines constituting the virtual scale M43 is set to the largest value.

[0161] As described above, the gaps G1 to G3 in the dashed lines that make up the virtual scales M41 to M43 are set to values ​​inversely proportional to the observation distance, making it possible to recognize the magnitude of dimensional errors from the gaps in the dashed lines.

[0162] The number of lines constituting the virtual scale is the same regardless of the observation distance. As shown in Figure 72, the number of lines constituting the virtual scales M51 to M53 is changed according to the observation distance. In Figure 72, the virtual scales M51, M52, and M53 are shown when images are taken at the far end Pz, near the center Py, and near end Px within the observation distance range Rx.

[0163] In the example shown in Figure 72(A), for the far end Pz in range Rx, the virtual scale M51 is composed of three lines, i.e., three concentric circles of different sizes. The three concentric circles represent actual sizes such as "5mm", "10mm", and "20mm". In the example shown in Figure 72(B), for the central Py in range Rx, the virtual scale M52 is composed of two lines, i.e., two concentric circles of different sizes. The two concentric circles represent actual sizes such as "5mm" and "10mm". In the example shown in Figure 72(C), for the near end Px in range Rx, the virtual scale M53 is composed of one line, i.e., one circular shape. The one circular shape represents an actual size such as "5mm". As described above, the number of lines that make up the virtual scales M51 to M53 is set to a value proportional to the observation distance, so it is possible to recognize the magnitude of the dimensional error from the number of lines.

[0164] As shown in Figure 73, the signal processing unit 45 of the extended processor device 17 includes a position identification unit 92 including a distance calculation unit 94 and an image processing unit 104. The image processing unit 104 includes an image selection unit 95, a scale table 62, a virtual scale setting unit 105, a virtual scale switching reception unit 106, and a length measurement image creation unit 107. The virtual scale setting unit 105 sets a virtual scale that represents the actual size of the object being observed on the subject according to the position of the spot SP and has a scale with the edges as the base point. The virtual scale switching reception unit 106 receives instructions to switch and set multiple virtual scales. The length measurement image creation unit 107 creates a length measurement image by superimposing the virtual scale set by the virtual scale setting unit 105 onto the captured image so that the position of the spot SP and the base point of the virtual scale scale overlap.

[0165] The functions of the virtual scale setting unit 105 and the length measurement image creation unit 107 will be described below. As shown in Figure 74, an image 109 in which the subject including the polyp 108 is illuminated is input to the signal processing unit 45. For example, since the polyp 108 has a spherical three-dimensional shape, the image 109 includes the polyp 108, the spot SP, and possibly the shadow 110.

[0166] Based on the captured image 109 input to the signal processing unit 45, the position identification unit 92 identifies the position of spot SP. The virtual scale setting unit 105 refers to the scale table 62 and sets a virtual scale that represents the actual size of the object being observed, corresponding to the position of spot SP, and has markings that start from the edges. The edges are parts of the virtual scale that are closer to the outer part than the central part, or the start or end point, etc.

[0167] As shown in Figure 75, the length measurement image creation unit 107 creates an image image 109 by superimposing a virtual scale 111, set by the virtual scale setting unit 105, onto the captured image 109, such that the position of the spot SP and the base point of the scale of the virtual scale 111 coincide. For more accurate measurement, it is preferable to superimpose the virtual scale 111 so that it is displayed at the position of the spot SP. Therefore, even when it is displayed at a position far from the spot SP, it is preferable to display it as close to the spot SP as possible. The virtual scale 111 is a straight line segment, and has scales at the start and end points of the line segment that are line segments perpendicular to the straight line segment. If the virtual scale 111 is a line segment or the like and has a start and an end point, the start and / or end point itself may be used as scales, in which case, for example, there may be no scales in the shape of line segments perpendicular to the straight line segment. The virtual scale 111 may also have the number "10" near the base end of the scale. The number "10" is the scale label 111a of the virtual scale 111, and is added to make it easy to recognize that the line segment of the virtual scale 111 is 10 mm in actual size. The numbers on the virtual scale below have the same meaning. The numerical value of the scale label 111a can be changed by setting, and the virtual scale 111 may not display the scale label 111a at all.

[0168] The virtual scale can use various types depending on the settings. For example, it can use shapes such as straight line segments or combinations of straight line segments, circles or combinations of circles, or combinations of straight line segments and circles.

[0169] As shown in Figure 76, for example, the captured image 113 includes a virtual scale 112 whose shape is a combination of straight line segments. The virtual scale 112 has a shape formed by combining straight line segments in an L-shape, with the corners of the L-shape serving as the base point, and the line segments extending in the direction of the paper and in the direction of the paper, with the base point serving as the starting point and each ending point having a scale. In addition, the virtual scale 112, like the virtual scale 111, has a numerical value of "10", which is the scale label 112a, near the base point of the scale.

[0170] As shown in Figure 77, for example, the captured image 114 includes a virtual scale 115 whose shape is a combination of a straight line segment and a circle. The virtual scale 115 is a shape that combines a circle and a line segment which is the diameter of this circle, and the intersections of the line segment and the circle are the respective scale divisions. A scale division 116 may be located at the point that divides the line segment in half or at the center of the circle. In addition, the virtual scale 115, like the virtual scale 111 or virtual scale 112, has the number "10", which is the scale label 116a, near the base point of the scale division. The scale label 116b represents half of the scale label 116a.

[0171] As shown in Figure 78, virtual scales can take on various shapes other than those shown, such as virtual scale 117 (Figure 78(A)) which includes a scale label 117a with a line segment extending to the left of the paper from the base point, virtual scale 118 (Figure 78(B)) which includes a scale label 118a with a line segment extending downwards from the base point, or virtual scale 119 (Figure 78(C)) which includes a scale label 119a with a line segment extending diagonally upwards to the right of the paper from the base point.

[0172] As shown in Figure 79, the signal processing unit 45 of the extended processor device 17 includes a position identification unit 92, a reference scale setting unit 120, a measurement value scale generation unit 121, and a length measurement image generation unit 122. The reference scale setting unit 120 sets a reference scale that indicates the actual size of the subject based on the position of the spot SP. The measurement value scale generation unit 121 generates a measurement value scale that indicates the measurement value of the measurement portion of the area of ​​interest based on the set reference scale. Note that the reference scale and the measurement value scale are virtual and displayed on the captured image, and therefore correspond to virtual scales.

[0173] The area of ​​interest is the region within the subject that the user should focus on. For example, an area of ​​interest might be a polyp, or another area that is highly likely to require measurement. The measurement area is the part of the area of ​​interest where length or other measurements are taken. For example, if the area of ​​interest is a reddened area, the measurement area would be the longest part of the reddened area; if the area of ​​interest is circular, the measurement area would be the diameter of the area.

[0174] The length measurement image generation unit 122 creates a length measurement image by superimposing a measurement value scale onto the captured image. The measurement value scale is superimposed onto the captured image in a manner that aligns with the measurement portion of the area of ​​interest. The length measurement image is displayed on the extended display 18.

[0175] As shown in Figure 80, the reference scale setting unit 120 includes a reference scale table 121a. The reference scale table 121a is correspondence information that associates the position of the spot SP with measurement information corresponding to the actual size of the subject. In length measurement mode, an image 114 in which the subject, including the polyp 123 which is the object of observation, is captured is input to the signal processing unit 45. As shown in Figure 81, in the image 124, the polyp 123 has a three-dimensional shape, for example, like overlapping spheres. For example, a spot SP is formed at the end of the polyp 123. Based on the image 124, the position identification unit 92 identifies the position of the spot SP. The reference scale setting unit 120 refers to the reference scale table 121a and sets a reference scale 131 that indicates the actual size of the subject, corresponding to the position of the identified spot SP.

[0176] The reference scale 131 is, for example, a line segment with a number of pixels corresponding to 20 mm in actual size, as well as a numerical value and unit indicating the actual size. The reference scale 131 is not normally displayed on the extended display 18, but when the reference scale 131 is displayed on the extended display 18, it is displayed as in the captured image 124.

[0177] As shown in Figure 82, the measurement value scale generation unit 121 includes a focus area extraction unit 125, a measurement portion determination unit 126, a measurement content reception unit 127, and a measurement value calculation unit 128. As shown in Figure 83, the focus area extraction unit 125 extracts a hatched area as the focus area 129, as shown in the captured image 124. Next, as shown in Figure 84, if the preset reference is, for example, the reference for measuring the portion of the focus area in the horizontal direction with the spot SP as the base point, the measurement portion determination unit 126 extracts the horizontal edge position 130 with the spot SP as the base point, as shown in the captured image 124. The area between the spot SP and the horizontal edge position 130 becomes the measurement portion.

[0178] The measurement value calculation unit 128 generates the measurement value scale 132 such that it satisfies the following equation (K1), for example, when the actual size of the reference scale is L0, the number of pixels of the reference scale 131 in the captured image 124 is Aa, the number of pixels of the measurement portion when the reference scale 131 is superimposed on the area of ​​interest 129 in the captured image 124 is Ba, and the actual size of the measurement value scale 132 is L1. Formula (K1) L1=L0×Ba / Aa

[0179] As shown in Figure 85, the measurement value calculation unit 128 uses the number of pixels Aa corresponding to the reference scale 131 shown in the captured image 124a and the number of pixels Bb corresponding to the measurement portion between the spot SP and the horizontal edge position 130 shown in the captured image 124b. For example, if Ba / Aa is 0.7, and the actual size of the reference scale 131 is 20 mm, then the unit calculates the actual size of the measurement value scale 132 as 13 mm, as shown in the captured image 124d.

[0180] The length measurement image generation unit 122 generates a length measurement image 133 by superimposing a measurement value scale 132 onto the captured image 124. For example, as shown in Figure 86, the measurement value scale 132 is superimposed on the captured image 124 in the form of a figure such as an arrow, which is the shape of a straight line segment. The length measurement image 133 may also include the numerical value of the actual size of the measurement value scale 132. The numerical value of the actual size of the measurement value scale 132 may be superimposed on the captured image 124 separately from the figure such as the arrow.

[0181] Multiple types of measurement scales 132 can be selected. The measurement content receiving unit 127 receives the setting of the content of the measurement scale and sends that content to the measurement scale generation unit 121. The measurement image generation unit 122 then generates a measurement scale 132 based on that content and uses it to generate a measurement image 133.

[0182] Furthermore, it is preferable that the focus region extraction unit 125 extracts the focus region using a trained model that has been trained on previously acquired captured images. Various models suitable for image recognition by machine learning can be used as the trained model. For the purpose of recognizing the focus region on an image, a model using a neural network is preferably used. When training these models, captured images containing information about the focus region are used as training data. Information about the focus region includes the presence or absence of the focus region, the location or range of the focus region, etc. Depending on the model, it may also be trained using captured images that do not contain information about the focus region.

[0183] Furthermore, it is preferable that the measurement area determination unit 126 also determines the measurement area using a trained model that has been learned from previously acquired captured images. The models used for the trained model are the same as those used for the region of interest extraction unit, but when training these models, they are trained using captured images that contain information about the measurement area. Information about the measurement area includes the measured value and the measured portion. Depending on the model, it may also be trained using captured images that do not contain information about the measurement area. The trained model used by the region of interest extraction unit 125 and the trained model used by the measurement area determination unit 126 may be the same. If the purpose is to extract the measurement area, a single trained model may be used to extract the measurement area from the captured image 124 without extracting the region of interest.

[0184] In the second signal processing unit 60, the scale table 62 for displaying a virtual scale deformed to match the position of the spot SP is updated from the representative point data table 66, which stores the irradiation position of the measurement light and the representative point of the virtual scale (see Figures 39 and 40). However, the scale table 62 may be created by other methods. For example, from the image obtained when imaging a square grid chart, a distorted grid region QN enclosing a circular virtual scale centered on the spot SP is obtained, as shown in Figure 87. In the distorted grid region QN, the grid is distorted due to the distortion aberration of the imaging optical system 21 as it moves away from the center of the screen. The distorted grid region QN is transformed into a square grid region SQ as shown in Figure 88 using an affine transformation matrix. In the square grid region SQ, the coordinates of the points representing the circular virtual scale are calculated. Then, the coordinates of the points of the virtual scale in the square grid region SQ are transformed into a distorted circular virtual scale distorted by the imaging optical system 21 using the inverse matrix of the affine transformation matrix. The coordinates of this distorted circular virtual scale are associated with the position of spot SP and stored in the scale table 62.

[0185] Furthermore, considering the distortion aberration of the imaging optical system 21, the display mode of the virtual scale may be changed between areas where measurement using the virtual scale is effective and areas where it is not. Specifically, as shown in Figure 89(a), when the spot SP is outside the effective measurement area (near end Px side), and as shown in Figure 89(c), when the spot SP is outside the effective measurement area (far end Pz side), measurement of tumor tm using the virtual scale is not effective, so the cross-shaped virtual scales MN and MF are displayed, respectively. On the other hand, as shown in Figure 89(b), when the spot SP is in an area where measurement using the circular virtual scale M is effective, the circular virtual scale M is displayed.

[0186] Furthermore, the line type of the virtual scale may be changed depending on whether the spot SP is within or outside the effective measurement area. In this case, it is preferable to display the movement trajectory MT of the spot SP, as shown in Figure 90, so that the change in the line type of the virtual scale can be seen. As shown in Figure 90(a), when the spot SP is outside the effective measurement area (near end Px side), and as shown in Figure 90(c), when the spot SP is outside the effective measurement area (far end Pz side), measurement of tumor tm using the virtual scale is not effective, so the circular virtual scales MpN and MpF are displayed as dotted lines, respectively. On the other hand, as shown in Figure 90(b), when the spot SP is in an area where measurement using the circular virtual scale Mp is effective, the circular virtual scale Mp is displayed as a solid line. Note that the line type of the virtual scale is changed between dotted and solid lines depending on whether the spot SP is inside or outside the effective measurement area, but different colors may also be used. For example, the line type of the virtual scale may be blue when the spot SP is outside the effective measurement area, and white when it is inside.

[0187] Details of still image acquisition in length measurement mode are described below. When no still image acquisition instruction has been given, the system control unit 41 controls the light source device 13 to emit illumination light and measurement light. As shown in Figure 91, when the still image acquisition instruction switch 12g is operated and a still image acquisition instruction is given, at the first timing including the still image acquisition instruction, the illumination light is turned on while the measurement light is turned off. At the second and third timings after the first timing has elapsed, the illumination light remains on and the measurement light is turned on again. Note that the second and third timings are the same, but they may be different.

[0188] At the first timing, a second image is obtained by capturing a subject illuminated by measurement light. At the second and third timings, a first image is obtained by capturing a subject illuminated by illumination light and measurement light. The system control unit 41 then saves the still images of the first and second images as saved images to be stored in the still image storage unit 42, as shown in Figures 91 and 92. The signal processing unit 45 of the extended processor device 17 acquires a still image of the third image, displaying a virtual scale Mxm set according to the position of the spot SP relative to the first image. The still image of the third image is sent to the processor device 14 and stored in the still image storage unit 42. For a certain period of time after the still images are saved, the display control unit 46 displays the second and third images on the extended display 18, as shown in Figure 93, to indicate that still images are being recorded. It is preferable that at least two of the first, second, and third images are saved in the still image storage unit 42 with a single still image acquisition instruction. For example, it is preferable to save two images: a second image and a third image. The third image corresponds to the image to be saved in the still image storage unit 42 from among the measurement images with the virtual scale superimposed, as described above.

[0189] As shown in Figure 94, the second or third timing may occur before the first timing. In this case, during length measurement mode, it is necessary to store several frames of the first captured images corresponding to the second or third timing in the temporary storage unit (not shown) of the processor device 14. When a still image acquisition instruction is given, the first captured images stored in the temporary storage unit are stored in the still image storage unit 42 as the first captured images of the second timing, and a third captured image obtained by adding a virtual scale to the first captured images stored in the temporary storage unit is also stored in the still image storage unit 42.

[0190] Furthermore, as shown in Figure 95, the second and third timings may be different. In this case, the first captured image obtained at the second timing is saved to the still image storage unit 42 in the same manner as described above. The first captured image obtained at the third timing is converted into a third captured image by adding a virtual scale to it before being saved to the still image storage unit 42.

[0191] As shown in Figure 96, the signal processing unit 45 of the expansion processor device 17 may also include a lesion recognition unit 135, a diagnostic information acquisition unit 136, and a learning unit 137, in addition to the first signal processing unit 59 and the second signal processing unit 60. The lesion recognition unit 135 processes the first captured image (an image based on illumination light and measurement light) and performs recognition processing. The recognition processing performed by the lesion recognition unit 135 involves detection processing to detect regions of interest, such as lesions, from the first captured image. It is preferable to use a machine learning model for the recognition processing. That is, in response to the input of the first captured image to the learning model, the learning model outputs the detection result of the region of interest. The learning model is preferably a machine learning model such as a Convolutional Neural Network (CNN). The recognition processing performed by the lesion recognition unit 135 may also be a differentiation process to differentiate the progression of the lesion, etc., based on the lesion recognized from the first captured image. Furthermore, the lesion recognition unit 135 may perform recognition processing by image processing on the second captured image (an image based solely on illumination light).

[0192] The diagnostic information acquisition unit 136 acquires diagnostic information relating to the first or second image from the diagnostic information management device 138. As diagnostic information, it acquires the medical record of the patient being examined. A medical record is information that records the progress of medical treatment or examinations for a patient, and includes, for example, the patient's name, gender and age, disease name, major symptoms, prescription or treatment details, or medical history. The lesion information recognized and processed by the lesion recognition unit 135, and the diagnostic information relating to the first or second image acquired by the diagnostic information acquisition unit 136, are stored in the still image storage unit 42 as supplementary data to the dataset DS, associated with the first or second image.

[0193] The learning unit 137 performs machine learning using the first or second captured image stored in the still image storage unit 42 and the associated data (dataset) for these first and second captured images. Specifically, the learning unit 137 trains the learning model of the lesion recognition unit 135 using machine learning. The second captured image is preferable as a candidate for training data for machine learning. The second captured image is obtained by a still image acquisition instruction during the measurement of tumor™, etc., and is therefore highly likely to contain the region of interest to be observed. Furthermore, since the second captured image is a normal endoscopic image without irradiation by measurement light, it is highly useful as training data for machine learning. In addition, since information on the lesion and diagnostic information are included as accompanying data, the user does not need to input this information when performing machine learning. As more second captured images are accumulated as candidate training data and machine learning is performed, the accuracy of the recognition processing in the lesion recognition unit 135 improves. When using the first captured image as a candidate for training data for machine learning, the first captured image may be used as is, but it is more preferable to use the part other than the area irradiated by the measurement light as the candidate training data.

[0194] The calibration method for creating a representative point data table 66 using the calibration device 200 shown in Figure 97 will be described below. The calibration device 200 comprises a calibration display 201, a moving mechanism 202, a calibration display control unit 204, a calibration image acquisition unit 206, and a calibration unit 208. The calibration display control unit 204, the calibration image acquisition unit 206, and the calibration unit 208 are provided in the calibration image processing device 210. The calibration image processing device 210 is electrically connected to the processor device 14, the calibration display 201, and the moving mechanism 202.

[0195] The movement mechanism 202 has a holding part (not shown) that holds the tip 12d of the endoscope 12 toward the calibration display 201, and changes the distance Z between the tip 12d of the endoscope 10 and the calibration display 201 by moving the holding part at specific intervals. Each time the distance Z is changed by the movement mechanism 202, the calibration display control unit 204 displays a virtual scale image in a first display mode on the calibration display 201 with respect to the irradiation position of the measurement light, without being affected by the imaging optical system 21. The virtual scale image in the first display mode does not take into account the effects of distortion etc. caused by the imaging optical system 21, and therefore is not displayed in a size or shape corresponding to the scale display position when displayed on the extended display 18.

[0196] The calibration image acquisition unit 206 acquires calibration images obtained by imaging the virtual scale of the first display mode displayed on the calibration display 201 using the endoscope 12. Calibration images are acquired by the endoscope 12 imaging each time the distance Z is changed, that is, each time the virtual scale of the first display mode is displayed. For example, if the virtual scale of the first display mode is displayed n times, n calibration images are obtained.

[0197] The calibration image includes a virtual scale image in a second display mode, which is affected by the imaging optical system 21 with respect to the irradiation position of the measurement light. The virtual scale image in the second display mode is displayed with a size or shape corresponding to the scale display position, taking into account the effects of distortion etc. caused by the imaging optical system 21.

[0198] The calibration unit 208 performs calibration for the display of the virtual scale on the extended display 18 based on the calibration image acquired by the calibration image acquisition unit 206. Specifically, the calibration unit 208 performs a representative point extraction process to extract representative points from the virtual scale image of the second display mode included in the calibration image, and a table creation process to create a representative point data table by associating representative point data with the irradiation position at the timing when the calibration image was acquired. The created representative point data table is sent to the extended processor device 17 and stored in the representative point data table 66.

[0199] As shown in Figure 98, the inspection system 300 is used for accuracy inspection of a scale, such as whether the virtual scale has a predetermined shape. The inspection system 300 includes a test chart 302, a display 15, and a moving mechanism 304. The display 15 is shared with the endoscope system 10, but a separate display for accuracy inspection may be provided.

[0200] As shown in Figure 99, the test chart 302 has a chart body 305, which is provided with an inspection area section 306 having an inspection area of ​​a specific shape, and an inspection reference position 308 that serves as a reference for aligning the irradiation position of the measurement light during accuracy inspection. The inspection area section 306 has three circular inspection areas 306a, 306b, and 306c as inspection areas of a specific shape. These three inspection areas 306a, 306b, and 306c are provided concentrically with respect to the inspection reference position 308. The inspection areas 306a, 306b, and 306c are illuminated by measurement light (e.g., spot SP) from the endoscope 12 used for confirmation inspection of a virtual scale of 5 mm (indicating a diameter of "5 mm"), a virtual scale of 10 mm (indicating a diameter of "10 mm"), and a virtual scale of 20 mm (indicating a diameter of "20 mm"), respectively, and an inspection image is acquired by imaging.

[0201] As shown in Figure 100, the inspection image is displayed on the display 15. In addition to the inspection area 306 and the inspection reference position 308, the inspection image displays a virtual scale M corresponding to the irradiation position of the measurement light (position of spot SP). During accuracy inspection, the moving mechanism 304 moves the test chart 302 so that the irradiation position of the measurement light (position of spot SP) matches the inspection reference position. When the irradiation position of the measurement light matches the inspection reference position, the user determines whether the virtual scale M is displayed correctly.

[0202] For example, in an inspection image, if the irradiation position of the measurement light is at the inspection reference position 308, and the virtual scale M is within the inspection area 306, the user will determine that the virtual scale M is displayed correctly. Conversely, as shown in Figure 101, if even a part of the virtual scale M is outside the inspection area 306, or if any part of the virtual scale M is not within the inspection area 306, the user will determine that the 5mm virtual scale M is not displayed correctly.

[0203] The scale table 62 may be created as follows. The relationship between the spot position and the size of the virtual scale can be obtained by imaging a chart in which a regular pattern of actual size is formed. For example, a spot-shaped measurement light is emitted towards the chart, and while changing the observation distance to change the spot position, a grid-like chart with the same grid size as actual size (5 mm) or finer grids (e.g., 1 mm) is imaged, and the relationship between the spot position (pixel coordinates on the imaging surface of the image sensor 32) and the number of pixels corresponding to the actual size (how many pixels represent the actual size of 5 mm) is obtained.

[0204] As shown in Figure 102, (x1, y1) represents the pixel positions in the X and Y directions of spot SP4 on the imaging surface of the image sensor 32 (the upper left is the origin of the coordinate system). At the position of spot SP4 (x1, y1), Lx1 is the number of pixels in the X direction corresponding to the actual size of 5 mm, and Ly1 is the number of pixels in the Y direction. This measurement is repeated while changing the observation distance. Figure 103 shows the same 5 mm grid chart as in Figure 102, but the shooting distance is closer to the far end than in Figure 102, and the grid lines appear closer together. In the state of Figure 103, Lx2 is the number of pixels in the X direction corresponding to the actual size of 5 mm at the position of spot SP5 (x2, y2) on the imaging surface of the image sensor 32, and Ly2 is the number of pixels in the Y direction. Then, the measurement as shown in Figures 102 and 103 is repeated while changing the observation distance, and the results are plotted. Note that in Figures 102 and 103, distortion aberration of the imaging optical system 21 is not taken into consideration.

[0205] Figure 104 shows the relationship between the X-coordinate of the spot's position and Lx (the number of pixels in the X direction of the first virtual scale), and Figure 105 shows the relationship between the Y-coordinate of the spot's position and Lx. From the relationship in Figure 104, Lx can be expressed as a function of the X-direction position as Lx = g1(x), and from the relationship in Figure 105, Ly can be expressed as a function of the Y-direction position as Ly = g2(y). g1 and g2 can be determined from the plot results described above, for example, by the least squares method.

[0206] Note that the X and Y coordinates of the spot correspond one-to-one, and using either function g1 or g2 will yield essentially the same result (the same number of pixels for the same spot position). Therefore, when calculating the size of the first virtual scale, either function can be used, or the function that is more sensitive to changes in the number of pixels in response to changes in position can be selected. Also, if the values ​​of g1 and g2 differ significantly, it may be judged that "the spot position could not be recognized."

[0207] Figure 106 shows the relationship between the X coordinate of the spot position and Ly (number of pixels in the Y direction), and Figure 107 shows the relationship between the Y coordinate of the spot position and Ly. From the relationship in Figure 106, Ly can be expressed as the coordinate of the X direction position as Ly = h1(x), and from the relationship in Figure 108, Ly can be expressed as the coordinate of the Y direction position as Ly = h2(y). Similar to Lx, either the function h1 or h2 can be used for Ly.

[0208] The functions g1, g2, h1, and h2 obtained as described above are stored in the scaling table 62 in lookup table format. Alternatively, functions g1 and g2 may be stored in the scaling table 62 in function format.

[0209] Furthermore, regarding the measurement light, a striped pattern light ZPL may be used, which, when irradiated onto the subject, forms a striped pattern of light on the subject as shown in Figure 108 (see, for example, Japanese Patent Publication No. 2016-198304). The striped pattern light ZPL is obtained by irradiating a liquid crystal shutter (not shown) with variable transmittance with a specific laser beam, and is formed from two different vertical stripe patterns in which regions that transmit the specific laser beam (transmitted regions) and regions that do not transmit the specific laser beam (non-transmitted regions) are periodically repeated in the horizontal direction by the liquid crystal shutter. When using striped pattern light as the measurement light, since the period of the striped pattern light changes depending on the distance to the subject, the period or phase of the striped pattern light is shifted by the liquid crystal shutter and irradiated multiple times, and the three-dimensional shape of the subject is measured based on multiple images obtained by shifting the period or phase.

[0210] For example, the subject is irradiated alternately with striped pattern light of phase X, striped pattern light of phase Y, and striped pattern light of phase Z. The striped pattern lights of phases X, Y, and Z have a vertical stripe pattern phase-shifted by 120° (2π / 3) each. In this case, the three-dimensional shape of the subject is measured using three types of images obtained based on each striped pattern light. For example, as shown in FIG. 109, it is preferable to irradiate the subject by switching the striped pattern light of phase X, the striped pattern light of phase Y, and the striped pattern light of phase Z in units of one frame (or several frames). Note that it is preferable to irradiate the subject with illumination light at all times.

[0211] Regarding the measurement light, when irradiated on the subject, as shown in FIG. 110, measurement light LPL of a grid pattern formed as a grid pattern may be used (see, for example, Japanese Patent Application Laid-Open No. 2017-217215). In this case, since the three-dimensional shape of the subject is measured based on the deformed state of the grid pattern when the measurement light LPL of the grid pattern is irradiated on the subject, it is required to accurately detect the grid pattern. Therefore, the measurement light LPL of the grid pattern is not a complete grid shape, but is slightly deformed from the grid shape, such as being made wavy, so as to improve the detection accuracy of the grid pattern. In addition, the grid pattern is provided with an S code indicating that the endpoints of the left and right horizontal lines are continuous. When detecting the grid pattern, the detection accuracy of the pattern is improved by detecting not only the pattern but also the S code. Note that the grid pattern may be a pattern in which vertical and horizontal lines are regularly arranged, or a pattern in which a plurality of spots are arranged in a grid pattern vertically and horizontally.

[0212] When using the measurement light LPL of a grid pattern as the measurement light, during the length measurement mode, the illumination light and the measurement light LPL of the grid pattern may be constantly irradiated on the subject. Also, as shown in FIG. 111, while the illumination light is constantly irradiated on the subject, the measurement light LPL of the grid pattern may be intermittently irradiated on the subject by repeating lighting and extinguishing (or dimming) every one frame (or every several frames). In this case, in the frame where the measurement light LPL of the grid pattern is lit, the three-dimensional shape is measured based on the measurement light LPL of the grid pattern. And it is preferable to superimpose and display the measurement result of the three-dimensional shape on the image obtained in the frame where only the illumination light is irradiated.

[0213] Regarding the measurement light, as shown in FIG. 112, a three-dimensional planar light TPL represented by ruling lines on the subject image may be used (see, for example, Japanese Patent Application Laid-Open No. 2017-508529). In this case, the tip 12d is moved so that the three-dimensional planar light TPL matches the measurement object. And when the three-dimensional planar light TPL intersects the measurement object, the distance of the intersection curve CC between the three-dimensional planar light TPL and the subject is calculated by a process based on a manual operation such as a user interface or an automatic process.

[0214] When using the three-dimensional planar light TPL as the measurement light, during the length measurement mode, the illumination light and the three-dimensional planar light TPL may be constantly irradiated on the subject. Also, as shown in FIG. 113, while the illumination light is constantly irradiated on the subject, the three-dimensional planar light TPL may be intermittently irradiated on the subject by repeating lighting and extinguishing (or dimming) every one frame (or every several frames).

[0215] In the above embodiment, the hardware structure of the processing unit that performs various processes, such as the receiving unit 38, signal processing unit 39, display control unit 40, system control unit 41, still image storage unit 42, data transmission / reception unit 43, data transmission / reception unit 44, signal processing unit 45, and display control unit 46 (including various control units or processing units provided in these control units, etc. (for example, length measurement mode control unit 50, first signal processing unit 59, etc.)), is a type of processor as shown below. The types of processors include a CPU (Central Processing Unit), which is a general-purpose processor that executes software (programs) and functions as various processing units; a Programmable Logic Device (PLD), which is a processor whose circuit configuration can be changed after manufacturing, such as an FPGA (Field Programmable Gate Array); and a dedicated electrical circuit, which is a processor with a circuit configuration specifically designed to perform various processes.

[0216] A single processing unit may be composed of one of these various processors, or it may be composed of a combination of two or more processors of the same or different types (for example, multiple FPGAs, or a combination of a CPU and an FPGA). Alternatively, multiple processing units may be composed of a single processor. Examples of composing multiple processing units with a single processor include, firstly, a configuration where one or more CPUs and software are combined to form a single processor, and this processor functions as multiple processing units, as is typical of computers such as client and server systems. Secondly, a configuration using a processor that realizes the functions of the entire system, including multiple processing units, on a single IC (Integrated Circuit) chip, as is typical of System-on-a-Chip (SoC) systems. Thus, various processing units are configured, in terms of hardware structure, using one or more of the above-mentioned various processors.

[0217] Furthermore, the hardware structure of these various processors is, more specifically, an electrical circuit formed by combining circuit elements such as semiconductor devices. The hardware structure of the memory unit is a storage device such as an HDD (hard disk drive) or SSD (solid state drive). [Explanation of Symbols]

[0218] 10 Endoscopy Systems 12 Endoscopes 12a Insertion section 12b Operation section 12c curved section 12d Tip 12f Observation Mode Switch 12g still image acquisition instruction switch 12h Zoom control section 13 Light source device 14 Processor Unit 15 displays 16 User Interface 17. Expansion Processor Unit 18-inch extended display 18a Additional information display area 18b Observation image display area 19 Balloons 19a Tip 19b Proximal end 19c bulge 20a, 20b rings 21 Imaging optical system 21a Objective lens 21b Zoom Lens 21c Tip surface 22 Illumination optical system 22a Illumination lens 22b Tip surface 23 Measurement light output part 23a light source 23b DOE 23c prism 24 Aperture 25 Air and water supply nozzles 25a Injection cylinder part 25b injection port 26 intestinal tract 27 tip cap 27a, 27b, 27c, 27d through holes 28 tip surface 28a, 28b planes 30 light source unit 31 light source processor 32 imaging device 33 imaging control unit 34 CDS / AGC circuit 35 A / D converter 36 communication I / F 37 communication I / F 38 receiving unit 39 signal processing unit 40 display control unit 41 system control unit 42 still image storage unit 43 data transmission / reception unit 44 data transmission / reception unit<​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​72 Location identification part 73 Image Processing Department 74. Noise component removal section 75 Color Information Conversion Unit 76. Binarization Processing Unit 77 Mask Image Generation Unit 78 Removal section 79. Color information of the measured light 80 Color information of noise components 81. Noise component region 82 Image Selection Section Table for 83 scale 84 First Signal Processing Unit 85 Second Signal Processing Unit 86 Mask Processing Unit 87. Binarization Processing Unit 88 Noise component removal section 89 Irradiation position detection unit 90 Irradiation area recognition unit 91 Learning Models 92 Location identification part 93 Image Processing Department 94 Distance Calculation Unit 95 Image Selection Section 97 Offset setting section 98 Offset distance calculation unit 99. Virtual scale generation unit for offset 100 polyps 100a top 100b flat area 101 Extension surface 101X Solid line 102, 103 Parallel surfaces 102X dotted line 104 Image Processing Department 105 Virtual Scale Setting Section 106 Virtual Scale Switching Reception Unit 107 Measurement Image Creation Unit 108 polyps 109 Acquired Images 110 Shadow 111, 112 Virtual Scale 111a Measuring label 113, 114 Acquired images 115 virtual scale 116 divisions 116a Measuring label 116b Measuring Labels 118 Virtual Scale 118a Measuring label 119 Virtual scaling 119a Measuring label 120 Reference scale setting section 121 Measurement Value Scale Generation Unit 121a Table for reference scale 122 Measurement Image Generation Unit 123 Polyps 124 captured images 125 Attention area extraction part 126 Measurement part determination unit 127 Measurement Details Reception Department 128 Measurement Value Calculation Unit 129 Areas of Interest 130 Horizontal edge position 131 Reference Scale 132 Measurement scale 133 Measurement image 135 Lesion Recognition Section 136 Diagnostic Information Acquisition Department 137 Learning Department 138 Diagnostic Information Management Device 140 Length Measurement Compatible Endoscope Feasibility Determination Unit 141 Measurement light ON / OFF switching section 142 Measurement Image Display Setting ON / OFF Switch 143 Measurement function operation status display ON / OFF switch 144 Virtual Scale Display Switching Control Unit 146 Scale display icon 147 Virtual Scale 147a, 147b, 147c virtual scale 148 Scale hidden icon 149 Image display settings save section before switching 200 Calibration Devices 201 Calibration Display 202 Moving mechanism 204 Calibration Display Control Unit 206 Calib Image Acquisition Unit 208 Calibration section 210 Calib Image Processing Equipment 300 Inspection Systems 302 Test Chart 304 Moving mechanism section 305 Chart Body 306 Inspection Area Department 306a, 306b, 306c Examination Area 308 Inspection reference position Aa, Ba Pixel count Ax optical axis BLC balloon control device BF Blue Color Filter CL1 First Feature Line CL2 Second Feature Line CC intersection curve CR1 White center region D1 1st direction D2 2nd direction D3 Third direction D5 distance D6 Offset distance DS1 diffraction spot DT interval EP measurement point G1, G2, G3 gaps GF Green Color Filter HT1, HT2 Height LG Light Guide Ls, Lt Line Lm measurement light Lx1, Lx2 X-direction pixel count Ly1, Ly2 Y-direction pixel count LPL lattice pattern measurement light M circular virtual scale M1, M2, M3 Cross-shaped virtual scaling M11, M12, M13, M14, M15 virtual scale M21, M22, M23 concentric circles M41, M42, M43 Virtual Scale M4A, M4B, M4C, M5A, M5B, M5C Concentric Markers M5A, M5B, M5C Virtual Scale M6A, M6B, M6C Distorted Concentric Virtual Scale MN, MF Cross Virtual Scale MpN, Mp, MpF Circular Virtual Scale MT Movement Trajectory Mx Scale Mxm Virtual Scale N1 First Noise Component N2 Second Noise Component P Polyp P1, P2, P3 Positions Px Proximal End Py Near the Center Pz Distal End RP, PRx Red Image PRy Binary Red Image GP, PGx Green Image PGy Binary Green Image BP, PBx Blue Image PBy Binary Blue Image PD1 First Difference Image Qx, Qy, Qz Arrows QN Distorted Grid Region RP, RP * Representative Point RF Red Color Filter SCC Specific Intersection Curve SP Spot SP1, SP2, SP3, SP4, SP5 Spots SPk1 First Spot SPk2 Second Spot SQ Square Grid Region SR1 Peripheral Region tm, tm1, tm2, tm3, tm4, tm5 Tumors TPL 3D Planar Light W11, W12, W13, W14, W15 Widths W21, W22, W23 Widths Wx Movable Range of Irradiation Position WMB measurement wavelength range ZPL striped pattern light

Claims

1. Endoscope and, The processor device to which the endoscope is connected, Equipped with a processor, The aforementioned processor, When the endoscope is connected to the processor device, it is determined whether the endoscope is a length-measuring endoscope. If the endoscope is a length-measuring endoscope, the switch to a length-measuring mode is enabled, in which measurement light is emitted from the endoscope and a virtual scale based on the measurement light is displayed on the display. When the observation mode is set to special observation mode, the system controls the illumination of a special light used to highlight a specific area of ​​the observation target, and the extinguishing of the measurement light. In the special observation mode, the red image used for detecting the measurement light in the length measurement mode is not used for image display. An endoscope system that, when the special observation mode is set, prohibits switching to the length measurement mode if such an operation is performed.

2. The aforementioned processor, In the special observation mode, the endoscope system according to claim 1 displays a pseudo-color image on the display in which the colors of the subject image are assigned to different color channels.

3. The endoscope system according to claim 1 or 2, wherein when set to the length measurement mode, the endoscope emits illumination light and measurement light different from the special light.

4. The aforementioned processor, The endoscope system according to any one of claims 1 to 3, wherein a scale display icon is displayed on the display in accordance with the operation of switching to the length measurement mode.

5. The aforementioned processor, The endoscope system according to any one of claims 1 to 4, wherein, in the operation to switch to the measurement mode, if the conditions for switching modes are not met, an indicator showing that the measurement function is not operational is turned ON to indicate that the virtual scale is not being displayed.

6. The endoscope system according to claim 5, wherein the mode switching conditions are setting conditions related to the endoscope, the processor device, and the processor, and are conditions used for executing the length measurement mode.

7. In the length measurement mode, the display mode of the virtual scale is changed by selecting from a plurality of scale patterns, as described in any one of claims 1 to 6.

8. The aforementioned processor, The endoscope system according to any one of claims 1 to 7, wherein, when the switching to the measurement mode is enabled, the operation to switch to the measurement mode includes at least one of the following: switching the measurement light ON or OFF, switching the measurement image display setting ON or OFF for the measurement image that displays the virtual scale, switching the measurement function operation status display ON or OFF to indicate that the virtual scale is being displayed on the display, and switching the display of the virtual scale ON, OFF, or changing the display mode.

9. The aforementioned processor, The endoscope system according to claim 8, wherein switching to the length measurement mode switches the measurement light ON, the length measurement image display setting ON, the length measurement function operation status display ON, and the virtual scale display ON.

10. The aforementioned processor, The endoscope system according to claim 9, wherein, in the operation to switch to the length measurement mode, if the conditions for switching modes are not met, the operation to switch the measurement light to ON, the length measurement image display setting to ON, the length measurement function operation status display to ON, and the display of the virtual scale to ON is prohibited.

11. The endoscope system according to claim 10, wherein instead of prohibiting switching the measurement function operation status display to ON, an indication that the virtual scale is not being displayed is turned ON, indicating that the measurement function operation status is unavailable.

12. Endoscope and, The processor device to which the endoscope is connected, Equipped with a processor, The aforementioned processor, When the endoscope is connected to the processor device, it is determined whether the endoscope is a length-measuring endoscope. If the endoscope is a length-measuring endoscope, the switch to a length-measuring mode is enabled, in which measurement light is emitted from the endoscope and a virtual scale based on the measurement light is displayed on the display. When the observation mode is set to special observation mode, the system controls the illumination of a special light used to highlight a specific area of ​​the observation target, and the extinguishing of the measurement light. The system switches the ON or OFF display of the measurement function operation status, which indicates that the virtual scale is being displayed on the display. By switching to the aforementioned length measurement mode, the measurement light is turned ON, the length measurement image display setting for the length measurement image displaying the virtual scale is turned ON, the length measurement function operation status display is turned ON, and the display of the virtual scale is turned ON. The aforementioned processor, When the aforementioned length measurement image display setting is turned ON, the image display setting before switching to the length measurement mode is saved. The measurement function operation status display is a scale display icon that is displayed in a different location from the virtual scale. The aforementioned scale display icon is shown when switching to the length measurement mode and disappears when switching from the length measurement mode to another mode in the endoscope system.

13. The endoscope system according to claim 11, wherein the display mode of the virtual scale is changed by selection from a plurality of scale patterns.

14. The aforementioned processor, The endoscope system according to claim 8, wherein switching from the length measurement mode to another mode turns off the measurement light, the length measurement image display setting, the length measurement function operation status display, and the virtual scale display.

15. Endoscope and, The processor device to which the endoscope is connected, Equipped with a processor, The aforementioned processor, When the endoscope is connected to the processor device, it is determined whether the endoscope is a length-measuring endoscope. If the endoscope is a length-measuring endoscope, the switch to a length-measuring mode is enabled, in which measurement light is emitted from the endoscope and a virtual scale based on the measurement light is displayed on the display. When the observation mode is set to special observation mode, the system controls the illumination of a special light used to highlight a specific area of ​​the observation target, and the extinguishing of the measurement light. The system switches the ON or OFF display of the measurement function operation status, which indicates that the virtual scale is being displayed on the display. The aforementioned processor, To turn off the measurement image display setting for the measurement image that displays the virtual scale, switch to the image display setting saved before switching to the measurement mode. The measurement function operation status display is a scale display icon that is displayed in a different location from the virtual scale. The aforementioned scale display icon is shown when switching to the length measurement mode and disappears when switching from the length measurement mode to another mode in the endoscope system.

16. In an endoscope, a processor device to which the endoscope is connected, and a method for operating an endoscope system comprising the processor, The aforementioned processor, When the endoscope is connected to the processor device, it is determined whether the endoscope is a length-measuring endoscope. If the endoscope is a length-measuring endoscope, the switch to a length-measuring mode is enabled, in which measurement light is emitted from the endoscope and a virtual scale based on the measurement light is displayed on the display. When the observation mode is set to special observation mode, a special light used to highlight a specific area of ​​the observation target is turned on, and the measurement light is turned off. In the special observation mode, the red image used for detecting the measurement light in the length measurement mode is not used for image display. An operating method for an endoscope system that, when the special observation mode is set, prohibits switching to the length measurement mode if such an operation is performed.

Images