Endoscope system and method of operation thereof

Abstract

This invention provides an endoscope system and its operating method that can determine whether a length measurement mode can be performed based on the connection of an endoscope. The endoscope system (10) includes an endoscope (12) and a processor device (14) having a system control unit (41) composed of an image control processor. When the endoscope (12) is connected to the processor device (14), the system control unit (41) determines whether the endoscope (12) is a length measurement-compatible endoscope, and if the endoscope (12) is a length measurement-compatible endoscope, enables the switching to the length measurement mode.

Description

Technical Field

[0001] This invention relates to an endoscope system that displays a virtual scale for measuring the size of a subject and its operating method. Background Technology

[0002] In an endoscopic system comprising a light source, an endoscope, and a processor, information such as the distance to the subject or the size of the subject is obtained. For example, in Patent Document 1, illumination light and measuring light are shone onto the subject, and the subject is made into a focused area of ​​measuring light by the illumination of the measuring light. Furthermore, a virtual scale for measuring the size of the subject is displayed on the image corresponding to the position of the focused light.

[0003] Previous technical documents

[0004] Patent documents

[0005] Patent Document 1: International Publication No. 2018 / 051680 Summary of the Invention

[0006] The technical problem to be solved by the invention

[0007] As in Patent Document 1, when using a measuring light to display a virtual scale, a length-measuring endoscope is needed that can illuminate the measuring light. In an endoscope system, in order to execute the length-measuring mode that displays a virtual scale, it is necessary to determine whether a length-measuring endoscope is connected.

[0008] The purpose of this invention is to provide an endoscope system and its working method that can determine whether a length measurement mode can be performed by connecting an endoscope.

[0009] means for solving technical problems

[0010] The endoscope system of the present invention includes: an endoscope; and a processor device having an image control processor, wherein when the endoscope is connected to the processor device, the image control processor determines whether the endoscope is a length-measuring endoscope, and when the endoscope is a length-measuring endoscope, enables the switching to the length-measuring mode.

[0011] Preferably, when the endoscope is a length-measuring endoscope, the endoscope can illuminate the measuring light and display a length-measuring image based on a virtual scale of the measuring light on the display. When the switching to the length-measuring mode is enabled, the image control processor performs at least one of the following switching operations through the length-measuring mode switching operation: switching the measuring light on or off, switching the length-measuring image display setting related to the length-measuring image on or off, switching the display indicating the length-measuring function operation status of the virtual scale on the display on or off, and switching the display of the virtual scale on or off or changing the display mode.

[0012] Preferably, the image control processor switches the measurement light to on, the length measurement image display setting to on, the length measurement function operation status display to on, and the virtual scale display to on during the length measurement mode switching operation. Preferably, during the length measurement mode switching operation, the image control processor prevents the measurement light from being switched to on, the length measurement image display setting from being switched to on, the length measurement function operation status display from being switched to on, and the virtual scale display from being switched to on if the mode switching conditions are not met. Preferably, setting the option indicating that the virtual scale is not a displayable length measurement function operation status is not displayed is enabled instead of disabling the length measurement function operation status display. Preferably, when the length measurement image display setting is enabled, the image control processor saves the image display settings from before the length measurement mode switching.

[0013] Preferably, the display mode of the virtual scale is changed by selecting from multiple scale patterns. Preferably, the image control processor switches the measurement light off, the length measurement image display setting off, the length measurement function operation status display off, and the virtual scale display off by switching from the length measurement mode to other modes. Preferably, when the length measurement image display setting is off, the image control processor switches to the image display setting saved before switching to the length measurement mode.

[0014] In the operation method of the endoscope system of the present invention, the endoscope system includes: an endoscope; and a processor device having an image control processor. In the operation method of the endoscope system, when the endoscope is connected to the processor device, the image control processor determines whether the endoscope is a length-measuring endoscope. If the endoscope is a length-measuring endoscope, the switching to the length-measuring mode is enabled.

[0015] Invention Effects

[0016] According to the present invention, it is possible to determine whether a length measurement mode can be performed by connecting an endoscope. Attached Figure Description

[0017] Figure 1 This is a schematic diagram of an endoscope system.

[0018] Figure 2 This is a 3D diagram of the balloon.

[0019] Figure 3 This is the front view of the balloon.

[0020] Figure 4 This is an illustration of a balloon representing the contracted state of the intestines.

[0021] Figure 5 This is an explanatory diagram showing the distended state of a balloon inside the intestine.

[0022] Figure 6 This is the front view of the endoscope.

[0023] Figure 7 This is a three-dimensional view of the front end of an endoscope.

[0024] Figure 8 It is a block diagram representing the function of an endoscope system.

[0025] Figure 9 This is an image representing the state of digital zoom being off (A) and on (B).

[0026] Figure 10 This is a schematic diagram showing the light emission section for measurement.

[0027] Figure 11 It is a cross-sectional view of the front end of an endoscope with a measuring light emission section.

[0028] Figure 12 It is a plan view showing the transparent cover.

[0029] Figure 13 This is a schematic diagram showing the direction of light travel in measurement.

[0030] Figure 14 It is a block diagram representing the functions of the system control unit.

[0031] Figure 15 It is an image that displays icons with or without a scale.

[0032] Figure 16 It is a block diagram representing the functions of the system control unit.

[0033] Figure 17 This is an explanatory diagram representing the first control.

[0034] Figure 18 It is an image representing the message displayed when the first control is in effect.

[0035] Figure 19 This is an explanatory diagram indicating that the special observation mode has been deactivated and the measurement mode has been switched.

[0036] Figure 20 This is an explanatory diagram representing the second control.

[0037] Figure 21 This is an image representing the message displayed when the second control is in effect.

[0038] Figure 22 This is an explanatory diagram representing the third control.

[0039] Figure 23 This is an image representing the message displayed when third-level control is in effect.

[0040] Figure 24 It is a block diagram representing the functions of the system control unit.

[0041] Figure 25 This is an explanatory diagram showing the first light emission control table.

[0042] Figure 26 This is an explanatory diagram representing coordinate regions 1 to 5.

[0043] Figure 27 This is an explanatory diagram showing the second light emission control table.

[0044] Figure 28 This is an explanatory diagram showing the light emission control in the length measurement mode.

[0045] Figure 29 This is an explanatory diagram showing the light emission and camera control of the first pattern in the length measurement mode.

[0046] Figure 30 This is an explanatory diagram showing the light emission and camera control of the second pattern in the length measurement mode.

[0047] Figure 31 It is a block diagram representing the functions of the signal processing unit.

[0048] Figure 32 It is an image diagram that represents a virtual scale displayed in the case of the near end Px.

[0049] Figure 33 This is an image representing a virtual scale displayed near the center of Py.

[0050] Figure 34 It is an image diagram that represents a virtual scale displayed at the remote point Pz.

[0051] Figure 35 It is an explanatory diagram that represents a virtual scale for various shapes.

[0052] Figure 36 It is an explanatory diagram representing virtual scales of different sizes.

[0053] Figure 37 It is an explanatory diagram representing virtual scales with different colors.

[0054] Figure 38 It is an explanatory diagram representing a virtual scale representing distorted concentric circles.

[0055] Figure 39 This is an explanatory diagram representing point data.

[0056] Figure 40 This is an explanatory diagram illustrating the processing of the table update section.

[0057] Figure 41 It is an image that represents a virtual scale displayed when illuminated by a planar measuring light.

[0058] Figure 42 This is an illustration of planar light including two first characteristic lines.

[0059] Figure 43 This is an explanatory diagram showing the functions of the signal processing unit.

[0060] Figure 44 It is an image diagram showing the image displayed when illuminated by planar light including two first feature lines.

[0061] Figure 45 This is an explanatory diagram showing the direction of travel of planar light when it is irradiated with planar light including two first characteristic lines.

[0062] Figure 46 This is an explanatory diagram showing the diffraction spot.

[0063] Figure 47 This is an explanatory diagram related to the calculation methods of two-dimensional and three-dimensional information of the subject when using diffraction spots.

[0064] Figure 48 It is a block diagram representing the functions of the signal processing unit.

[0065] Figure 49 This is an explanatory diagram illustrating the processing used to obtain the first camera image after interference removal.

[0066] Figure 50 This is an explanatory diagram representing the binarized first color information image.

[0067] Figure 51 This is an explanatory diagram representing the binarized second color information image.

[0068] Figure 52 This is an explanatory diagram representing the binarized second color information image.

[0069] Figure 53 This is an explanatory diagram representing the binarized first color information image.

[0070] Figure 54 It is a block diagram representing the functions of the signal processing unit.

[0071] Figure 55 This is an illustrative diagram showing the process used to obtain the first difference image or the second difference image.

[0072] Figure 56 This is an explanatory diagram representing the first difference process.

[0073] Figure 57 This is an explanatory diagram illustrating the second difference process.

[0074] Figure 58 It is a block diagram representing the functions of the signal processing unit.

[0075] Figure 59 This is a schematic diagram showing a light spot that includes a white central area and a surrounding area.

[0076] Figure 60 It is a graph that represents the distribution of pixel values ​​of various images in a camera image.

[0077] Figure 61 It is a graph showing the correlation between the transmission distribution of each color filter and the wavelength range of the measured light.

[0078] Figure 62 This is a block diagram representing the function of the illumination area recognition unit.

[0079] Figure 63 This is an illustrative diagram showing an example of a pattern of light spots deformed from a circle.

[0080] Figure 64 It is a block diagram representing the functions of the signal processing unit.

[0081] Figure 65 This is a schematic diagram representing a convex polyp.

[0082] Figure 66 This is an explanatory diagram showing the height of the light spot.

[0083] Figure 67 This is an explanatory diagram related to the calculation of offset distance D6.

[0084] Figure 68 It is a schematic diagram of a virtual scale with different line widths.

[0085] Figure 69This is a schematic diagram representing a virtual scale with concentric circles.

[0086] Figure 70 This is a schematic diagram of a virtual scale with grayscale added to the lines.

[0087] Figure 71 It is a schematic diagram of a virtual scale with varying gaps between the dotted lines.

[0088] Figure 72 It is a schematic diagram of a virtual scale with varying numbers of lines.

[0089] Figure 73 It is a block diagram representing the functions of the signal processing unit.

[0090] Figure 74 It is an image showing the spot of light shining on the periphery of the polyp.

[0091] Figure 75 This is an explanatory diagram showing a virtual scale that aligns the base point with the position of the light spot.

[0092] Figure 76 This is an explanatory diagram showing two virtual scales aligned with the position of the base and the light spot.

[0093] Figure 77 This is an explanatory diagram showing a virtual scale of circles and line segments aligned with the position of the base point and the light spot.

[0094] Figure 78 This is an explanatory diagram showing a virtual scale that aligns the base point with the position of the light spot.

[0095] Figure 79 It is a block diagram representing the functions of the signal processing unit.

[0096] Figure 80 This is a block diagram illustrating the functions of the reference scale setting unit.

[0097] Figure 81 It is an image representing a virtual scale superimposed on the polyp.

[0098] Figure 82 This is a block diagram representing the function of the measurement scale generation unit.

[0099] Figure 83 It is an image representing the region of interest.

[0100] Figure 84 This is an explanatory diagram showing the measurement section.

[0101] Figure 85 It is an image graph related to the scale of the measured values.

[0102] Figure 86It is an image representing a scale of measurements superimposed on the polyp.

[0103] Figure 87 This is an explanatory diagram showing the distorted mesh region.

[0104] Figure 88 This is an explanatory diagram representing a square grid area.

[0105] Figure 89 This is an illustrative diagram showing examples of virtual scales displaying different shapes both within and outside the effective measurement area.

[0106] Figure 90 This is an explanatory diagram showing an example of a virtual scale displaying different line types within and outside the effective measurement area.

[0107] Figure 91 This is an explanatory diagram related to still image acquisition in length measurement mode.

[0108] Figure 92 This is an explanatory diagram showing the first to third camera images.

[0109] Figure 93 This is an image diagram representing the second and third camera images displayed when capturing still images.

[0110] Figure 94 This is an explanatory diagram related to still image acquisition in length measurement mode.

[0111] Figure 95 This is an explanatory diagram related to still image acquisition in length measurement mode.

[0112] Figure 96 It is a block diagram representing the functions of the signal processing unit.

[0113] Figure 97 This is a block diagram representing the function of the calibration device.

[0114] Figure 98 This is a schematic diagram of the inspection system.

[0115] Figure 99 It is a plan view representing the test chart.

[0116] Figure 100 It is an image that represents the inspection reference position, light spot, and virtual scale displayed on the inspection image.

[0117] Figure 101 It is an image that represents the inspection reference position, light spot, and virtual scale displayed on the inspection image.

[0118] Figure 102 It is a chart image with 5mm grid lines.

[0119] Figure 103 It is a chart image with 5mm grid lines (in comparison) Figure 102 (Shooting from a more distant side).

[0120] Figure 104 This is an explanatory diagram showing the pixel position of the light spot in the X direction.

[0121] Figure 105 This is an explanatory diagram showing the pixel position of the light spot in the Y direction.

[0122] Figure 106 This is an explanatory diagram showing the pixel position of the light spot in the X direction.

[0123] Figure 107 This is an explanatory diagram showing the pixel position of the light spot in the Y direction.

[0124] Figure 108 This is an explanatory diagram representing the ZPL (Zero-Plastic Particle Light) pattern.

[0125] Figure 109 This is an explanatory diagram showing the emission pattern of ZPL, a striped pattern light representing phases X, Y, and Z.

[0126] Figure 110 This is an explanatory diagram representing the LPL (Limited Particulate Light) measuring beam with a grid pattern.

[0127] Figure 111 This is an explanatory diagram showing the luminescence pattern of measuring light intermittently illuminating a grid-like pattern.

[0128] Figure 112 This is an explanatory diagram representing the three-dimensional planar light TPL.

[0129] Figure 113 This is an explanatory diagram showing the luminescent pattern of a three-dimensional planar light TPL that is intermittently irradiated. Detailed Implementation

[0130] like Figure 1 As shown, the endoscope system 10 includes an endoscope 12, a light source device 13, a processor device 14, a display 15, a user interface 16, an extended processor device 17, and an extended display 18. The endoscope 12 is optically connected to the light source device 13 and electrically connected to the processor device 14. The endoscope 12 has an insertion portion 12a that is inserted into the body of the object being observed, an operating portion 12b provided at the base of the insertion portion 12a, a bending portion 12c provided at the front end of the insertion portion 12a, and a front end portion 12d. The bending portion 12c is bent by operating the operating portion 12b. The front end portion 12d is oriented in a desired direction by bending the bending portion 12c.

[0131] Furthermore, the operation unit 12b is equipped with an observation mode switching switch 12f used in the observation mode switching operation, a still image acquisition command switch 12g used in the still image acquisition command of the observed object, and a zoom operation unit 12h used in the operation of the zoom lens 21b.

[0132] The processor device 14 is electrically connected to the display 15 and the user interface 16. The display 15 outputs images or information about the observed object processed by the processor device 14. The user interface 16 has a keyboard, mouse, touchpad, microphone, etc., and has input operation functions such as receiving function settings. The extended processor device 17 is electrically connected to the processor device 14. The extended display 18 outputs images or information processed by the extended processor device 17.

[0133] The endoscope 12 has a normal observation mode, a special observation mode, and a length measurement mode, which can be switched via an observation mode switch 12f. The normal observation mode illuminates the object being observed with illumination light. The special observation mode illuminates the object with a special light different from the illumination light. In the length measurement mode, the object is illuminated with either illumination light or measuring light, and a virtual scale for measuring the size of the object is displayed on the subject image obtained by photographing the object. The subject image without the virtual scale is displayed on the monitor 15, while the subject image with the virtual scale superimposed is displayed on the extended monitor 18.

[0134] Additionally, the illumination light is used to impart brightness to the entire object being observed. The special light is used to emphasize specific areas within the object being observed. The measurement light is used to display a virtual scale. Furthermore, in this embodiment, the virtual scale displayed on the image is described; however, an actual scale can be installed within the actual lumen to allow confirmation of the actual scale through the image. In this case, it is possible to insert the actual scale through the forceps channel of the endoscope 12, allowing the actual scale to protrude from the front end 12d.

[0135] When the user operates the still image acquisition command switch 12g, the screen of the display 15 freezes and a warning sound (e.g., "beep-") indicating that still image acquisition is in progress is emitted. Then, still images of the subject acquired before and after the timing of the operation of the still image acquisition command switch 12g are stored in the still image storage unit 42 within the processor device 14 (see reference). Figure 8Additionally, the still image storage unit 42 can be a storage unit such as a hard disk or a USB (Universal Serial Bus) memory. If the processor device 14 is capable of being connected to a network, still images of the subject can be stored on a still image storage server (not shown) connected to the network, in addition to or in addition to the still image storage unit 42.

[0136] Alternatively, still image acquisition commands can be performed using an operating device other than the still image acquisition command switch 12g. For example, a foot pedal can be connected to the processor device 14, and a still image acquisition command can be performed when the user operates the foot pedal (not shown). Mode switching can also be performed using the foot pedal. Furthermore, a gesture recognition unit (not shown) that recognizes the user's gestures can be connected to the processor device 14, and a still image acquisition command can be performed when the gesture recognition unit recognizes a specific gesture performed by the user. Mode switching can also be performed using the gesture recognition unit.

[0137] Furthermore, a gaze input unit (not shown) located near the display 15 can be connected to the processor device 14, and a still image acquisition command can be issued when the gaze input unit detects that the user's gaze has lingered on a predetermined area on the display 15 for a certain period of time. Also, a voice recognition unit (not shown) can be connected to the processor device 14, and a still image acquisition command can be issued when the voice recognition unit recognizes a specific voice spoken by the user. Mode switching can also be performed using the voice recognition unit. Furthermore, an operation panel (not shown) such as a touch panel can be connected to the processor device 14, and a still image acquisition command can be issued when the user performs a specific operation on the operation panel. Mode switching can also be performed using the operation panel.

[0138] like Figure 2 As shown, the front end 12d includes a camera optical system 21 for receiving light from the subject, an illumination optical system 22 for illuminating the subject with illumination light, a measurement light emitting section 23 for emitting measurement light to the subject in length measurement mode, an opening 24 for protruding the treatment device toward the subject, and an air and water delivery nozzle 25 for delivering air and water.

[0139] A balloon 19, serving as a fixing component, is detachably installed in the insertion section 12a. The balloon 19 is a disposable type, discarded after one or fewer uses and replaced with a new product. Furthermore, the "number of uses" mentioned here refers to the number of cases, and "fewer than 10 uses" means less than 10 times.

[0140] The balloon 19 is formed into a generally cylindrical shape with its ends tightened by an elastic material such as rubber. The balloon 19 has a small-diameter front end 19a and a base end 19b, and a central protrusion 19c. Regarding the balloon 19, after the insertion part 12a is inserted into the interior and positioned in a predetermined position, rubber rings 20a and 20b are inserted into the front end 19a and the base end 19b, thereby fixing it to the insertion part 12a.

[0141] like Figure 3 As shown, the balloon 19 is fixed at a predetermined position in the insertion portion 12a, closer to the base end of the insertion portion 12a than the bending portion 12c, preferably at a position where the front end of the balloon 19's front portion 19a coincides with the base end of the bending portion 12c. Therefore, the balloon 19 does not obstruct the bending action of the bending portion 12c, and the bending portion 12c does not obstruct the inflation or deflation of the balloon 19. For the balloon 19, as described later, inflation or deflation is controlled by a balloon control device BLC. The balloon control device BLC is preferably operated by a user interface 16.

[0142] like Figure 4 As shown, when the balloon 19 is contracted, the insertion part 12a is not fixed to the intestine 26. If the measurement light is irradiated and photographed while the balloon is contracted, the position of the front part 12d will move in the up, down, left and right directions, and sometimes it is impossible to accurately irradiate the object to be measured by the user.

[0143] Therefore, as Figure 5 As shown, the balloon 19 is inflated under the control of the balloon control device BLC. The inflated balloon 19 is formed with an outer diameter matching the inner diameter of the intestine 26, thus fixing the insertion portion 12a within the intestine 26. Because the insertion portion 12a is fixed to the intestine 26, measurement light can be accurately irradiated onto the object to be measured. Furthermore, the term "fixed state" here includes a fixed position of the insertion portion 12a in the insertion direction, but the orientation of the tip 12d can be finely adjusted.

[0144] like Figure 6 As shown, the anterior endpiece 12d is approximately circular, and along the first direction D1, it is provided with an imaging optical system 21, an illumination optical system 22, an opening 24, and an air / water supply nozzle 25. Two illumination optical systems 22 are provided on either side of the imaging optical system 21 in a second direction orthogonal to the first direction. A measurement light emission section 23 is positioned in the first direction between the imaging optical system 21 and the air / water supply nozzle 25. Therefore, the air / water outlet of the air / water supply nozzle 25 faces both the imaging optical system 21 and the measurement light emission section 23, allowing for cleaning of both by supplying air or water.

[0145] like Figure 7As shown, a front end cap 27 is mounted on the front end 12d. A front end face 28 is provided on the front end cap 27. The front end face 28 has a plane 28a, a plane 28b, and a guide surface 28c. Plane 28a is a plane orthogonal to the axial direction Z. Plane 28b is parallel to plane 28a and is located further forward than plane 28a in the axial direction Z. The guide surface 28c is disposed between plane 28a and plane 28b.

[0146] A through hole 27a is provided on plane 28b to expose the front end face 21c of the camera optical system 21 and a through hole 27b to expose the front end face 22b of a pair of illumination optical systems 22. The front end face 21c, the front end face 22b and plane 28b are disposed on the same plane.

[0147] Through holes 27c and 27d are provided in plane 28a. An air- and water-supplying nozzle 25 protrudes from through hole 27c. That is, plane 28a is the mounting position of the air- and water-supplying nozzle 25 in the Z-axis direction. A spray tube portion 25a is formed on the front end side of the air- and water-supplying nozzle 25. The spray tube portion 25a is formed as a cylinder protruding from the base end of the air- and water-supplying nozzle 25 in a direction bent at, for example, 90 degrees, and has a spray port 25b at its front end. The spray tube portion 25a is positioned to protrude from through hole 27c towards the front end side in the Z-axis direction.

[0148] The nozzle 25b is positioned toward the camera optical system 21. Thus, the air / water supply nozzle 25 sprays cleaning fluid or gas as a fluid toward the front end face 21c and its surrounding area of ​​the camera optical system 21.

[0149] When cleaning water or gas is sprayed from the air / water nozzle 25 into the camera optical system 21, the flow velocity F1 of the cleaning water reaching the camera optical system 21, i.e., the outer periphery of the camera optical system 21, is preferably 2 m / s or more, and the flow velocity F2 of the gas at the outer periphery of the camera optical system 21 is preferably 40 m / s or more. Furthermore, the flow velocities F1 and F2 preferably satisfy the above values ​​regardless of the orientation of the front end 12d. For example, when the air / water nozzle 25 is located vertically below the camera optical system 21, the flow velocity decreases due to the gravity of the cleaning water or gas, but it is still preferable to satisfy the above values ​​in this case.

[0150] The front end face of the measuring light emitting section 23, which protrudes from the through hole 27d, is disposed on plane 28a. That is, the mounting position of the air-water delivery nozzle 25 and the front end face of the measuring light emitting section 23 are disposed at the same position in the Z-axis direction. The measuring light emitting section 23 is disposed within the fluid spray range of the air-water delivery nozzle 25 and between the imaging optical system 21 and the air-water delivery nozzle 25. In this embodiment, when viewing the front end face 28 from the Z-axis direction, the measuring light emitting section 23 is disposed in the area connecting the spray port 25b of the air-water delivery nozzle 25 to the outer periphery of the imaging optical system 21. Therefore, when fluid is sprayed from the air-water delivery nozzle 25 to the imaging optical system 21, fluid can also be sprayed to the measuring light emitting section 23 simultaneously.

[0151] The guide surface 28c is formed as a continuous surface connecting plane 28a and plane 28b. The guide surface 28c is an inclined surface that is flat from the position contacting the outer periphery of the measuring light emitting section 23 to the position contacting the outer periphery of the imaging optical system 21. The guide surface 28c is disposed within the fluid spray range of the air-water delivery nozzle, so when fluid is sprayed from the air-water delivery nozzle 25, the fluid is also sprayed onto the guide surface 28c. The fluid sprayed onto the guide surface 28c diffuses and is blown onto the imaging optical system 21. Alternatively, the guide surface 28c may be entirely included within the fluid spray range of the air-water delivery nozzle, or only a portion of the guide surface 28c may be included. In this embodiment, the entire guide surface 28c is included in the region connecting the spray port 25b of the air-water delivery nozzle 25 and the outer periphery of the imaging optical system 21.

[0152] like Figure 8 As shown, the light source device 13 includes a light source unit 30 and a light source processor 31. The light source unit 30 generates illumination light or special light for illuminating the subject. The illumination light or special light emitted from the light source unit 30 enters the light guide LG and illuminates the subject through the illumination lens 22a. As the light source unit 30, a white light source emitting white light or multiple light sources including a white light source and a light source emitting other colors of light (e.g., a blue light source emitting blue light) can be used as the light source unit 30. Furthermore, as the light source unit 30, a broadband light source emitting a narrow-band blue light for emphasizing surface information such as superficial blood vessels can be used as the light source for the special light. In addition, the illumination light can be a white mixed light composed of at least one of purple, blue, green, or red light. In this case, it is preferable to design the illumination optical system 22 in such a way that the illumination range of green light is larger than that of red light.

[0153] The light source processor 31 controls the light source unit 30 according to commands from the system control unit 41. In addition to issuing commands related to light source control to the light source processor 31, the system control unit 41 also controls the light source 23a (see reference) of the light emission unit 23. Figure 10 In normal observation mode, the system control unit 41 controls the illumination light to turn off the measuring light. In special observation mode, it controls the illumination of a special light to turn off the measuring light. In length measurement mode, the system control unit 41 controls the illumination light or the measuring light to turn off.

[0154] The illumination optical system 22 has an illumination lens 22a through which light from the light guide LG illuminates the object being observed. The imaging optical system 21 has an objective lens 21a, a zoom lens 21b, and an imaging element 32. Reflected light from the object being observed enters the imaging element 32 through the objective lens 21a and the zoom lens 21b. Thus, a reflected image of the object being observed is formed on the imaging element 32.

[0155] The zoom lens 21b, by moving between the telephoto and wide-angle ends, provides an optical zoom function, magnifying or reducing the size of the subject. The opening and closing of the optical zoom function can be controlled by the zoom operation unit 12h (see reference) provided in the operation unit 12b of the endoscope. Figure 1 Switching to the optical zoom function enabled allows for further operation of the zoom control unit 12h, thereby magnifying or reducing the subject at a specific magnification.

[0156] The imaging element 32 is a color camera sensor that captures the reflected image of the subject and outputs an image signal. This imaging element 32 is preferably a CCD (Charge Coupled Device) camera sensor, a CMOS (Complementary Metal-Oxide Semiconductor) camera sensor, or the like. The imaging element 32 used in this invention is a color camera sensor for obtaining red (red), green (green), and blue images (R, G, and B). The red image is the image output from red pixels in the imaging element 32 that have a red color filter. The green image is the image output from green pixels in the imaging element 32 that have a green color filter. The blue image is the image output from blue pixels in the imaging element 32 that have a blue color filter. The imaging element 32 is controlled by the camera control unit 33.

[0157] The image signal output from the imaging element 32 is sent to the CDS / AGC circuit 34. The CDS / AGC circuit 34 performs Correlated Double Sampling (CDS) and Automatic Gain Control (AGC) on the image signal, which is an analog signal. The image signal via the CDS / AGC circuit 34 is converted into a digital image signal by an A / D converter (A / D converter) 35. The A / D converted digital image signal is input to the communication I / F (Interface) 37 of the light source device 13 via the communication I / F (Interface) 36.

[0158] In the processor device 14, programs related to various processing or control are embedded in a program memory (not shown). The system control unit 41, which is composed of an image control processor, operates by making the programs embedded in the program memory work, thereby realizing the functions of the receiving unit 38, the signal processing unit 39, and the display control unit 40, which are connected to the communication I / F (Interface) 37 of the light source device 13.

[0159] The receiving unit 38 receives image signals transmitted from the communication I / F 37 and transmits them to the signal processing unit 39. The signal processing unit 39 has a built-in memory that temporarily stores the image signals received from the receiving unit 38, and processes the set of image signals stored in the memory, i.e., the image signal group, to generate a subject image. In addition, the receiving unit 38 can directly send control signals associated with the light source processor 31 to the system control unit 41.

[0160] In the signal processing unit 39, when the normal viewing mode is set, signal allocation processing is performed to allocate the blue image of the subject image to the B channel of the display 15, the green image of the subject image to the G channel of the display 15, and the red image of the subject image to the R channel of the display 15, thereby displaying a color subject image on the display 15. The same signal allocation processing is performed for the length measurement mode as for the normal viewing mode.

[0161] On the other hand, in the signal processing unit 39, when the special observation mode is set, the red image of the subject image is not used for display on the display 15. The blue image of the subject image is assigned to the B and G channels of the display 15, and the green image of the subject image is assigned to the R channel of the display 15, thereby displaying a pseudo-color subject image on the display 15. Furthermore, in the signal processing unit 39, when the length measurement mode is set, the subject image, including the position of the measuring light, is sent to the data transceiver unit 43. The data transceiver unit 43 sends data related to the subject image to the extended processor device 17. Additionally, the data transceiver unit 43 can receive data from the extended processor device 17. The received data can be processed by the signal processing unit 39 or the system control unit 41.

[0162] In the signal processing unit 39, as a zoom function, when the digital zoom function is set to be turned on through the user interface 16, a portion of the subject image is cropped and enlarged or reduced, thereby enlarging or reducing the subject by a specific magnification. Figure 9 (A) shows an image of the subject with digital zoom off. Figure 9 (B) shows the shearing Figure 9 (A) is a subject image in which the digital zoom function, which magnifies the central part of the subject image, is turned on.

[0163] Additionally, when digital zoom is off, the subject is not enlarged or reduced by cropping the subject image.

[0164] The display control unit 40 displays the subject image generated by the signal processing unit 39 on the display 15. The system control unit 41 performs various controls on the endoscope 12, the light source device 13, the processor device 14, and the extended processor device 17. The system control unit 41 controls the imaging element 32 via the camera control unit 33 provided on the endoscope 12. The camera control unit 33 also controls the CDS / AGC circuit 34 and the A / D converter 35 based on the control of the imaging element 32.

[0165] The extended processor device 17 receives data transmitted from the processor device 14 via the data transceiver unit 44. The signal processing unit 45 performs processing related to the length measurement mode based on the data received by the data transceiver unit 44. Specifically, the processing involves determining the size of a virtual scale based on an image of the subject including the illumination position of the measuring light, and overlaying the determined virtual scale onto the subject image. The display control unit 46 displays the subject image with the overlaid virtual scale on the extended display 18. Furthermore, the data transceiver unit 44 can transmit data, etc., to the processor device 14.

[0166] like Figure 10 As shown, the measurement light emitting section 23 is positioned relative to the optical axis Ax of the imaging optical system 21 (reference). Figure 13 The measuring light is emitted at an angle. The measuring light emitting section 23 includes a light source 23a, a diffractive optical element 23b (DOE), a prism 23c, and an emitting section 23d. The light source 23a emits light of a color detectable by the pixels of the imaging element 32 (specifically, visible light), and includes a light-emitting element such as a laser light source LD (Laser Diode) or an LED (Light Emitting Diode) and a condenser lens that focuses the light emitted from the light-emitting element. In addition, the light source 23a is disposed on the observer circuit board (not shown). The observer circuit board is disposed at the front end portion 12d of the endoscope, receives power from the light source device 13 or the processor device 14, and supplies power to the light source 23a. In addition, the light source 23a is disposed at the front end portion 12d of the endoscope, but it may also be disposed inside the connector used to connect the endoscope 12 and the processor device 14. At this time, the components of the measuring light emitting section 23 other than the light source 23a (diffractive optical element DOE 23b, prism 23c and emitting section 23d) are also provided at the front end 12d of the endoscope.

[0167] In this embodiment, the wavelength of the light emitted by the light source 23a is, for example, a red laser (the color of the beam) with a wavelength of 600 nm or more and 650 nm or less, but light of other wavelength bands, such as green light with a wavelength of 495 nm or more and 570 nm or less, can be used. The light source 23a is controlled by the system control unit 41, and light emission is performed according to commands from the system control unit 41. The DOE 23b converts the light emitted from the light source into measurement light for obtaining measurement information. In addition, regarding the measurement light, it is preferable to adjust the light intensity from the viewpoint of protecting the human body, eyes, and internal organs, and adjust the light intensity to a sufficiently white (pixel saturation) level within the observation range of the endoscope 12.

[0168] Prism 23c is an optical component used to change the direction of travel of the measurement light converted by DOE 23b. Prism 23c changes the direction of travel of the measurement light so that it intersects with the field of view of the imaging optical system 21, which includes objective lens 21a. Details regarding the direction of travel of the measurement light will be described later. The measurement light Lm emitted from prism 23c illuminates the subject.

[0169] like Figure 11As shown, the measuring light emitting section 23 is housed in a measuring light emitting section receiving section 47 provided at the front end portion 12d of the endoscope. The measuring light emitting section receiving section 47 has an opening corresponding to the size of the measuring light emitting section 23. The measuring light emitting section receiving section 47 is closed by a transparent cover 48. The transparent cover 48 is a transparent plate, with one end face being a flat portion 48a. The transparent cover 48 is configured such that the flat portion 48a is on the same plane as the front end face 28 of the front end portion 12d. By providing a transparent cover 48 that is on the same plane as the front end face 28, foreign objects that may sometimes obstruct the emission of measuring light are not trapped.

[0170] like Figure 12 As shown, a prism 49 is disposed between the transparent cover 48 and the prism 23c. The prism 49 has a first sealing surface 49a and a second sealing surface 49b, such that the first sealing surface 49a is sealed to the prism 23c, and the second sealing surface 49b is sealed to the transparent cover 48. Through the prism 49, gas is expelled from between the transparent cover 48 and the prism 23c, making the space airtight. By making it airtight in this way, condensation can be prevented. That is, problems such as attenuation, diffusion, convergence, and refraction of the measurement light caused by condensation can be prevented.

[0171] Furthermore, an example was described where, when the refractive index of prism 23c is set to "n1" and the refractive index of prism 49 is set to "n2", the light-emitting surface of prism 23c is tilted towards the optical axis Ax, satisfying "n1 < n2". However, the structure can also be the opposite. It can be set to "n1 > n2", with the light-emitting surface of prism 23c positioned opposite to the optical axis Ax. However, in this case, total internal reflection may occur at the light-emitting surface of prism 23c, therefore, restrictions need to be placed on the light-emitting surface of prism 23c.

[0172] Alternatively, instead of using an optical component to form the prism 23c, a measurement assistance slit can be formed at the front end 12d of the endoscope. Furthermore, when the prism 23c is composed of an optical component, it is preferable to apply an anti-reflection coating (AR coating) (anti-reflection part) to the exit surface. This anti-reflection coating is provided because if the measurement light is reflected instead of passing through the exit surface of the prism 23c, thereby reducing the proportion of measurement light illuminating the subject, the illumination position detection unit 61, described later, will have difficulty identifying the position of the light spot SP formed on the subject by the measurement light.

[0173] Furthermore, the measurement light emitting section 23 only needs to be able to emit measurement light toward the field of view of the imaging optical system 21. For example, the light source 23a can be provided in the light source device, and the light emitted from the light source 23a can be guided to the DOE 23b through an optical fiber. Moreover, it can be configured such that the orientation of the light source 23a and the DOE 23b is tilted relative to the optical axis Ax of the imaging optical system 21 without using the prism 23c, thereby emitting the measurement light Lm in a direction spanning the field of view of the imaging optical system 21.

[0174] like Figure 13 As shown, regarding the direction of travel of the measuring light, the measuring light is emitted when the optical axis Lm of the measuring light intersects the optical axis Ax of the imaging optical system 21. It is known that if observation can be performed within the observation distance range Rx, the position of the light spot SP formed on the subject by the measuring light Lm within the imaging range (indicated by arrows Qx, Qy, and Qz) at each point in the near end Px, near the center Py, and far end Pz of the range Rx (the points where arrows Qx, Qy, and Qz intersect with the optical axis Ax) is different. Furthermore, the photographic angle of the imaging optical system 21 is represented by the area between the two solid lines 101X, and measurements are performed in the central region (the area between the two dashed lines 102X) where aberrations are minimal within this photographic angle. Also, the third direction D3 is the direction intersecting with the first direction D1 and the second direction D2 (…). Figure 45 (The same applies).

[0175] As described above, the measuring light Lm is emitted with its optical axis Lm intersecting the optical axis Ax, thereby allowing the size of the subject to be measured based on the movement of the spot position relative to the observation distance. Then, the imaging element 32 captures an image of the subject illuminated by the measuring light, thereby obtaining an image of the subject including the spot SP. In the subject image, the position of the spot SP varies depending on the relationship between the optical axis Ax of the imaging optical system 21 and the optical axis Lm of the measuring light, as well as the observation distance. However, the closer the observation distance, the more pixels display the same actual size (e.g., 5 mm); the farther the observation distance, the fewer pixels.

[0176] like Figure 14 As shown, the system control unit 41 includes a length measurement corresponding endoscope determination unit 140, a measurement light on / off switching unit 141, a length measurement image display setting on / off switching unit 142, a length measurement function operation status display on / off switching unit 143, a virtual scale display switching control unit 144, and a previous image display setting saving unit 149.

[0177] When the endoscope 12 is connected to the processor device 14, the length-measuring endoscope-corresponding determination unit 140 determines whether the endoscope 12 is a length-measuring endoscope. If the endoscope 12 is a length-measuring endoscope, switching to the length-measuring mode is enabled. A length-measuring endoscope is an endoscope capable of irradiating and receiving measurement light and capable of displaying a length-measuring image based on a virtual scale of the measurement light on an extended display 18 (which may be a display 15). In addition, the length-measuring endoscope-corresponding determination unit 140 has an observer ID table (not shown) that associates the observer ID set on the endoscope 12 with a flag indicating whether a length-measuring endoscope is present (for example, setting the case of a length-measuring endoscope to "1" and the case of other endoscopes to "0"). Furthermore, when the endoscope 12 is connected, the length-measuring endoscope-corresponding determination unit 140 reads the endoscope observer ID. Refer to the markers in the observer ID table to determine whether the read observer ID corresponds to the endoscope used for length measurement.

[0178] The measuring light on / off switch 141 controls the light source 23a, switching the measuring light on (on) or off (off). The length measurement image display setting on / off switch 142, through the user interface 16, etc., can (on) or not (off) execute various image display settings (hue, etc.) for the length measurement image in the length measurement mode. The virtual scale display switching control 144 switches the virtual scale on the extended display 18 to any of the following: displayed (on), not displayed (off), or display mode changed.

[0179] When the switching to the length measurement mode is enabled, the system control unit 41 performs at least one of the following switching operations by using the observation mode switching switch 12f: switching the length measurement image display setting to on or off, switching the measurement light to on or off, switching the length measurement function operation status display to on or off, switching the virtual scale display to on or off, or switching the display mode to change.

[0180] For example, the system control unit 41 preferably switches the measuring light to on, the length measurement image display setting to on, the length measurement function operation status display to on, and the virtual scale display to on by switching to the length measurement mode. On the other hand, it is preferable to switch the measuring light to off, the length measurement image display setting to off, the length measurement function operation status display to off, and the virtual scale display to off by switching from the length measurement mode to other modes (normal observation mode, special observation mode).

[0181] like Figure 15As shown, the length measurement function operation status is preferably displayed as icon 146 in the scale display on the auxiliary information display area 18a of the extended display 18. Icon 146 is displayed when switching to the length measurement mode, and not displayed when switching from the length measurement mode to another mode. Furthermore, the virtual scale 147 is preferably displayed in the observation image display area 18b of the extended display 18. Changes to the display mode of the virtual scale 147 are performed by the virtual scale display switching control unit 144.

[0182] Virtual scale 147 includes virtual scale 147a (5mm), virtual scale 147b (10mm), and virtual scale 147c (20mm). Virtual scales 147a, 147b, and 147c each have a circular scale (represented by dashed lines) and a linear scale (represented by solid lines), respectively. The "5" in virtual scale 147a indicates a 5mm scale, the "10" in virtual scale 147b indicates a 10mm scale, and the "20" in virtual scale 147c indicates a 20mm scale.

[0183] The display method of the virtual scale can be changed, for example, by selecting from a set of predefined scale patterns. For instance, as multiple scale patterns, such as... Figure 15 As shown, besides scale patterns formed by combining three virtual scales 147a, 147b, and 147c that have both circular and linear scales, there are also scale patterns formed by combining two virtual scales 147b and 147c that have both circular and linear scales, and scale patterns formed by combining three virtual scales 147a, 147b, and 147c that only contain linear segments. Scale patterns are represented by combining one or more of the shapes of multiple scales, such as multiple scale dimensions, circular scales, and linear scales.

[0184] Furthermore, when the length measurement image display setting is turned on, it is preferable to save the image display setting before switching to the length measurement mode in the image display setting saving unit 149 before switching. For example, if the observation mode before switching to the length measurement mode is the normal observation mode, it is preferable to save the image display setting of the normal observation mode set by the signal processing unit 39 in the image display setting saving unit 149 before switching. Furthermore, when the length measurement image display setting is turned off, it is preferable to switch to the image display setting saved in the image display setting saving unit 149 before switching. For example, if the image display setting of the normal observation mode is saved in the image display setting saving unit 149 before switching, the signal processing unit 39 sets the image display setting of the normal observation mode saved in the image display setting saving unit 149 before switching according to the switch to the normal observation mode.

[0185] On the other hand, during the switching operation to the length measurement mode, the system control unit 41 prohibits switching the measurement light to on, the length measurement image display setting to on, the length measurement function operation status display to on, and the virtual scale display to on if the mode switching conditions are not met. The mode switching conditions are conditions suitable for executing the length measurement mode under the setting conditions related to the endoscope 12, the light source device 13, the processor device 14, and the extended processor device 17. The mode switching conditions are preferably conditions that do not fall under the following prohibited setting conditions. In addition, if the mode switching conditions are not met, it is preferable to set the indicator that the length measurement function operation status, which indicates that the virtual scale 147 is not displayed on the extended display 18, is not displayed as "displayable" instead of prohibiting the display of the scale display icon 146. The indicator that the length measurement function operation status is not displayed is preferably displayed as the scale not displayed icon 148 in the accompanying information display area 18a.

[0186] like Figure 16 As shown, the system control unit 41 is equipped with a length measurement mode control unit 50 that performs controls related to whether or not the length measurement mode can be executed. The length measurement mode control unit 50 performs at least one of the following controls: First control: when switching to the length measurement mode via the observation mode switch 12f, if the setting conditions in the current settings related to the endoscope 12, the light source device 13, and the processor device 14 meet a predetermined prohibition setting condition, the switching to the length measurement mode is prohibited; Second control: when changing the setting conditions via the user interface 16 in the length measurement mode, if the setting conditions to be changed by the setting change operation meet the prohibition setting condition, the setting change operation is invalidated; or Third control: when changing the setting conditions via the user interface 16 in the length measurement mode, if the setting conditions to be changed by the setting change operation meet the prohibition setting condition, the setting mode is automatically switched from the length measurement mode to another mode.

[0187] The setting conditions related to the light source device 13 include illumination conditions for the illumination light used in normal observation mode or length measurement mode, illumination conditions for special light used in special observation mode, or illumination conditions for the measuring light used in length measurement mode. Illumination conditions include, for example, the amount of illumination light. The setting conditions related to the endoscope 12 include imaging conditions related to the photographing of the subject. Imaging conditions include, for example, shutter speed. The setting conditions related to the processor device 14 include processing conditions such as image processing related to the subject image. Processing conditions include, for example, color balance, brightness correction, and various emphasis processing. In addition, it is preferable to optimize the position detection of the light spot SP in length measurement mode and set the setting conditions (illumination light amount, shutter speed, color balance, brightness correction, and various emphasis processing) to meet the visual recognizability when the user measures the size.

[0188] The prohibited setting conditions include: a first prohibited setting condition that prevents the detection of the illumination position of the measuring light from the subject image in length measurement mode; and a second prohibited setting condition that prevents the accurate display of a virtual scale corresponding to the observation distance in the length measurement image. Examples of the first prohibited setting conditions include, for instance, a special observation mode, emphasis on brightness or red in the subject image. In the special observation mode, the red image used for detecting spot SP, etc., in length measurement mode is not used for image display, making it difficult to detect the illumination position of the measuring light. Furthermore, in length measurement mode, compared to the normal observation mode or the special observation mode, it is preferable to reduce the brightness in the subject image and suppress red tones.

[0189] Furthermore, as a second prohibited setting condition, the use (on) of zoom functions, including optical zoom or digital zoom, is prohibited. This is because the virtual scale displayed in the measurement image is determined based on the position of the spot SP, rather than based on the magnification of the zoom function. Therefore, when the zoom function is on, the virtual scale is difficult to display corresponding to the observation distance.

[0190] For example, if the device is set to special observation mode and then switched to length measurement mode via observation mode switch 12f, such as... Figure 17 As shown, the length measurement mode control unit 50 performs a first control that prevents switching to the length measurement mode and maintains the special observation mode. When the first control is performed, as... Figure 18 As shown, the length measurement mode control unit 50 displays a message prohibiting switching to the length measurement mode on the extended display 18 (and may emit a warning sound). Alternatively, instead of prohibiting switching to the length measurement mode, as shown... Figure 19 As shown, the length measurement mode control unit 50 can control the deactivation of the special observation mode and the switching to the length measurement mode.

[0191] Furthermore, in the length measurement mode, if a setting change operation to enable the zoom function is performed based on the operation of the zoom control unit 12h, such as... Figure 20 As shown, the length measurement mode control unit 50 performs a second control to disable the setting change operation for enabling the zoom function. When the second control is performed, as... Figure 21 As shown, the length measurement mode control unit 50 displays a message on the extended display 18 indicating that the setting change operation to enable the zoom function is invalid (and may emit a warning sound).

[0192] Furthermore, in the length measurement mode, if a setting change operation to enable the zoom function is performed based on the operation of the zoom control unit 12h, such as... Figure 22As shown, the length measurement mode control unit 50 deactivates the length measurement mode and switches to the normal observation mode as a third control for other modes. When the third control is activated, as... Figure 23 As shown, the length measurement mode control unit 50 will display a message on the extended display 18 indicating that the length measurement mode has been deactivated (the virtual scale is not displayed) and the normal observation mode has been switched (a warning sound may be emitted). Additionally, when the third control is performed, the zoom function is activated, allowing the zoom function to magnify or reduce the size of the subject in the image.

[0193] like Figure 24 As shown, a brightness information calculation unit 53, an illumination light intensity level setting unit 54, a first light emission control table 55, and a second light emission control table 56 can be provided in the system control unit 41. The brightness information calculation unit 53 calculates brightness information related to the brightness of the subject based on an image obtained in normal observation mode or a first photographic image (based on illumination light and measurement light) obtained in length measurement mode. The illumination light intensity level setting unit 54 sets the illumination light intensity level based on the brightness information. The illumination light intensity levels are set to five stages: level 1, level 2, level 3, level 4, and level 5. Information related to the illumination light intensity level is sent to the light source processor 31. The light source processor 31 controls the light source unit 30 to make the illumination light intensity the illumination light intensity level.

[0194] Table 55, used for the first emission control, is for controlling the light intensity of the measured light and stores the coordinate information of the light spot SP and the first relationship between the light intensity level of the measured light. Specifically, as shown in Table 55... Figure 25 As shown, for the five coordinate regions to which the coordinate information of the light spot SP belongs, the light intensity levels of the measurement light are determined as level 1, level 2, level 3, level 4, and level 5, respectively. The system control unit 41 refers to the first emission control table 55 to determine the light intensity level corresponding to the coordinate region to which the position of the light spot SP belongs. The system control unit 41 controls the light source 23a and the light intensity of the measurement light to achieve the determined light intensity level. Furthermore, the choice between using the first emission control table 55 and the second emission control table 56 to control the light intensity of the measurement light is appropriately set via the user interface 16.

[0195] like Figure 26As shown, coordinate region 1 is the area set at the bottom in the first image, and is at the closest viewing distance when the light spot SP belongs to coordinate region 1. Therefore, for coordinate region 1, the smallest level 1 is assigned as the light intensity level of the measurement light. Furthermore, coordinate region 2 is the area set higher than coordinate region 1, and is at a farther viewing distance than when the light spot SP belongs to coordinate region 2. Therefore, a larger level 2 is assigned as the light intensity level of the measurement light than level 1. Additionally, the direction of movement of the light spot SP changes with the direction of intersection between the optical axis Ax of the imaging optical system 21 and the optical axis Lm of the measurement light.

[0196] Similarly, coordinate region 3 is located higher than coordinate region 2. Since the light spot SP belongs to coordinate region 3, it is at a greater observation distance than coordinate region 2, and therefore, a higher level (level 3) is assigned to it as the light intensity level for measurement. Similarly, coordinate region 4 is located higher than coordinate region 3. Since the light spot SP belongs to coordinate region 4, it is at a greater observation distance than coordinate region 3, and therefore, a higher level (level 4) is assigned to it as the light intensity level for measurement. Furthermore, coordinate region 5 is located at the very top. Since the light spot SP belongs to coordinate region 5, it is at the furthest observation distance compared to coordinate regions 1 through 4, and therefore, the highest level (level 5) is assigned to it as the light intensity level for measurement.

[0197] Table 56, used for the second light emission control, is for controlling the light intensity of the measured light. It stores the coordinate information of the light spot SP and the second relationship between the light intensity level of the illumination light and the light intensity level of the measured light. Specifically, as shown in Table 56... Figure 27 As shown, the light intensity level of the measuring light was determined for the five coordinate regions to which the light spot SP belongs and the light intensity levels of the illumination light (levels 1, 2, 3, 4, and 5). For example, if the light spot SP belongs to coordinate region 1 and the light intensity level of the illumination light is level 3, then level 3 was assigned as the light intensity level of the measuring light.

[0198] The system control unit 41 refers to the second light emission control table 56 and determines the light intensity level of the measuring light based on the coordinate region to which the light spot SP belongs and the light intensity level of the illumination light. The system control unit 41 controls the light source 23a and controls the light intensity of the measuring light to achieve the determined light intensity level.

[0199] In Table 56 for the second emission control, the light intensity levels of the illumination light and the measurement light are set to a ratio that enables the determination of the position of the light spot SP. This is because if the ratio of the light intensity of the illumination light to the light intensity of the measurement light is inappropriate, the contrast of the light spot SP will be low, making it difficult to determine the position of the light spot SP.

[0200] In length measurement mode, the light source, processed by processor 31, continuously emits illumination light for overall illumination of the observed object; simultaneously, it pulses the measurement light Lm. Therefore, as... Figure 28 As shown, the frames emitting light in the length measurement mode include: an illumination light-only emission frame FLx, which emits illumination light but not measurement light; and a measurement light emission frame FLy, which emits both illumination light and measurement light. Furthermore, in the length measurement mode, the position of the light spot SP is detected from the first camera image obtained in the measurement light emission frame FLy, while a virtual scale is displayed on the second camera image obtained in the illumination light-only emission frame FLx. Additionally, with... Figure 28 The solid line corresponding to the illumination or measuring light indicates the light emission state in a frame. The period when the solid line is in the part corresponding to "on" indicates the period during which illumination or measuring light is emitted, and the period when the solid line is in the part corresponding to "off" indicates the period during which illumination or measuring light is not emitted.

[0201] The patterns for light emission and imaging in the length measurement mode are as follows. The first pattern uses a CCD (Global Shutter Imaging Element) as the imaging element 32, which performs exposure and charge readout at the same time for each pixel and outputs an image signal. Furthermore, in the first pattern, measurement light is emitted every two frames as a specific frame interval.

[0202] In the first pattern, as Figure 29 As shown, based on the exposure of the illumination light at time T1 in normal observation mode, when switching between normal observation mode and length measurement mode (from time T1 to time T2), the charge is read out simultaneously (global shutter), thereby obtaining a second image N containing only the illumination light component. This second image N is displayed on the extended display 18 at time T2. Furthermore, regarding... Figure 29 The "CCD (frame-period) global shutter" indicates that a global shutter has been activated when switching from timing T1 to timing T2, indicated by the vertically erected line 57. This applies to all other vertical lines 57 as well.

[0203] Furthermore, at time T2, illumination light and measurement light are emitted. Based on the exposure of the illumination light and measurement light at time T2, the charge is read out simultaneously when switching from time T2 to time T3, thereby obtaining a first image N+Lm containing the components of the illumination light and measurement light. The position of the light spot SP is detected based on the first image N+Lm. A virtual scale corresponding to the position of the light spot SP is displayed on the second image N displayed at time T2. Thus, at time T3, the length measurement image S of the virtual scale is displayed on the second image N of time T2.

[0204] Furthermore, the second camera image N at time T2 (first time) is displayed on the extended display 18 not only at time T2 but also at time T3. That is, the second camera image at time T2 is displayed for two consecutive frames until time T4 (second time) when the next second camera image is obtained (the same subject image is displayed at time T2 and T3). In addition, at time T3, the first camera image N+Lm is not displayed on the extended display 18. Here, compared to the normal viewing mode where the second camera image N is changed and displayed every frame, in the first pattern of the length measurement mode, as described above, the same second camera image N is displayed for two consecutive frames, thereby the frame rate of the first pattern of the length measurement mode is substantially half that of the normal viewing mode.

[0205] The same procedure applies after time T4. That is, at times T4 and T5, the second camera image of time T4 is continuously displayed on the length measurement image S, and at times T6 and T7, the second camera image N of time T6 is continuously displayed on the length measurement image S. However, at times T4, T5, T6, and T7, the first camera image N+Lm is not displayed on the extended display 18. As described above, by displaying the second camera image N, which does not contain the component of the measurement light, in the length measurement image S, although the frame rate is slightly reduced, there is no situation where the emission of the measurement light might hinder the visual recognition of the observed object.

[0206] As the second pattern, an imaging element 32 is used, which has multiple lines for photographing an object illuminated by illumination light or measurement light, and exposes each line at a different exposure time, reads out the charge at a different readout time for each line, and outputs an image signal. Furthermore, in the second pattern, measurement light is emitted every three frames as a specific frame interval.

[0207] In the second pattern, as Figure 30 As shown, by exposing the illumination light and reading out the charge for each line at time T1, and ending the charge reading (rolling shutter) when switching from normal observation mode to length measurement mode (from time T1 to time T2), a second image N containing only the illumination light component can be obtained. This second image N is displayed on the extended display 18 at time T2. Furthermore, regarding... Figure 30 The “CMOS (frame-period) rolling shutter” indicates the timing of light exposure and charge readout, with the slash 58 representing the start of online exposure and charge readout at Ls and the end of online exposure and charge readout at Lt.

[0208] Furthermore, at time T2, illumination light and measurement light are emitted. By using a rolling shutter based on the illumination light from time T1 to time T2 and the measurement light at time T2, a first image N+Lm, including components of both illumination light and measurement light, is obtained when switching from time T2 to time T3. Similarly, when switching from time T3 to time T4, a first image N+Lm, also including components of both illumination light and measurement light, is obtained. The position of the light spot SP is detected based on this first image N+Lm. Measurement light is not emitted during times T3 and T4.

[0209] A virtual scale corresponding to the position of the light spot SP is displayed on the second camera image N displayed at time T2. Thus, at times T3 and T4, the length measurement image S with the virtual scale is displayed on the second camera image N at time T2. Furthermore, the second camera image N at time T2 (the first time) is also displayed on the extended display 18 not only at time T2, but also at times T3 and T4. That is, the second camera image at time T2 is displayed continuously for 3 frames until time T5 (the second time) when the next second camera image is obtained (the same subject image is displayed at times T2, T3, and T4). Therefore, at times T3 and T4, the first camera image N+Lm is not displayed on the extended display 18. Furthermore, in the second pattern of the length measurement mode, the same second camera image N is displayed continuously for 3 frames, thus the frame rate of the second pattern in the length measurement mode becomes substantially 1 / 3 of that in the normal observation mode.

[0210] The same procedure applies after timing T5. At timings T5, T6, and T7, the second image captured at timing T5 is displayed on the length measurement image S. Conversely, at timings T5, T6, and T7, the first image N+Lm is not displayed on the extended display 18. As described above, by displaying the second image, which does not include the component of the measurement light, in the length measurement image S, although the frame rate is reduced, there is no situation where the visual recognition of the observed object might be hindered due to the emission of planar measurement light.

[0211] like Figure 31 As shown, in order to identify the position of the light spot SP and set the virtual scale, the signal processing unit 45 of the extended processor device 17 includes: a first signal processing unit 59, which detects the position of the light spot SP in the captured image; and a second signal processing unit 60, which sets the virtual scale based on the position of the light spot SP. Furthermore, the captured image includes not only the captured image obtained when both the illumination light and the measuring light are continuously illuminated, but also a first captured image obtained when both the illumination light and the measuring light are illuminated, while the illumination light is continuously illuminated and the measuring light is either illuminated or extinguished.

[0212] The first signal processing unit 59 includes an illumination position detection unit 61 for detecting the illumination position of the light spot SP from the camera image. In the illumination position detection unit 61, it is preferable to obtain the centroid coordinates of the light spot SP as the illumination position of the light spot SP.

[0213] The second signal processing unit 60 sets a first virtual scale as a virtual scale for measuring the size of the subject based on the illumination position of the light spot SP, and sets the scale display position of the first virtual scale. The second signal processing unit 60 refers to a scale table 62, which stores virtual scale images that display differently based on the illumination position and scale display position of the light spot SP, and sets a virtual scale corresponding to the illumination position of the light spot SP. The virtual scale may vary in size or shape depending on the illumination position and scale display position of the light spot SP. The display of the virtual scale image will be described later. Furthermore, the scale table 62 retains its contents even when the power to the extended processor device 17 is turned off. In addition to storing the virtual scale image in association with the illumination position, the scale table 62 may also store the distance to the subject corresponding to the illumination position (the distance between the front end 12d of the endoscope 12 and the subject) in association with the virtual scale image.

[0214] Furthermore, since the virtual scale image is required for each illumination position, the data capacity increases. Therefore, considering the storage capacity of the memory in the endoscope 12, startup time, and processing time, it is preferable to store it in the extended processor device 17 (or processor device 14) rather than in a memory (not shown) stored within the endoscope 12. Also, as described later, the virtual scale image is created from representative points of the virtual scale image obtained through calibration. However, if the virtual scale image is created from representative points in length measurement mode, time loss occurs, and the real-time processing is compromised. Therefore, by connecting the endoscope 12 to the endoscope connection unit, once the virtual scale image is created from representative points and the scale table 62 is updated, the virtual scale image is not created from representative points, but rather displayed using the updated scale table 62. Furthermore, in the second signal processing unit 60, in emergency situations where overlapping display of images becomes difficult, instead of displaying a virtual scale image overlapping on the length measurement image, a reference scale that determines the size of the scale is displayed on the length measurement image based on the relationship between the illumination position of the spot SP and the number of pixels corresponding to the actual size of the subject.

[0215] Furthermore, the second signal processing unit 60 includes a table update unit 64, which updates the scale table 62 when the endoscope 12 is connected to the endoscope connection unit. As described above, the scale table 62 can be updated because the positional relationship between the optical axis Lm of the measuring light and the imaging optical system 21 of the endoscope 12 varies depending on the model and serial number, and consequently, the display method of the virtual scale image also changes. In the table update unit 64, a representative point data table 66 is used to establish an association between representative point data related to representative points extracted from the virtual scale image and the illumination position. Details of the table update unit 64 and the representative point data table 66 will be described later. In addition, regarding the representative point data table 66, the distance to the subject corresponding to the illumination position (the distance between the front end 12d of the endoscope 12 and the subject) can be established and stored in association with the representative point data.

[0216] When the display control unit 46 displays a length measurement image with a virtual scale superimposed on the camera image on the extended display 18, it controls the display method of the virtual scale to vary depending on the illumination position of the spot SP and the display position of the scale. Specifically, the display control unit 46 displays the length measurement image with a first virtual scale superimposed on the extended display 18, centered on the spot SP. For example, a circular measurement mark is used as the first virtual scale. At this time, as... Figure 32 As shown, at the observation distance near the proximal end Px (reference) Figure 13 In the case of ), the virtual scale M1, which represents the actual size of 5 mm (horizontal and vertical dimensions of the image), is displayed by aligning the center of the light spot SP1 formed on the tumor tm1 of the subject with the center of the light spot SP1.

[0217] The virtual scale M1 is displayed at the periphery of the photographic image affected by distortion caused by the camera optical system 21, and therefore the virtual scale M1 is elliptical in shape to correspond to the effects of distortion, etc. The virtual scale M1 is almost identical to the range of the tumor tm1, thus the tumor tm1 can be measured to be approximately 5 mm. Alternatively, the first virtual scale can be displayed without showing a light spot on the photographic image.

[0218] And, as Figure 33 As shown, when the observation distance is close to the center Py, a virtual scale M2 representing the actual size of 5 mm (horizontal and vertical dimensions of the image) is displayed, aligned with the center of the light spot SP2 formed on the tumor tm2 of the subject. The scale display position of the virtual scale M2 is located in the center of the image, which is not easily affected by the distortion caused by the camera optical system 21. Therefore, the virtual scale M2 is circular and is not affected by distortion.

[0219] And, as Figure 34As shown, a virtual scale bar M3, representing an actual size of 5 mm (horizontal and vertical dimensions of the image), is displayed, aligned with the center of the light spot SP3 formed on the tumor tm3 of the subject. The virtual scale bar M3 is positioned at the periphery of the image, which is affected by distortion caused by the camera optical system 21; therefore, the virtual scale bar M3 is elliptical in shape to correspond to the effects of distortion, etc. As described above... Figures 32-34 As shown, the size of the first virtual scale, which corresponds to the same actual size of 5mm, decreases as the viewing distance increases. Furthermore, the shape of the first virtual scale also varies depending on the display position, corresponding to the distortion caused by the camera optical system 21.

[0220] In addition, Figures 32-34 In the first virtual scale, the center of the light spot SP is aligned with the center of the mark. However, if there are no issues with measurement accuracy, the first virtual scale can be displayed at a position far from the light spot SP. Nevertheless, it is preferable to display the first virtual scale near the light spot. Furthermore, the first virtual scale, in a state where the distortion of the corrected image is not distorted, can be displayed on the corrected image, rather than distorting the first virtual scale.

[0221] Furthermore, in Figures 32-34 The image displays a first virtual scale corresponding to the actual size of the subject, 5mm. However, the actual size of the subject can be set to any value (e.g., 2mm, 3mm, 10mm, etc.) depending on the object being observed and the purpose of observation. Furthermore, in... Figures 32-34 In the middle, the first virtual scale is set to be roughly circular, but as... Figure 35 As shown, it can also be a cross shape where vertical and horizontal lines intersect. Furthermore, it can be a graduated cross shape with at least one of the vertical and horizontal lines having a scale Mx added. Moreover, as the first virtual scale, it can be a distorted cross shape with at least one of the vertical or horizontal lines tilted. Furthermore, the first virtual scale can be a combination of a cross shape and a circle, forming a circle and a cross shape. Alternatively, the first virtual scale can be a measurement point group type formed by combining light spots with multiple measurement points EP corresponding to the actual size. Furthermore, the number of first virtual scales can be one or more, and the color of the first virtual scale can be changed according to the actual size.

[0222] Additionally, as the first virtual scale, such as Figure 36As shown, three concentric virtual scales M4A, M4B, and M4C (with diameters of 2mm, 5mm, and 10mm respectively) of different sizes can be displayed on the photographic image centered on the spot SP4 formed on the tumor tm4. Regarding these three concentric virtual scales, displaying multiple virtual scales eliminates the effort and time required for switching between them, and allows for measurement even when the subject has a non-linear shape. Furthermore, when multiple concentric virtual scales are displayed centered on the spot, multiple combinations of conditions can be prepared in advance, and the size and color can be selected from these combinations, rather than specifying them for each virtual scale individually.

[0223] exist Figure 36 In this example, all three concentric virtual scale bars are displayed in the same color (black). However, when displaying multiple concentric scale bars, it is possible to set multiple colored concentric scale bars that change color according to the virtual scale bars. For example... Figure 37 As shown, virtual scale M5A is represented by a dashed line in red, virtual scale M5B by a solid line in blue, and virtual scale M5C by a single-dot dashed line in white. By changing the colors of the virtual scales in this way, their legibility is improved, making measurements easier to take.

[0224] Furthermore, as the first virtual scale, in addition to multiple concentric virtual scales, such as Figure 38 As shown, multiple distorted concentric circle virtual scales can also be used, formed by distorting each concentric circle. In this case, the distorted concentric circle virtual scales M6A, M6B, and M6C are displayed in the image with the spot SP5 formed on the tumor tm5 as the center.

[0225] When the endoscope 12 is connected to the endoscope connection unit, the table update unit 64 refers to the representative point data table 66, creates a virtual scale image corresponding to the model and / or serial number of the endoscope 12, and updates the scale table 62.

[0226] Representative point data table 66 stores representative point data associated with representative points in the virtual scale image obtained during calibration, and stores this data in relation to the illumination position of the spot SP. Representative point data table 66 is created using the calibration method described later. Figure 39 As shown, the representative point data includes the coordinate information (X coordinate, Y coordinate) of several representative points RP extracted from the image M, which serves as a virtual scale image. The representative point data stored in the representative point data table 66 is the data when the positional relationship between the optical axis Lm of the measurement light and the camera optical system 21 is the default positional relationship.

[0227] When the endoscope 12 is connected to the endoscope connection unit, the table update unit 64 acquires information related to the positional relationship between the optical axis Lm of the measuring light and the imaging optical system 21, and updates the scale table 62 using the positional relationship and the representative point data table 66. Specifically, based on the difference between the positional relationship between the optical axis Lm of the measuring light in the endoscope 12 connected to the endoscope connection unit and the imaging optical system 21 and the default positional relationship, the difference in coordinate information of the representative point RP is calculated. Furthermore, as... Figure 40 As shown, the table update unit 64 obtains the representative point RP based on the difference calculated by adjusting the coordinate displacement of the default representative point RP. * Creating a virtual scale image M * In the creation of virtual scale images, it is preferable to use representative points RP. * Interpolation processing is performed between the points. The resulting interpolated virtual scale image is associated with the illumination position through table update unit 64. Thus, the scale is updated using table 62. Furthermore, in Figure 40 In the middle, only representative points RP and RP * A portion of the annotation symbols.

[0228] Additionally, regarding the measuring light, light that forms a spot when it illuminates the subject was used, but other types of light can be used. For example, such as... Figure 41 As shown, a planar measuring light, forming intersecting lines 67 on the subject, can be used when illuminating the subject. At this time, a second virtual scale is generated, consisting of the intersecting lines 67 and scales 68, where the scales 68 on the intersecting lines 67 serve as indicators of the size of the subject (e.g., a polyp P). When using the planar measuring light, the illumination position detection unit 61 detects the position of the intersecting lines 67 (the illumination position of the measuring light). The lower the intersecting lines 67 are, the closer the observation distance; the higher the intersecting lines 67 are, the farther the observation distance. Therefore, the lower the intersecting lines 67 are, the larger the intervals of the scales 68; the higher the intersecting lines 67 are, the smaller the intervals of the scales 68.

[0229] Furthermore, regarding the measurement of light, such as Figure 42As shown, the light can be composed of a planar light comprising at least two first characteristic lines CL1. When the measuring light illuminates the subject, a cross curve cc is formed based on the undulations of the subject, and a first light spot SPk1 is formed on the cross curve cc at positions corresponding to the two first characteristic lines CL1. Furthermore, the measuring light includes multiple second characteristic lines CL2, different from the first characteristic lines CL1, between the two first characteristic lines CL1. When the measuring light comprising the first characteristic lines CL1 and the second characteristic lines CL2 illuminates the subject, a second light spot SPk2 is formed at positions corresponding to the multiple second characteristic lines CL2. The second light spots SPk2 are smaller than the first light spots SPk1, and the spacing between the second light spots SPk2 is smaller. Therefore, a specific cross curve SCC is formed on the cross curve by the multiple second light spots SPk2. Measurement information is calculated based on the position of the specific cross curve SCC.

[0230] like Figure 43 As shown, in order to perform position identification and measurement information calculation for the first light spot SPk1 or the second light spot SPk2, the signal processing unit 45 of the extended processor device 17 includes a position determination unit 69 and a measurement information processing unit 70. The position determination unit 69 determines the position of the first light spot SPk1 or the second light spot SPk2 based on the captured image. As a method for determining the position, for example, the captured image is binarized, and the centroid of the white part (pixels with a signal intensity higher than the binarization threshold) in the binarized image is determined as the position of the first light spot SPk1 or the second light spot SPk2.

[0231] The measurement information processing unit 70 calculates measurement information based on the position of the first light spot SPk1 or the second light spot SPk2. The calculated measurement information is displayed on the camera image via the display control unit 46. When the measurement information is calculated based on the positions of the two first light spots SPk1, the measurement information can be accurately calculated even when the subject has a three-dimensional shape.

[0232] like Figure 44 As shown, the measurement information includes a first straight-line distance representing the straight-line distance between two first light spots SPk1 and second light spots SPk2. The measurement information processing unit 70 calculates the first straight-line distance using the following method. In the measurement information processing unit 70, as follows... Figure 45As shown, the coordinates (xp1, yp1, zp1) representing the actual size of the first light spot SPk1 are obtained based on its position. xp1 and yp1 are obtained from the coordinates of the first light spot SPk1 in the captured image, corresponding to its actual size. zp1 is obtained from the coordinates of the first light spot SPk1 and the coordinates of a predetermined specific light spot SPk, corresponding to its actual size. Similarly, the coordinates (xp2, yp2, zp2) representing the actual size of the second light spot SPk2 are obtained based on its position. xp2 and yp2 are obtained from the coordinates of the second light spot SPk2 in the captured image, corresponding to its actual size. zp2 is obtained from the coordinates of the second light spot SPk2 and the coordinates of a predetermined specific light spot SPk, corresponding to its actual size. Furthermore, the first straight-line distance is calculated using the following formula.

[0233] The first straight-line distance (equation) = ((xp2 - xp1) 2 +(yp2-yp1) 2 +(zp2-zp1) 2 ) 0.5

[0234] The calculated first straight-line distance is used as measurement information 71 (in Figure 44 The image is displayed as "20mm" in the image. Additionally, a specific spot SPk may or may not be displayed on the extended display 18.

[0235] Furthermore, regarding the measurement light, multiple focused beams arranged in a grid pattern at prescribed intervals in both the longitudinal and transverse directions can be used. This grid-like arrangement of focused beams can be used to photograph tumors, etc., within the subject. Figure 46 As shown, an image of the diffraction spot DS1 is acquired. The signal processing unit 45 of the extended processor device 17 measures the interval DT of the diffraction spot DS1. The interval DT corresponds to the number of pixels on the imaging surface of the imaging element 32. In addition, the interval of a specific portion of the plurality of diffraction spots DS1 (e.g., the interval near the center of the imaging surface) can be measured.

[0236] If the spacing of the diffraction spots DS1 is measured, the direction and distance to the subject are calculated based on the measurement results. This process uses the relationship between the spacing (number of pixels) of the diffraction spots DS1 and the distance to the subject. Specifically, as... Figure 47 As shown, the direction (α, β) and distance (r) of the diffraction spot DS1 of the object being measured are calculated. If the direction and distance to the subject are calculated, then the two-dimensional or three-dimensional information of the subject is calculated based on the calculated direction and distance. As the two-dimensional and three-dimensional information of the subject, the two-dimensional space of the subject can be calculated. Figure 47In the XY plane or three-dimensional space Figure 47 The shape, size, and area within the XYZ space. Furthermore, the transformation from (α, β, r) to (X, Y, Z) can be performed using equations A), B), and C), and the shape, size, and area can be calculated based on the (X, Y, Z) coordinates of each point on the subject.

[0237] Equation A) X = r × cosα × cosβ

[0238] Equation B) Y = r × cosα × sinβ

[0239] Equation C) Z = r × sinα

[0240] like Figure 48 As shown, in order to identify the position of the light spot and set the virtual scale, the signal processing unit 45 of the extended processor device 17 includes: a position determination unit 72, which determines the position of the light spot SP in the first camera image (an image based on the measurement light and the illumination light); and an image processing unit 73, which processes the first camera image or the second camera image (an image based on the illumination light) according to the position of the light spot SP to generate a length measurement image.

[0241] The position determination unit 72 includes an interference removal unit 74 that removes interfering components that hinder the determination of the position of the light spot SP. In the first image captured by the camera, if the image contains a color that is different from, but close to, the color of the measuring light that forms the light spot SP (approximate color of the measuring light), it is sometimes impossible to accurately determine the position of the light spot SP. Therefore, the interference removal unit 74 removes components of the approximate color of the measuring light as interference components from the first image captured by the camera. The position determination unit 72 determines the position of the light spot SP based on the first image captured by the camera after interference removal, where the interference components have been removed.

[0242] The interference removal unit 74 includes a color information conversion unit 75, a binarization processing unit 76, a mask image generation unit 77, and a removal unit 78. Utilizing... Figure 49 The process for obtaining the first image image after interference removal is described. The color information conversion unit 75 converts the first image image, which is an RGB image, into a first color information image, and converts the second image image, which is an RGB image, into a second color information image. The color information is preferably set to, for example, HSV (H(Hue)) and S(Saturation)). o n(chroma)) and V(value(lightness)). Furthermore, as color information, it can be set as color difference Cr and Cb.

[0243] The binarization processing unit 76 binarizes the first color information image to create a binarized first color information image, and binarizes the second color information image to create a binarized second color information image. The threshold used for binarization is set to a binarization threshold that includes the color of the measured light. Figure 50 , Figure 51 As shown, in the binarized first color information image, in addition to the color information 79 of the measured light, the color information 80 of the interference components is also included.

[0244] The mask image generation unit 77 generates a mask image for removing color information from the first camera image and extracting color information of the measurement light based on the binarized first color information image and the binarized second color information image. For example... Figure 52 As shown, the mask image generation unit 77 determines the region 81 containing the interference components based on the interference components included in the binarized second camera image. The region 81 containing the interference components is preferably set to be larger than the region occupied by the color information 80 of the interference components. This is because, in the event of hand shaking, the region of the color information 80 of the interference components becomes larger compared to the case without hand shaking. Furthermore, as... Figure 53 As shown, the mask image generation unit 77 generates a mask image that sets the region of color information 79 of the measurement light in the binarized first color information image as the extraction region for extracting color information, and sets the region 81 of the interfering components as the non-extraction region for not extracting color information. Furthermore, Figures 50-53 This diagram is schematically shown to illustrate the binarized first color information image, the binarized second color information image, the region of interference components, and the mask image.

[0245] The removal unit 78 extracts color information from the first color information image using a mask image, thereby obtaining color information with interference components removed and an interference-removed first color information image containing the color information of the measurement light. The interference-removed first color information image is then processed by RGB conversion to restore the color information to an RGB image, resulting in an interference-removed first photographic image. The position determination unit 72 determines the position of the light spot SP based on the interference-removed first photographic image. Since the interference components are removed from the interference-removed second photographic image, the position of the light spot SP can be accurately determined.

[0246] The image processing unit 73 includes an image selection unit 82 and a scale table 62. The image selection unit 82 selects a processing target image from either the first or second camera image as the object image to be processed based on the position of the light spot SP. The image processing unit 73 processes the selected image as the processing target image based on the position of the light spot SP. The image selection unit 82 selects the processing target image according to a state related to the position of the light spot SP. Alternatively, the image selection unit 82 can select the processing target image according to a command given by the user. For example, a user interface 16 can be used for such a command.

[0247] Specifically, when the spot SP is within a specific range during a specific period, the movement of the subject or the tip 12d of the endoscope is considered minimal, therefore the second image is selected as the image to be processed. As described above, when the movement is minimal, it is considered that alignment with the lesion contained in the subject can be easily achieved even without the spot SP. Furthermore, the second image does not include the color component of the measured light, therefore the color reproduction of the subject is not impaired. On the other hand, when the position of the spot SP is not within a specific range during a specific period, the movement of the subject or the tip 12d of the endoscope is considered significant, therefore the first image is selected as the image to be processed. As described above, when the movement is significant, the user manipulates the endoscope 12 to position the spot SP at the lesion. This facilitates alignment with the lesion.

[0248] The image processing unit 73 generates a first virtual scale as a virtual scale, displaying the actual size of the subject, based on the position of the light spot SP in the first captured image. The image processing unit 73 refers to a scale table 62 that stores the relationship between the position of the light spot SP in the first captured image and the first virtual scale representing the actual size of the subject, and calculates the size of the virtual scale based on the position of the light spot SP. Furthermore, the image processing unit 73 generates a first virtual scale corresponding to the size of the virtual scale.

[0249] like Figure 54 As shown, in order to identify the position of the spot SP and set the virtual scale, the signal processing unit 45 of the extended processor device 17 includes: a first signal processing unit 84 for detecting the position of the spot SP in the camera image; and a second signal processing unit 85 for setting the virtual scale based on the position of the spot SP.

[0250] The first signal processing unit 84 includes a mask processing unit 86, a binarization processing unit 87, an interference component removal unit 88, and an illumination position detection unit 89. Utilizing... Figures 55-57The process of removing interference components in the first signal processing unit 84 will be explained. The mask processing unit 86 performs mask processing on the red, green, and blue images in the captured image to extract a roughly parallelogram-shaped movable range Wx representing the movable range of the illumination position of the measurement light on the subject. Thus, as... Figure 56 , Figure 57 As shown, the red image PRx, green image PGx, and blue image PBx, after mask processing, are obtained by extracting the movable range Wx of the illumination position. For pixels within the movable range of the illumination position, interference components are removed, and the illumination position of the light spot SP is detected.

[0251] Next, the binarization processing unit 87 performs a first binarization process on pixels within the movable range of the illumination position in the masked red image PRx to obtain a binarized red image PRY (binarized first spectral image). In the first binarization process, as a threshold condition for the first binarization process, pixels with a value of "225" or higher are set to "1", and pixels with a value lower than "225" are set to "0". Through this first binarization process, the light spot SP, a component of the measurement light, is detected. However, in the first binarization process, in addition to the high-brightness component of the red component in the illumination light, i.e., the first interference component N1, white spots (pixel saturation) generated by the illumination light, i.e., the second interference component N2, are also detected. These first and second interference components are the main reasons hindering the detection of the illumination position of the light spot SP. In addition, the threshold condition is not only related to the threshold representing the boundary between the pixel values ​​of pixels that are set to "0" through binarization and the pixel values ​​of pixels that are set to "1" through binarization, but also refers to the condition that determines the range of pixel values ​​of pixels that are set to "0" through binarization and the range of pixel values ​​of pixels that are set to "1" through binarization.

[0252] Therefore, in order to remove the first interference component, the interference component removal unit 88 performs a first difference processing on the binarized red image PRY and the binarized green image PGy (binarized second spectral image) obtained by binarizing the green image PGx through the second binarization processing. In the first difference image PD1 obtained by the first difference processing, the first interference component N1 is removed. However, in the first difference image PD1, the second interference component N2 usually remains and is not removed. For pixels that become "0" or less after the first difference processing, the pixel value is set to "0". In addition, in the second binarization processing, as a threshold condition for the second binarization processing, pixels with pixel values ​​in the range of "30" to "220" are set to "1", and pixels in other ranges, that is, from "0" to less than "30" or more than "220", are set to "0". Furthermore, the first interference component is removed by the first difference processing of the binarized red image and the binarized green image, but it can also be removed by other first operation processing.

[0253] Furthermore, in order to remove the second interfering component, such as Figure 57 As shown, the interference removal unit 88 performs a second difference process between the first difference image PD1 and the binarized blue image PBy obtained by binarizing the blue image PBx through the third binarization process. In the second difference image PD2 obtained through the second difference process, the second interference component, which is difficult to remove through the first difference process, is removed. Similar to the first difference process, pixels that become "0" or lower after the second difference process are set to "0". Furthermore, in the third binarization process, as a threshold condition for the third binarization process, pixels with a value of "160" or higher are set to "1", and pixels with a value lower than "160" are set to "0". While the second difference process between the first difference image and the binarized blue image removes the second interference component, other second operation processes can also be used to remove the second interference component.

[0254] The illumination position detection unit 89 detects the illumination position of the light spot SP based on the first difference image or the second difference image. In the illumination position detection unit 89, it is preferable to obtain the centroid coordinates of the light spot SP as the illumination position of the light spot SP.

[0255] The second signal processing unit 85 sets a first virtual scale, representing the actual size of the subject, as a virtual scale based on the position of the light spot SP. The second signal processing unit 85 refers to a scale table 62 that stores the relationship between the position of the light spot SP and the first virtual scale representing the actual size of the subject, and calculates the size of the virtual scale based on the position of the light spot. Furthermore, the second signal processing unit 85 sets a first virtual scale corresponding to the size of the virtual scale.

[0256] like Figure 58 As shown, the signal processing unit 45 of the extended processor device 17 includes an illumination area recognition unit 90 and a second signal processing unit 60. The illumination area recognition unit 90 identifies a measurement light illumination area with a specific shape pattern from the captured image. Specifically, as... Figure 59 As shown, the specific shape pattern includes a white central region CR1 and a peripheral region SR1 that covers the central region and has characteristic quantities based on the measurement light. If the area illuminated by the measurement light is the aforementioned light spot SP, then the specific shape pattern is circular. In this case, the white central region CR1 is circular, and the peripheral region SR1 is annular.

[0257] Figure 60 The distribution of pixel values ​​for each color image in a captured image, comprising multiple color images including a red image RP, a green image GP, and a blue image BP, is shown. When the pixel values ​​of the red image RP, green image GP, and blue image BP in the central region CR1 reach their maximum pixel value (e.g., 255), the central region CR1 becomes white. At this time, when measurement light is incident on the imaging element 32, as... Figure 61 As shown, in the wavelength region WMB of the measurement light, the measurement beam light is transmitted not only through the red filter RF of the imaging element 32, but also through the maximum transmittance of the green filter GF and the blue filter BF. On the other hand, in the peripheral region SR1, the pixel value of the red image RP becomes larger than the pixel value of the green image GP or the blue image BP. Therefore, the peripheral region SR1 has a reddish tint. In addition, the light source 23a emits the measurement light at a specific intensity, thereby setting the pixel values ​​of the red image RP, the green image GP, and the blue image BP in the central region CR1 to the maximum pixel value.

[0258] In the illumination area recognition unit 90, a light spot SP having the specific shape and characteristic quantity described above can be identified. Specifically, as... Figure 62 As shown, the illumination area recognition unit 90 preferably has a learning model 91, which identifies the light spot SP by outputting a light spot SP as the measurement light illumination area from the input of the camera image. The learning model 91 performs machine learning based on multiple teacher data that associate the camera image with the identified measurement light illumination area. As the machine learning method, a CNN (Convolutional Neural Network) is preferably used.

[0259] By using the learning model 91 to identify the light spot SP, it is possible to identify not only the circular light spot SP composed of the circular central region CR1 and the annular peripheral region SR1 (see reference). Figure 59 It can also identify light spots SP that are patterns deformed from specific shapes, namely circles. For example, such as Figure 63As shown in (A), it is also possible to identify the longitudinally deformed light spot SP. Furthermore, as... Figure 63 As shown in (B), it is also possible to identify a circular spot SP that is partially damaged and deformed. Furthermore, as feature quantities of the surrounding region SR1 that can be identified by the learning model 91, in addition to red (the color of the measured light), there are also blue and green. Moreover, as feature quantities of the surrounding region SR1 that can be identified by the learning model 91, there are the brightness, lightness, chroma, and hue of the measured light. Furthermore, regarding the brightness, lightness, chroma, and hue of the measured light, it is preferable to obtain them by performing brightness conversion processing or lightness, chroma, and hue conversion processing on the surrounding region of the spot SP included in the captured image.

[0260] like Figure 64 As shown, in order to identify the position of the light spot SP, calculate the observation distance to the subject, and set the virtual scale, the signal processing unit 45 of the extended processor device 17 includes: a position determination unit 92, which determines the position of the light spot SP in the image and calculates the observation distance; and an image processing unit 93, which sets various virtual scales according to the observation distance and generates a length measurement image obtained by processing the image using various virtual scales.

[0261] The position determination unit 92 includes a distance calculation unit 94. The position determination unit 92 determines the position of the light spot SP formed on the subject by the measuring light based on the captured image obtained by illuminating the subject with illumination light and measuring light. The distance calculation unit 94 calculates the observation distance based on the position of the light spot SP.

[0262] The image processing unit 93 includes an image selection unit 95, a scale table 62, an offset setting unit 97, an offset distance calculation unit 98, and an offset virtual scale generation unit 99. The image selection unit 95 selects an image as the object to be processed based on the position of the spot SP. The offset setting unit 97 sets an offset amount corresponding to the height of the spot SP of the convex polyp 100 for the viewing distance. The offset distance calculation unit 98 calculates the offset distance by adding the offset amount to the viewing distance. The offset virtual scale generation unit 99 generates an offset virtual scale based on the offset distance.

[0263] The following explains the concept of offset. First, the convexity of a subject refers to a shape that protrudes from the surroundings within the subject. Therefore, any shape that partially protrudes from the surroundings is acceptable, and there are no limitations on its size, width, height and / or number of protruding parts, continuity of height, or other shapes.

[0264] More specifically, for example, such as Figure 65As shown, the subject has a convex polyp 100. The polyp 100 has a shape that protrudes from the surrounding subject. The polyp 100 has a top 100a and a flat portion 100b. In Figure 5, the polyp 100 is shown when the height direction of the polyp 100 is set as the vertical direction, viewed from a horizontal direction. However, the polyp 100 is three-dimensional and therefore exists in both the forward direction and the depth direction of the paper. The plane formed by the flat portion of the subject surrounding the polyp 100 and the portion on which the polyp 100 is formed is defined as the flat portion 100b of the polyp 100. The plane formed by the flat portion surrounding the polyp 100 is the extension surface 101 of the flat portion 100b.

[0265] Next, the height of the light spot SP of the convex polyp 100 will be explained. In this embodiment, the height of the light spot SP of the polyp 100 is the distance in the vertical direction from the light spot SP of the polyp 100 to the flat portion 100b of the polyp 100. More specifically, as... Figure 66 As shown, the light spot SP1 is formed on the top 100a of the polyp 100. Therefore, if the surface parallel to the extension surface 101 and passing through the top 100a of the polyp 100 is designated as the parallel surface 102, then the distance between the parallel surface 102 and the extension surface 101 becomes the height HT1 of the light spot SP1 of the polyp 100 (the distance in the vertical direction from the top 100a of the polyp 100 to the flat portion 100b). In addition, the polyp 100 and the height HT1 (the same applies to the height HT2 below) are schematically shown. As long as it is a portion that protrudes from the surrounding portion, the type, shape, size, etc. of the convexity are not limited.

[0266] Furthermore, in Figure 66 In this process, the light spot SP2 is formed in a region that is not the top 100a of the polyp 100. That is, it is formed between the top 100a of the polyp 100 and the end of the polyp 100. Therefore, if the surface of the light spot SP2 that is parallel to the extension surface 101 and passes through the polyp 100 is designated as the parallel surface 103, then the distance between the parallel surface 103 and the extension surface 101 is the height HT2 of the light spot SP2 of the polyp 100. Therefore, the height HT2 of the light spot SP2 of the polyp 100 is the distance in the vertical direction from the light spot SP2 of the polyp 100 to the flat portion 100b.

[0267] The following explains the observation distance and offset. Figure 67As shown, by measuring light Lm, a light spot SP1 is formed on the top 100a of the polyp 100. The observation distance obtained from the light spot SP1 is the distance D5 between the position P1 of the anterior endoscope 12d and the position P2 of the light spot SP1. The virtual scale corresponding to the distance D5 is a virtual scale that matches the actual measurement on the parallel plane 102. Therefore, when the light spot SP1 is formed on the top 100a of the polyp 100, if a virtual scale corresponding to the distance D5 is generated and displayed, a virtual scale matching the actual measurement of the subject on the parallel plane 102 is displayed. Therefore, for the actual measurement value of the subject on the extended plane 101, a virtual scale offset to the side with the same scale size is displayed.

[0268] Therefore, the offset setting unit 97 sets the height HT1 of the spot SP of the polyp 100 as the offset for the observation distance D5. Next, the offset distance calculation unit 98 adds the height HT1 of the spot SP1 of the polyp 100, which is the offset, to the observation distance D5 to calculate the offset distance D6. Therefore, the offset distance calculation unit 98 calculates the offset distance D6 using the following formula (OS). Furthermore, HT1 is the distance between positions P2 and P3.

[0269] Formula OS)D6=D5+HT1

[0270] Next, the offset virtual scale generation unit 99 generates a virtual scale based on the viewing distance D6 as the offset virtual scale. More specifically, the offset virtual scale generation unit 99 refers to the scale table 62 and uses the virtual scale at the viewing distance D6 as the offset virtual scale. The offset virtual scale represents the actual distance or size of the subject on the extended surface 101.

[0271] The image processing unit 93 performs processing to overlay the generated offset virtual scale onto the photographic image, thereby generating a length measurement image. For more accurate measurement, the offset virtual scale is preferably overlaid at the position where the light spot SP is formed. Therefore, if it is displayed at a position separate from the light spot SP, it is displayed as close as possible to the light spot SP. The length measurement image with the offset virtual scale overlaid is displayed on the extended display 18 via the display control unit 46.

[0272] In addition, such as Figure 68 As shown, regarding the virtual scale, it can be that the smaller the viewing distance, the larger the number of pixels of the line width that constitutes the virtual scale is set, and the larger the viewing distance, the smaller the number of pixels of the line width that constitutes the virtual scale is reduced.

[0273] exist Figure 68In the example shown in (A), when the viewing distance is greater than or equal to 21mm at the far end Pz within the range Rx, the line width W11 constituting the virtual scale M11 is set to the minimum setting value, i.e., 1 pixel. Figure 68 In the example shown in (B), when the viewing distance is 13mm to 20mm between the far end Pz and the central vicinity Py within the range Rx, the line width W12 constituting the virtual scale M12 is set to 2 pixels. Figure 68 In the example shown in (C), when the line width W13 constituting the virtual scale M13 is set to the set middle value of 3 pixels, near the center of Py within the range Rx and the viewing distance is 8 to 12 mm.

[0274] exist Figure 68 In the example shown in (D), when the viewing distance is 4–7 mm between the center Py and the near end Px within the range Rx, the line width W14 constituting the virtual scale M14 is set to 4 pixels. Figure 68 In the example shown in (E), when the viewing distance is less than 3mm and the near end Px within the range Rx is within the range Rx, the line width W15 that constitutes the virtual scale M15 is set to the maximum setting value of 5 pixels.

[0275] As described above, by varying the lines constituting the virtual scales M11 to M15 according to the observation distance, doctors can easily measure the accurate dimensions of the subject. Furthermore, by setting the line widths W11 to W15 of the virtual scales M11 to M15 to values ​​inversely proportional to the observation distance, the magnitude of dimensional errors can be identified based on the line width. For example, considering the identified error, if the tumor tm is located inside the lines constituting the virtual scales M11 to M15, it is determined that it is indeed smaller than the set actual size (if...). Figure 68 The example shown is within 5mm.

[0276] And, as Figure 69 As shown, a virtual scale M2 consisting of three concentric circles of different sizes can be set based on the position of a light spot SP. The three concentric circles M21, M22, and M23 constituting the virtual scale M2 represent, for example, the actual dimensions of "5mm", "10mm", and "20mm". Furthermore, for ease of illustration, the lines forming the circles in the concentric circles M21, M22, and M23 constituting the virtual scale M2 are shaded with intersecting lines, but in reality, each line is filled with a single color.

[0277] The width W22 of one of the outermost concentric circles M22, located at the innermost concentric circle M21, is set to be larger than the width W21 of concentric circle M21. The width W23 of the outermost concentric circle M23 is also set to be larger than the width W22 of concentric circle M22. Figure 69In the example shown, the width W23 is set to √2 times the width W22 and twice the width W21.

[0278] While maintaining the ratio of width W21 to W23 (in Figure 69 In the example shown, with a ratio of W21∶W22∶W23=1∶√2∶2, the value is set to be inversely proportional to the observation distance. Therefore, the magnitude of the dimensional error can be identified based on the line width. For example, considering the identified error, if the tumor tm is outside the line forming concentric circle M22 and inside the line forming concentric circle M23, it can be reliably determined that it falls within the range of the set actual size (if it is...). Figure 69 The example shown is between 10mm and 20mm.

[0279] like Figure 70 As shown, a grayscale with gradually decreasing density from the center outwards can be added to the lines constituting the virtual scale M3. Furthermore, in this case, the line width with the added grayscale is adjusted according to the viewing distance.

[0280] like Figure 71 As shown, the lines constituting the virtual scales M41 to M43 are depicted as dashed lines, and the gaps between these dashed lines are set to values ​​inversely proportional to the observation distance. Additionally, in Figure 71 In the figure, circular virtual scales M41, M42, and M43 are shown for shooting at points Pz (far end), Py (near center), and Px (near end) within the observation distance range Rx.

[0281] exist Figure 71 In the example shown in (A), at the far end Pz within the range Rx, the gap G1 of the dashed lines constituting the virtual scale M41 is set to the minimum value. Additionally, in Figure 71 In the example shown in (A), the virtual scale M41 is formed by dashed lines. However, only at the far end Pz can the gap G1 be set to 0, meaning the virtual scale M41 can be formed by solid lines. Figure 71 In the example shown in (B), when the center of Py is within the range Rx, the gap G2 of the dashed lines constituting the virtual scale M42 is set to a predetermined intermediate value. Figure 71 In the example shown in (C), in the case of the near end Px within the range Rx, the gap G3 of the dashed line constituting the virtual scale M43 is set to the maximum setting value.

[0282] As described above, the gaps G1 to G3 of the dashed lines that constitute the virtual scales M41 to M43 are set to values ​​that are inversely proportional to the observation distance, so that the magnitude of the dimensional error can be identified based on the gaps of the dashed lines.

[0283] Regardless of the viewing distance, the lines constituting the virtual scale are always made up of the same number. For example... Figure 72 As shown, the number of lines constituting the virtual scales M51 to M53 is varied according to the viewing distance. Additionally, in Figure 72 The virtual scales M51, M52, and M53 are shown for shooting at points Pz (far end), Py (near center), and Px (near end) within the observation distance range Rx.

[0284] exist Figure 72 In the example shown in (A), at the far end Pz within the range Rx, the virtual scale M51 is composed of three lines, that is, three concentric circles of different sizes. These three concentric circles represent, for example, actual dimensions of "5mm", "10mm", and "20mm". Figure 72 In the example shown in (B), near the center of Py within the range Rx, the virtual scale M52 is composed of two lines, that is, two concentric circles of different sizes. These two concentric circles, for example, represent actual dimensions of "5mm" and "10mm". Figure 72 In the example shown in (C), at the near end Px within the range Rx, the virtual scale M53 is composed of a single line, that is, a single circle. A single circle, for example, represents the actual size of "5mm". As described above, by setting the number of lines constituting the virtual scales M51 to M53 to a value proportional to the observation distance, the magnitude of the size error can be identified based on the number of lines.

[0285] like Figure 73 As shown, the signal processing unit 45 of the extended processor device 17 includes a position determination unit 92 including a distance calculation unit 94 and an image processing unit 104. The image processing unit 104 includes an image selection unit 95, a scale table 62, a virtual scale setting unit 105, a virtual scale switching receiver 106, and a length measurement image creation unit 107. The virtual scale setting unit 105 sets a virtual scale representing the actual size of the observed object on the subject based on the position of the light spot SP and has a scale with the end as the base point. The virtual scale switching receiver 106 receives commands to switch and set multiple virtual scales. The length measurement image creation unit 107 creates a length measurement image that overlaps the virtual scale set by the virtual scale setting unit 105 on the captured image in a manner where the position of the light spot SP overlaps with the base point of the scale of the virtual scale.

[0286] The functions of the virtual scale setting unit 105 and the length measurement image generation unit 107 will be explained below. For example... Figure 74As shown, a photographic image 109 obtained by illuminating a subject including polyp 108 is input to the signal processing unit 45. For example, polyp 108 has a spherical three-dimensional shape, so the photographic image 109 includes polyp 108, light spot SP, and shadow 110 depending on the situation.

[0287] The position determination unit 92 determines the position of the light spot SP based on the image 109 input to the signal processing unit 45. The virtual scale setting unit 105 refers to the scale table 62 and sets a virtual scale corresponding to the position of the light spot SP, which represents the actual size of the observed object and has a scale with the end as the base point. The end refers to the part or the starting point or ending point of the virtual scale that is closer to the outer part than the central part.

[0288] like Figure 75 As shown, the length measurement image generation unit 107 generates a length measurement image on the camera image 109 that overlaps the position of the light spot SP with the base point of the scale of the virtual scale 111, which is set by the virtual scale setting unit 105. For more accurate measurement, the virtual scale 111 is preferably overlapped at the position of the light spot SP. Therefore, even if it is displayed at a position separate from the light spot SP, it is preferable to display it as close as possible to the light spot SP. The virtual scale 111 is a line segment, and has line segments, i.e., scales, perpendicular to the line segment at the start and end points of the line segment. When the virtual scale 111 is a line segment or the like, and has a start and end point, the start and / or end point itself can be used as the scale. In this case, for example, there may be no scale in the shape of a line segment perpendicular to the line segment. Furthermore, the virtual scale 111 may have the number "10" near the base point of the scale. The number "10" is the scale label 111a of the virtual scale 111, indicating that the actual size of the line segment of the virtual scale 111 is 10mm, which can be easily identified. Hereafter, the numbers on the virtual scale have the same meaning. The value of the scale label 111a can be changed according to settings, and the virtual scale 111 can be one where the scale label 111a itself is not displayed.

[0289] Regarding virtual scales, various types can be used depending on the settings. For example, you can use line segments or combinations of line segments that are straight lines, circles or combinations of circles, or combinations of line segments and circles, etc.

[0290] like Figure 76As shown, for example, the photographic image 113 includes a virtual scale 112 composed of line segments in the shape of straight lines. The virtual scale 112 is formed by combining the line segments into an L-shape, with the corners of the L serving as base points, extending along the plane of the paper and in the direction of the paper, and each segment having a graduation at its endpoint, starting from the base point. Furthermore, similar to the virtual scale 111, the virtual scale 112 has a numerical value "10" as a graduation label 112a near the base point of the graduation.

[0291] like Figure 77 As shown, for example, the photographic image 114 includes a virtual scale 115 consisting of a combination of line segments and circles. The virtual scale 115 is a shape formed by combining a circle with a line segment that serves as the diameter of the circle, and the intersections of the line segments and the circle are designated as graduations. Graduations 116 may be present at points where the line segments are halved or at the center of the circle. Furthermore, similar to virtual scales 111 or 112, the virtual scale 115 has the number "10" as a graduation label 116a near the base point of the graduation. Graduation label 116b represents half of graduation label 116a.

[0292] like Figure 78 As shown, in addition to these, the virtual scale can also include, for example, a virtual scale 117 (including scale labels 117a) with line segments extending from the base point along the left side of the paper. Figure 78 (A) A virtual scale 118, including scale label 118a, extends from the base point along the underside of the paper. Figure 78 (B) or a virtual scale 119 including the scale label 119a, where the line segment extends from the base point in a diagonal upward direction along the right side of the paper. Figure 78 (C) and other shapes.

[0293] like Figure 79 As shown, the signal processing unit 45 of the extended processor device 17 includes a position determination unit 92, a reference scale setting unit 120, a measurement value scale generation unit 121, and a length measurement image generation unit 122. The reference scale setting unit 120 sets a reference scale representing the actual size of the subject based on the position of the light spot SP. The measurement value scale generation unit 121 generates a measurement value scale representing the measurement values ​​of the measurement portion of the area of ​​interest measured based on the set reference scale. Furthermore, both the reference scale and the measurement value scale are virtual scales displayed on the captured image, and therefore correspond to virtual scales.

[0294] The area of ​​interest refers to the region within the subject that the user should focus on. An area of ​​interest might be, for example, a polyp, and is an area where measurement is highly likely. Furthermore, the measurement portion refers to the part of the area of ​​interest where length, etc., is measured. For example, if the area of ​​interest is a reddish portion, the measurement portion is the longest part of the reddish portion; and if the area of ​​interest is circular, the measurement portion is the diameter of the area of ​​interest.

[0295] The length measurement image generation unit 122 generates a length measurement image that overlays a measurement scale onto the camera image. The measurement scale is overlaid on the camera image with the measurement portion of the area of ​​interest aligned with it. The length measurement image is displayed on the extended display 18.

[0296] like Figure 80 As shown, the reference scale setting unit 120 includes a reference scale table 121a. The reference scale table 121a contains correspondence information that establishes a relationship between the position of the light spot SP and measurement information corresponding to the actual size of the subject. In length measurement mode, an image 114 of the subject, including the polyp 123 as the object of observation, is captured and input to the signal processing unit 45. (As shown...) Figure 81 As shown, in the captured image 124, the polyp 123 has, for example, a three-dimensional shape like an overlapping sphere. For example, a spot SP is formed at the end of the polyp 123. The position determination unit 92 determines the position of the spot SP based on the captured image 124. The reference scale setting unit 120 refers to the reference scale table 121a and sets a reference scale 131 that corresponds to the determined position of the spot SP and represents the actual size of the subject.

[0297] The reference scale 131 is, for example, a line segment having a number of pixels corresponding to the actual size of 20 mm, as well as a numerical value and unit representing the actual size. The reference scale 131 is not normally displayed on the extended display 18, but when the reference scale 131 is displayed on the extended display 18, it is displayed as in the photographic image 124.

[0298] like Figure 82 As shown, the measurement scale generation unit 121 includes a region of interest extraction unit 125, a measurement portion determination unit 126, a measurement content receiving unit 127, and a measurement value calculation unit 128. Figure 83 As shown in the image 124, the region of interest extraction unit 125 extracts the region marked with a shaded line as the region of interest 129. Next, as... Figure 84 As shown, for example, when a pre-set reference is used as the reference for measuring a portion of the area of ​​interest in the horizontal direction with the spot SP as the base point, as in the photographed image 124, the measurement portion determination unit 126 extracts the horizontal edge position 130 with the spot SP as the base point. The area between the spot SP and the horizontal edge position 130 becomes the measurement portion.

[0299] For example, when the actual size of the reference scale is set to L0, the number of pixels of the reference scale 131 in the image 124 is set to Aa, the number of pixels of the measurement portion when the reference scale 131 is superimposed on the region of interest 129 in the image 124 is set to Ba, and the actual size of the measurement scale 132 is set to L1, the measurement calculation unit 128 generates the measurement scale 132 in a manner that satisfies the following formula (K1).

[0300] Equation (K1)L1=L0×Ba / Aa

[0301] like Figure 85 As shown, when the actual size of the reference scale 131 is 20 mm, based on the number of pixels Aa corresponding to the reference scale 131 shown in the photographed image 124a and the number of pixels Bb corresponding to the measurement portion between the spot SP shown in the photographed image 124b and the horizontal edge position 130, for example, when Ba / Aa is 0.7, as in the photographed image 124d, the measurement value calculation unit 128 calculates the actual size of the measurement value scale 132 as 13 mm.

[0302] The length measurement image generation unit 122 generates a length measurement image 133 that overlays the measurement scale 132 onto the camera image 124. For example, as... Figure 86 As shown, the measurement scale 132 is superimposed on the photographic image 1.24 as a straight line segment or arrow shape. The measurement image 133 may include the actual size value of the measurement scale 132. The actual size value of the measurement scale 132 may be superimposed on the photographic image 124 in a state separate from the arrow or other shapes.

[0303] The types of measurement scale 132 can be selected from multiple types. The measurement content receiving unit 127 receives the setting of the measurement scale content and sends the content to the measurement scale generation unit 121. The length measurement image generation unit 122 uses the measurement scale 132 generated by the measurement scale generation unit 121 based on the content to generate the length measurement image 133.

[0304] Furthermore, the region of interest extraction unit 125 preferably uses a learned model that has been trained from previously acquired camera images to extract the region of interest. Regarding the model used for the trained model, various models suitable for recognizing images through machine learning can be used. For the purpose of recognizing regions of interest in an image, a model using a neural network is preferred. When these models are trained, camera images containing information about regions of interest are used as teacher data for training. Information about regions of interest can include the presence or absence of a region of interest, its location, or its extent. Alternatively, training can be performed using camera images that do not contain information about regions of interest, based on the model.

[0305] Furthermore, the measurement portion determination unit 126 preferably uses a learned model that has been learned from previously acquired camera images to determine the measurement portion. The model used for the learned model is the same as that used in the region of interest extraction unit, but when these models are learned, they are learned from camera images containing information about the measurement portion. The information about the measurement portion includes the measurement value and the measurement portion itself. Alternatively, the model can be learned from camera images that do not contain information about the measurement portion. Furthermore, the learned model used by the region of interest extraction unit 125 and the learned model used by the measurement portion determination unit 126 can be interchangeable. For the purpose of extracting the measurement portion, a single learned model can be used to extract the measurement portion without extracting the region of interest from the camera image 124.

[0306] Furthermore, in the second signal processing unit 60, based on the representative point data table 66 which stores the illumination position of the measurement light and the representative points of the virtual scale, the scale table 62 (reference) used to display the virtual scale that is distorted according to the position of the light spot SP is updated. Figure 39 , Figure 40 However, scales can also be created using other methods, such as Table 62. Figure 87 As shown, from an image obtained by photographing a square grid-shaped chart, a distorted grid region QN surrounding a circular virtual scale centered on the light spot SP is obtained. Regarding the distorted grid region QN, as it separates from the center of the image, the grid is distorted due to the distortion aberrations of the camera optical system 21. The distorted grid region QN is transformed into... using an affine transformation matrix. Figure 88 The square grid region SQ is shown. Within the square grid region SQ, the coordinates of the points representing the circular virtual scale are calculated. Furthermore, by using the inverse of the affine transformation matrix, the coordinates of the points of the virtual scale in the square grid region SQ are transformed into a distorted circular virtual scale distorted by the camera optical system 21. The coordinates of this distorted circular virtual scale are then associated with the position of the light spot SP and stored in the scale table 62.

[0307] Furthermore, considering the distortion aberrations of the camera optical system 21, the display method of the virtual scale can be changed within the effective measurement area and in other areas based on the virtual scale. Specifically, such as... Figure 89 As shown in (a), when the spot SP exists outside the effective measurement area (near the Px side), and as... Figure 89 As shown in (c), when the spot SP exists outside the effective measurement area (distal Pz side), the measurement of tumor tm based on the virtual scale is invalid; therefore, the cross-shaped virtual scales MN and MF are shown respectively. On the other hand, as... Figure 89 As shown in (b), the circular virtual scale M is displayed when the spot SP exists in the area where the measurement based on the circular virtual scale M is effective.

[0308] Furthermore, the line type of the virtual scale can be changed depending on whether the light spot SP is within or outside the effective measurement area. At this time, such as... Figure 90 As shown, it is preferable to display the movement trajectory MT of the light spot SP in order to know the changes in the line type of the virtual scale. For example... Figure 90 As shown in (a), when the spot SP exists outside the effective measurement area (near the Px side), and as... Figure 90 As shown in (c), when the spot SP exists outside the effective measurement area (distal Pz side), the measurement of tumor tm based on the virtual scale is invalid, therefore circular virtual scales MpN and MpF are displayed respectively. On the other hand, as Figure 90 As shown in (b), when the spot SP exists within the effective measurement area based on the circular virtual scale Mp, the circular virtual scale Mp is displayed as a solid line. Furthermore, depending on whether the spot SP is outside or within the effective measurement area, the line type of the virtual scale is changed using dashed or solid lines, but it can also be set to other different colors. For example, when the spot SP is outside the effective measurement area, the line type of the virtual scale is set to blue, and when it is within the effective measurement area, the line type of the virtual scale is set to white.

[0309] The details of acquiring still images in length measurement mode are explained below. When no still image acquisition command is issued, the system control unit 41 controls the light source device 13 to emit illumination light and measurement light. For example... Figure 91 As shown, when a still image acquisition command command is executed by operating the still image acquisition command switch 12g, the illumination light is turned on (opened) during the first timing including the still image acquisition command, while the measurement light is turned off (closed). During the second and third timings after the first timing, the measurement light is turned on again while the illumination light remains on. The second and third timings can be set to the same timing, but they can also be set to different timings.

[0310] At the first timing, a second image can be obtained by photographing the subject illuminated by the measuring light. At the second and third timings, a first image can be obtained by photographing the subject illuminated by the illumination light and the measuring light. Furthermore, as... Figure 91 and Figure 92 As shown, the system control unit 41 saves still images of the first and second camera images as saved images in the still image storage unit 42. Furthermore, in the signal processing unit 45 of the extended processor device 17, a still image of the third camera image, which displays a virtual scale Mxm set according to the position of the light spot SP, is acquired. The still image of the third camera image is sent to the processor device 14 and saved in the still image storage unit 42. And, for a certain period of time after saving the still image, such as... Figure 93 As shown, the display control unit 46 displays the second and third camera images on the extended display 18 to notify that a still image has been recorded. Furthermore, it is preferable to save at least two of the first, second, and third camera images to the still image storage unit 42 using a single still image acquisition command. For example, it is preferable to save both the second and third camera images. And, as described above, the third camera image corresponds to the image stored in the still image storage unit 42 within the length measurement image that overlays a virtual scale.

[0311] In addition, such as Figure 94 As shown, the second or third timing can be performed before the first timing. In this case, during length measurement mode, several frames of the first camera image corresponding to the second or third timing need to be saved in the temporary storage unit (not shown) of the processor device 14. When a still image acquisition command is executed, the first camera image saved in the temporary storage unit is saved in the still image storage unit 42 as the first camera image for the second timing, and a third camera image with a virtual scale appended to the first camera image saved in the temporary storage unit is saved in the still image storage unit 42.

[0312] And, as Figure 95 As shown, the second and third timings can be set to different timings. In this case, the first camera image obtained at the second timing is saved in the still image storage unit 42 in the same manner as described above. Furthermore, before the first camera image obtained at the third timing is saved in the still image storage unit 42, a virtual scale is added to it to make it the third camera image, and then it is saved in the still image storage unit 42.

[0313] like Figure 96As shown, in the signal processing unit 45 of the extended processor device 17, in addition to the first signal processing unit 59 and the second signal processing unit 60, a lesion recognition unit 135, a diagnostic information acquisition unit 136, and a learning unit 137 may also be provided. The lesion recognition unit 135 performs image processing on the first camera image (an image based on illumination light and measurement light) and performs recognition processing. As part of the recognition processing performed by the lesion recognition unit 135, detection processing is performed to detect lesions and other regions of interest from the first camera image. A machine learning model is preferably used in the recognition processing. That is, for the first camera image as input to the learning model, the detection result of the region of interest is output from the learning model. The learning model is preferably a machine learning model such as a convolutional neural network (CNN). Furthermore, the recognition processing performed by the lesion recognition unit 135 may be a discrimination process that identifies the degree of lesion progression, etc., based on the lesions identified from the first camera image. Also, the lesion recognition unit 135 can perform image processing on the second camera image (an image based solely on illumination light) and perform recognition processing.

[0314] The diagnostic information acquisition unit 136 acquires diagnostic information related to the first or second camera image from the diagnostic information management device 138. As diagnostic information, it acquires the patient's medical record as the subject of the examination. The medical record refers to information documenting the patient's diagnosis, treatment, or examination process, such as the patient's name, gender, age, disease name, main symptoms, prescription or treatment details, or past medical history. The information about the lesion identified and processed by the lesion recognition unit 135, along with the diagnostic information related to the first or second camera image acquired by the diagnostic information acquisition unit 136, is stored as supplementary data to the dataset Ds by the still image storage unit 42, which establishes a link between the still image and the first or second camera image.

[0315] The learning unit 137 uses supplementary data (datasets) obtained by associating the first or second camera image stored in the still image storage unit 42 with these first and second camera images to perform machine learning. Specifically, the learning unit 137 performs machine learning on the learning model of the lesion recognition unit 135. The second camera image is preferred as a candidate for teacher data in machine learning. The second camera image is an image obtained from a still image acquisition command when measuring tumor tm, etc., and therefore is an image that is highly likely to include the area of ​​interest that is the object of observation. Furthermore, the second camera image is a typical endoscopic image without measurement light, so it is highly useful as teacher data for machine learning. In addition, information such as lesion and diagnostic information are also included as supplementary data, so the user does not need to input them when performing machine learning. By storing the second camera image as a candidate for teacher data, the accuracy of the lesion recognition unit 135's recognition processing is improved as machine learning is performed. In addition, when using the first camera image as a teacher data candidate for machine learning, the first camera image can be used directly, but it is more preferable to use the part other than the area illuminated by the measurement light as a teacher data candidate.

[0316] The following uses Figure 97 The calibration apparatus 200 shown illustrates a calibration method for creating a representative point data table 66. The calibration apparatus 200 includes a calibration display 201, a moving mechanism 202, a calibration display control unit 204, a calibration image acquisition unit 206, and a calibration unit 208. The calibration display control unit 204, the calibration image acquisition unit 206, and the calibration unit 208 are disposed within a calibration image processing apparatus 210. The calibration image processing apparatus 210 is electrically connected to the processor device 14, the calibration display 201, and the moving mechanism 202.

[0317] The moving mechanism 202 has a holding part (not shown) that holds the front end portion 12d of the endoscope 12 toward the calibration display 201. By moving the holding part at specific intervals, the distance Z between the front end portion 12d of the endoscope 12 and the calibration display 201 is changed. Each time the distance Z is changed by the moving mechanism 202, the calibration display control unit 204 displays an image of a virtual scale in the calibration display 201, which is unaffected by the imaging optical system 21, relative to the position of the measurement light. The image of the virtual scale in the first display mode does not take into account the effects of distortion caused by the imaging optical system 21, and therefore is not displayed in a size or shape corresponding to the scale display position when displayed on the extended display 18.

[0318] The calibration image acquisition unit 206 acquires calibration images obtained by taking pictures of the virtual scale displayed on the calibration display 201 in the first display mode through the endoscope 12. Regarding the calibration images, they are acquired by taking pictures by the endoscope 12 each time the distance Z is changed, that is, each time the virtual scale in the first display mode is displayed. For example, if the virtual scale in the first display mode is displayed n times, n calibration images can be obtained.

[0319] The calibration image includes a virtual scale image in a second display mode, which is affected by the imaging optical system 21 relative to the illumination position of the measurement light. The virtual scale image in the second display mode takes into account the effects of distortion and other factors caused by the imaging optical system 21, and is therefore displayed in a size or shape corresponding to the scale display position.

[0320] The calibration unit 208 performs calibration related to the display of the virtual scale on the extended display 18 based on the calibration image acquired by the calibration image acquisition unit 206. Specifically, in the calibration unit 208, a representative point data table is sent to the extended processor device 17 and stored in the representative point data table 66. This representative point data table is created by performing representative point extraction processing to extract representative points from the image of the virtual scale of the second display mode contained in the calibration image, and establishing a correlation between the representative point data related to the representative points and the illumination position at the timing of acquiring the calibration image.

[0321] like Figure 98 As shown, the inspection system 300 is used to check the accuracy of a virtual scale to ensure it has a specified shape. The inspection system 300 includes a test chart 302, a display 15, and a movement mechanism 304. The display 15 is shared with the endoscope system 10, but a separate display for accuracy checking can also be provided.

[0322] like Figure 99As shown, the test chart 302 has a chart body 305, which includes an inspection area 306 with a specific shape and an inspection reference position 308, which serves as a reference corresponding to the position of the measuring light when checking accuracy. The inspection area 306 has three circular inspection areas 306a, 306b, and 306c as inspection areas with a specific shape. These three inspection areas 306a, 306b, and 306c are arranged concentrically around the inspection reference position 308. For the inspection areas 306a, 306b, and 306c, inspection images are acquired by illuminating the chart body 205 with measuring light (e.g., light spot SP) from the endoscope 12 for confirming checks of virtual scales of 5mm (representing a diameter of "5mm"), 10mm (representing a diameter of "10mm"), and 20mm (representing a diameter of "20mm"), respectively.

[0323] like Figure 100 As shown, the inspection image is displayed on the monitor 15. In the inspection image, in addition to the inspection area 306 and the inspection reference position 308, a virtual scale M corresponding to the illumination position of the measuring light (the position of the light spot SP) is also displayed. When checking accuracy, the test chart 302 is moved by the moving mechanism 304 to align the illumination position of the measuring light (the position of the light spot SP) with the inspection reference position. When the illumination position of the measuring light aligns with the inspection reference position, the user judges whether the virtual scale M is displayed appropriately.

[0324] For example, in an inspection image, when the position of the measuring light coincides with the inspection reference position 308, and the virtual scale M is within the inspection area 306a, the user determines that the virtual scale M is appropriately displayed. In this regard, as... Figure 101 As shown, if a portion of the virtual scale M extends beyond the inspection area 306a, or if any part of the virtual scale M is not within the inspection area 306a, the user will determine that the 5mm virtual scale M is not being displayed appropriately.

[0325] Regarding Table 62 for the scale, it can be created as follows. The relationship between the position of the light spot and the size of the virtual scale can be obtained by photographing a chart with a pattern regularly formed to the actual size. For example, by shining a light spot-shaped measuring beam onto the chart and changing the position of the light spot by changing the viewing distance, the chart is photographed with grid lines of the same size as the actual size (5 mm) or smaller (e.g., 1 mm). The relationship between the position of the light spot (pixel coordinates on the imaging surface of the imaging element 32) and the number of pixels corresponding to the actual size (how many pixels represent the actual size of 5 mm).

[0326] like Figure 102 As shown, (x1, y1) represent the pixel positions of the light spot SP4 in the X and Y directions on the imaging surface of the imaging element 32 (the upper left is the origin of the coordinate system). The number of pixels in the X direction at the position (x1, y1) of the light spot SP4, corresponding to the actual size of 5mm, is set as Lx1, and the number of pixels in the Y direction is set as Ly1. This measurement is repeated while changing the observation distance. Figure 103 Showing the photos taken with Figure 102 The same 5mm grid line chart state, but for the same Figure 102 Compared to when the camera is closer to the far end, the grid lines appear narrower in this shot. Figure 103 In this state, the number of pixels in the X direction corresponding to the actual size of 5mm at the position (x2, y2) of the light spot SP5 on the imaging surface of the imaging element 32 is set to Lx2, and the number of pixels in the Y direction is set to Ly2. Furthermore, while changing the observation distance, this process is repeated... Figure 102 , 103 Such measurements are then used to plot the results. Additionally, in Figure 102 , 103 In the display, distortion and aberration of the camera optical system 21 are not considered.

[0327] Figure 104 This shows the relationship between the X coordinate of the light spot's position and Lx (the number of pixels in the X direction of the first virtual scale). Figure 105 This shows the relationship between the Y-coordinate of the light spot's position and Lx. Regarding Lx, according to... Figure 104 The relationship, as a function of position in the X direction, is expressed as Lx = g1(x), and, with respect to Ly, according to Figure 105 The relationship between them, as a function of position in the Y direction, is expressed as Ly = g2(y). g1 and g2 can be obtained from the above plotting results, for example, through the least squares method.

[0328] Furthermore, the X and Y coordinates of the light spot correspond one-to-one. Using either function g1 or g2 will yield essentially the same result (the same number of pixels for the same light spot position). Therefore, when calculating the size of the first virtual scale, any function can be used, or the function with higher sensitivity to changes in the number of pixels relative to position can be chosen. Moreover, if the values ​​of g1 and g2 differ significantly, it can be determined that "the position of the light spot cannot be identified."

[0329] Figure 106 This represents the relationship between the X coordinate of the light spot position and Ly (the number of pixels in the Y direction). Figure 107 This represents the relationship between the Y-coordinate of the light spot position and Ly. According to... Figure 106 The relationship is such that Ly, as the coordinate of the position in the X direction, is represented as Ly = h1(x), according to... Figure 108The relationship is such that Ly, as the coordinate of the position in the Y direction, is represented as Ly = h2(y). Similarly, Ly can also be represented using either function h1 or h2, just like Lx.

[0330] The functions g1, g2, h1, and h2 obtained in the above manner are stored in the scale table 62 in the form of a lookup table. Additionally, the functions g1 and g2 can be stored in the scale table 62 in function form.

[0331] Additionally, regarding the measurement light, when it illuminates the subject, such as... Figure 108 As shown, a striped pattern light ZPL (see, for example, Japanese Patent Application Laid-Open No. 2016-198304) can be used, which is light that forms a striped pattern on a subject. Regarding the striped pattern light ZPL, it is obtained by irradiating a specific laser onto a liquid crystal shutter (not shown) with variable transmittance, and is formed by two different vertical stripe patterns that are periodically repeated in the horizontal direction: a region (transmitting region) that transmits the specific laser through the liquid crystal shutter and a region (non-transmitting region) that does not transmit the specific laser. When using striped pattern light as the measuring light, since the period of the striped pattern light changes according to the distance from the subject, the striped pattern light is irradiated multiple times by shifting the period or phase of the striped pattern light through the liquid crystal shutter, and the three-dimensional shape of the subject is determined based on the multiple images obtained by shifting the period or phase.

[0332] For example, a striped pattern of light with phase X, phase Y, and phase Z is alternately applied to the subject. The striped patterns of phases X, Y, and Z cause a phase shift in the vertical stripe pattern every 120° (2π / 3). The three-dimensional shape of the subject is then determined using the three images obtained from each striped pattern. For example, as... Figure 1 As shown in .09, it is preferable to switch the striped pattern light of phase X, phase Y, and phase Z on the subject in units of one frame (or several frames). Furthermore, it is preferable to continuously illuminate the subject with the illumination light.

[0333] Additionally, regarding the measurement light, when it illuminates the subject, such as... Figure 110As shown, a measuring light LPL with a grid-like pattern can be used (for example, see Japanese Patent Application Laid-Open No. 2017-217215). In this case, the three-dimensional shape of the subject is determined based on the deformation state of the grid-like pattern when the measuring light LPL illuminates the subject, thus requiring accurate detection of the grid-like pattern. Therefore, the measuring light LPL with the grid-like pattern is not perfectly grid-like, but slightly deformed from a grid shape, such as in a wavy shape, to improve the detection accuracy of the grid-like pattern. Furthermore, S-strings indicating that the endpoints of the left and right horizontal line segments are continuous are provided in the grid-like pattern. When detecting the grid-like pattern, not only the pattern itself is detected, but also the S-strings, thereby improving the detection accuracy of the pattern. In addition to patterns formed by regularly arranged vertical and horizontal lines, the grid-like pattern can also be a pattern formed by arranging multiple light spots in a grid pattern in both the longitudinal and transverse directions.

[0334] When using a grid-patterned measuring light LPL as the measuring light, in length measurement mode, both the illumination light and the grid-patterned measuring light LPL can be continuously illuminating the subject, and, as... Figure 111 As shown, the illumination light continuously shines on the subject, while the grid-patterned measurement light LPL is repeatedly turned on and off (or dimmed) every frame (or every few frames), thereby intermittently illuminating the subject with the grid-patterned measurement light LPL. During this time, in the frames where the grid-patterned measurement light LPL is illuminated, a three-dimensional shape measurement based on the grid-patterned measurement light LPL is performed. Then, preferably, the measurement results of the three-dimensional shape are overlaid on the image obtained in the frames where only the illumination light is illuminating.

[0335] Additionally, regarding the measurement of light, such as Figure 112 As shown, a three-dimensional planar light TPL represented by grid lines on the subject image can be used (for example, refer to Japanese Patent Application Publication No. 2017-508529). At this time, the front end is moved 12d to match the three-dimensional planar light TPL with the object being measured. Then, when the three-dimensional planar light TPL intersects with the object being measured, the distance between the three-dimensional planar light TPL and the cross curve CC of the subject is calculated through manual or automatic processing based on a user interface or similar method.

[0336] When using a three-dimensional planar light (TPL) as the measurement light, in length measurement mode, the illumination light and the TPL can be continuously illuminating the subject, and, as... Figure 113 As shown, the illumination light continuously shines on the subject, while the three-dimensional planar light TPL is repeatedly turned on and off (or dimmed) every frame (or every few frames), thereby intermittently illuminating the subject with the three-dimensional planar light TPL.

[0337] In the above embodiments, the hardware structure of the receiving unit 38, signal processing unit 39, display control unit 40, system control unit 41, still image storage unit 42, data transceiver unit 43, data transceiver unit 44, signal processing unit 45, display control unit 46 (including various control units or processing units provided in these control units, such as length measurement mode control unit 50, first signal processing unit 59, etc.) that perform various processes is as shown below. Among the various processors are general-purpose processors that execute software (programs) and function as various processing units, i.e., CPUs (Central Processing Units), processors whose circuit structure can be changed after manufacturing, such as FPGAs (Field Programmable Gate Arrays), i.e., programmable logic devices (PLDs), and processors with circuit structures specifically designed for performing various processes, i.e., dedicated circuits.

[0338] A processing unit can be composed of one of these various processors, or it can be composed of a combination of two or more processors of the same or different types (e.g., multiple FPGAs, a combination of CPUs and FPGAs). Furthermore, a single processor can also constitute multiple processing units. As examples of a single processor constituting multiple processing units, firstly, as exemplified by client and server computers, a processor is composed of a combination of one or more CPUs and software, and this processor functions as multiple processing units. Secondly, as exemplified by System-on-Chip (SoC), a processor is used to implement the functions of the entire system including multiple processing units using a single IC (Integrated Circuit) chip. Thus, as a hardware structure, various processing units are constructed using one or more of the aforementioned processors.

[0339] Furthermore, more specifically, the hardware structure of these various processors is a circuit composed of combined semiconductor components and other circuit elements. And the hardware structure of the storage section is a storage device such as an HDD (hard disk drive) or an SSD (solid state drive).

[0340] Symbol Explanation

[0341] 10-Endoscope system, 12-Endoscope, 12a-Insertion section, 12b-Operating section, 12c-Bend section, 12d-Anterior end section, 12f-Observation mode switching switch, 12g-Still image acquisition command switch, 12h-Zoom operating section, 13-Light source device, 14-Processor device, 15-Display, 16-User interface, 17-Extended processor device, 18-Extended display, 18a-Supplementary information display area, 18b-Observation image display area, 19-Balloon, 19a-Anterior end section, 19b-Base end section, 19c-Protrusion, 20a, 20b-Rings, 21-Camera optical system, 21a-Objective lens, 21b-Zoom lens, 21c-Anterior end face, 22-Illumination optical system, 22a - Illumination lens, 22b- Front end face, 23- Measuring light emitting part, 23a- Light source, 23b-DOE, 23c- Prism, 24- Opening, 25- Air and water supply nozzle, 25a- Jet tube part, 25b- Jet port, 26- Intestine, 27- Front end cover, 27a, 27b, 27c, 27d- Through hole, 28- Front end face, 28a, 28b- Plane, 30- Light source, 31- Light source processor, 32- Imaging element, 33- Camera control unit, 34- CDS / AGC circuit, 35- A / D converter, 36- Communication I / F, 37- Communication I / F, 38- Receiver, 39- Signal processing unit, 40- Display control unit, 41- System control unit, 42- Still image storage unit, 4 3-Data transceiver unit, 44-Data transceiver unit, 45-Signal processing unit, 46-Display control unit, 47-Receiving unit for measuring light emission unit, 48-Transparent cover, 49-Prism, 50-Length measurement mode control unit, 53-Brightness information calculation unit, 54-Illumination brightness level setting unit, 55-First light emission control meter, 56-Second light emission control meter, 57-Vertical line, 58-Diagonal line, 59-First signal processing unit, 60-Second signal processing unit, 61-Illumination position detection unit, 62-Scale meter, 64-Meter update unit, 66-Representative point data table, 67-Cross line, 68-Scale, 69-Position determination unit, 70-Measurement information processing unit, 71-Measurement information, 72-Position determination unit, 73-Image processing unit The following are the components of the signal processing unit: 74-Interference Component Removal Unit, 75-Color Information Conversion Unit, 76-Binarization Processing Unit, 77-Mask Image Generation Unit, 78-Removal Unit, 79-Measured Light Color Information, 80-Color Information of Interference Components, 81-Region of Interference Components, 82-Image Selection Unit, 84-First Signal Processing Unit, 85-Second Signal Processing Unit, 86-Mask Processing Unit, 87-Binarization Processing Unit, 88-Interference Component Removal Unit, 89-Illumination Position Detection Unit, 90-Illumination Area Recognition Unit, 91-Learning Model, 92-Position Determination Unit, 93-Image Processing Unit, 94-Distance Calculation Unit, 95-Image Selection Unit, 97-Offset Setting Unit, 98-Offset Distance Calculation Unit, 99-Offset Virtual Scale Generation Unit.100-Polyp, 100a-Top, 100b-Flat area, 101-Extended surface, 101X-Solid line, 102, 103-Parallel surface, 102X-Dash line, 104-Image processing unit, 105-Virtual scale setting unit, 106-Virtual scale switching receiving unit, 107-Length measurement image production unit, 108-Polyp, 109-Camera image, 110-Shadow, 111, 112-Virtual scale, 111a-Scale label, 113, 114-Camera image, 115-Virtual scale, 116-Scale, 116a-Scale label, 116h-Scale label, 118-Virtual scale, 118a-Scale label, 119-Virtual scale, 119a-Scale label, 12 0-Reference scale setting unit, 121-Measurement value scale generation unit, 121a-Reference scale table, 122-Length measurement image generation unit, 123-Polycyst, 124-Camera image, 125-Region of interest extraction unit, 126-Measurement part determination unit, 127-Measurement content receiving unit, 128-Measurement value calculation unit, 129-Region of interest, 130-Horizontal edge position, 131-Reference scale, 132-Measurement value scale, 133-Length measurement image, 135-Lysmal lesion identification unit, 136-Diagnostic information acquisition unit, 137-Learning unit, 138-Diagnostic information management device, 140-Determination unit for whether the length measurement corresponds to the endoscope, 141-Measurement light on / off switching unit, 142-Length measurement image display unit 143 - Length Measurement Function Operation Status Display On / Off Switch; 144 - Virtual Scale Display Switch Control; 146 - Icon in Scale Display; 147 - Virtual Scale; 147a, 147b, 147c - Virtual Scale; 148 - Icon when Scale is Not Displayed; 149 - Image Display Setting Saving Unit Before Switching; 200 - Calibration Device; 201 - Calibration Display; 202 - Moving Mechanism; 204 - Calibration Display Control Unit; 206 - Calibration Image Acquisition Unit; 208 - Calibration Unit; 210 - Calibration Image Processing Device; 300 - Inspection System; 302 - Test Chart; 304 - Moving Mechanism Unit; 305 - Chart Body; 306 - Inspection Area Unit. 306a, 306b, 306c - Inspection area; 308 - Inspection reference position; Aa, Ba - Pixel count; Ax - Optical axis; BLC - Balloon control device; BF - Blue filter; CL1 - First feature line; CL2 - Second feature line; cc - Cross curve; CR1 - White center area; D1 - First direction; D2 - Second direction; D3 - Third direction; D5 - Distance; D6 - Offset distance; DS1 - Diffraction spot; DT - Spacing; EP - Measurement point; G1, G2, G3 - Gap; GF - Green filter; HT1, HT2 - Height; LG - Light guide; Ls, Lt - Line; Lm - Measurement light; Lx1, Lx2 - Pixel count in the X direction; Ly1, Ly2 - Pixel count in the Y direction.LPL - Measuring light with a grid pattern; M - Circular virtual scale; M1, M2, M3 - Virtual scale; M11, M12, M13, M14, M15 - Virtual scale; M21, M22, M23 - Concentric circles; M41, M42, M43 - Virtual scale; M4A, M4B, M4C, M5A, M5B, M5C - Virtual scale; M51, M52, M53 - Virtual scale; M6A, M6B, M6C - Distorted concentric circle virtual scale; MN, MF - Cross-shaped virtual scale; MpN, Mp, MpF - Circular... Virtual scale bar, MT - movement trajectory, Mx - scale, Mxm - virtual scale bar, N1 - first interference component, N2 - second interference component, P - polyp, P1, P2, P3 - location, Px - proximal, Py - near center, Pz - distal, RP, PRx - red image, Pry - binarized red image, GP, PGx - green image, PGy - binarized green image, BP, PBx - blue image, PBy - binarized blue image, PD1 - first difference image, Qx, Qy, Qz - arrows, QN - distorted grid region, RP, RP... * - Representation point, RF - Red filter, SCC - Specific cross curve, SP - Spot, SP1, SP2, SP3, SP4, SP5 - Spot, SPk1 - First spot, SPk2 - Second spot, SQ - Square grid area, SR1 - Peripheral area, tm, tm1, tm2, tm3, tm4, tm5 - Tumor, TPL - Three-dimensional planar light, W11, W12, W13, W14, W15 - Width, W21, W22, W23 - Width, Wx - Movable range of illumination position, WMB - Wavelength region of measurement light, ZPL - Striped pattern light.

Claims

1. An endoscope system comprising: Endoscope; and The processor device includes an image control processor. When the endoscope is connected to the processor device, the image control processor reads the observer ID from the endoscope and determines whether the endoscope is the length-measuring endoscope based on the observer ID. When the endoscope is the endoscope corresponding to the length measurement, the switching to the length measurement mode is made effective. When the switching to the length measurement mode is enabled, the measurement light is switched on or off by switching to the length measurement mode. Referring to the representative point data table, create a virtual scale image corresponding to the model and / or serial number of the endoscope. The virtual scale image is superimposed on the image captured by the endoscope, and the image with the superimposed virtual scale image is displayed on the monitor.

2. The endoscope system according to claim 1, wherein, When the endoscope is the endoscope corresponding to the length measurement, the endoscope can illuminate the measurement light and display a length measurement image based on a virtual scale of the measurement light on the display screen. When the switching to the length measurement mode is enabled, the image control processor performs at least one of the following switching operations: switching the length measurement image display settings related to the length measurement image to be turned on or off; switching the display indicating the length measurement function operation status of the virtual scale on the display to be turned on or off; and switching the display of the virtual scale to be turned on, off, or changing the display mode. The change of the display mode of the virtual scale is performed by selecting from multiple scale patterns.

3. The endoscope system according to claim 2, wherein, The image control processor switches the measurement light to on, the length measurement image display setting to on, the length measurement function operation status display to on, and the virtual scale display to on by switching to the length measurement mode.

4. The endoscope system according to claim 3, wherein, During the switching operation to the length measurement mode, if the mode switching conditions are not met, the image control processor shall prevent the measurement light from being switched to on, the length measurement image display setting from being switched to on, the length measurement function operation status display from being switched to on, and the virtual scale display from being switched to on.

5. The endoscopic system according to claim 4, wherein, Instead of disabling the display of the length measurement function's operation status, the option to set the virtual scale as "not displayed" to "on" will be enabled.

6. The endoscope system according to claim 3, wherein, When the image control processor sets the length measurement image display setting to "on", it saves the image display settings before switching to the length measurement mode.

7. The endoscope system according to claim 2, wherein, The image control processor switches the measuring light off, the length measurement image display setting off, the length measurement function operation status display off, and the virtual scale display off by switching from the length measurement mode to other modes.

8. The endoscope system according to claim 2 or 7, wherein, When the length measurement image display setting is turned off, the image control processor switches to the image display setting saved before switching to the length measurement mode.

9. A method of operating an endoscope system, the endoscope system comprising: an endoscope; and a processor device having an image control processor. In the working method of the endoscopic system, When the endoscope is connected to the processor device, the image control processor reads the observer ID from the endoscope and determines whether the endoscope is the length-measuring endoscope based on the observer ID. When the endoscope is the endoscope corresponding to the length measurement, the switching to the length measurement mode is made effective. When the switching to the length measurement mode is enabled, the measurement light is switched on or off by switching to the length measurement mode. Referring to the representative point data table, create a virtual scale image corresponding to the model and / or serial number of the endoscope. The virtual scale image is superimposed on the image captured by the endoscope, and the image with the superimposed virtual scale image is displayed on the monitor.

Images