Endoscope system and working method thereof

Abstract

The present invention provides an endoscope system and an operating method thereof that can stably detect an irradiation area of ​​a measuring light beam. A receiving unit (38) acquires a subject image including a light spot (SP) that appears in the subject due to irradiation of the measuring light beam. An irradiation area recognition unit (58) recognizes the position of the light spot (SP) from the subject image. The irradiation area recognition unit (58) recognizes the light spot (SP) having a circular pattern including a white central area (CR1) and a peripheral area (SR1) covering the periphery of the central area (CR1) and having a characteristic amount based on the measuring light beam.

Description

Technical Field

[0001] The present invention relates to an endoscope system for displaying a measurement mark for measuring the size of a subject and an operating method thereof. Background Art

[0002] In an endoscope system having a light source device, an endoscope, and a processor device, the distance to a subject or the size of the subject is acquired. For example, in Patent Document 1, illumination light and a measuring light beam are irradiated to a subject, and a light beam irradiation area such as a light spot appears on the subject by the irradiation of the light beam. In addition, a measuring mark for measuring the size of the subject is displayed on the subject image corresponding to the position of the light spot.

[0003] Previous technical literature

[0004] Patent Literature

[0005] Patent Document 1: International Publication No. 2018 / 051680 Summary of the invention

[0006] Technical issues to be solved by the invention

[0007] As a method for accurately detecting the position of the light spot from the subject image, there is a color separation method such as RGB / HSV that uses the color / brightness / chroma of the image. However, when the subject includes diffuse reflections in the living body, residues, or deteriorated (discolored) tissues, the color of the image changes due to these diffuse reflections in the living body, so it is sometimes difficult to detect the position of the light spot. Therefore, even in the case of diffuse reflections in the living body, it is required to stably detect the irradiation area of ​​the measurement light beam such as the position of the light spot.

[0008] An object of the present invention is to provide an endoscope system and an operating method thereof that can stably detect an irradiation area of ​​a measuring light beam.

[0009] Means for solving technical problems

[0010] The endoscope system of the present invention comprises: an endoscope having a beam light emitting portion for irradiating a subject with a measuring beam light, and an imaging optical system arranged at a front end portion at a position different from the beam light emitting portion and receiving light from the subject, the measuring beam light being emitted obliquely relative to the optical axis of the imaging optical system; and a processor device, the processor device having a processor, the processor performing the following processing: acquiring a subject image obtained based on the light received by the imaging optical system, the subject image including a beam irradiation area that appears in the subject by irradiation with the measuring beam light; identifying the beam irradiation area from the subject image; and displaying a measuring image on a display in which a measuring mark for measuring the size of the subject is displayed on the subject image based on the beam irradiation area, the beam irradiation area in the subject image having a pattern of a specific shape, the pattern of the specific shape including a white central area and a peripheral area covering the periphery of the central area and having a characteristic amount based on the measuring beam light, the processor identifying the beam irradiation area having the pattern of the specific shape.

[0011] Preferably, the processor identifies the beam irradiation area by outputting a learning model of the beam irradiation area for an input of a subject image. Preferably, the processor identifies the beam irradiation area by performing pattern matching processing on the subject image using a template image of a predetermined beam irradiation area.

[0012] Preferably, the processor can identify the beam irradiation area of ​​the pattern of specific shape deformation. Preferably, the feature quantity is at least one of the color of the measuring beam, the color other than the color of the measuring beam, the brightness, lightness, chroma, and hue of the measuring beam.

[0013] Preferably, the specific shape is a circle. Preferably, the specific shape is a line. Preferably, the processor receives a subject image including a plurality of color images, sets the light amount of the measuring light beam to the specific light amount, and sets the pixel value of each color image to the maximum pixel value, thereby making the central area of ​​the light beam irradiation area white.

[0014] Preferably, the processor displays a measurement mark corresponding to the position of the light beam irradiation area on the measurement image. Preferably, the processor displays a measurement mark corresponding to the pattern of the light beam irradiation area on the measurement image.

[0015] The present invention is a working method of an endoscope system, the endoscope system comprising: an endoscope having a beam light emitting portion for irradiating a subject with a measuring beam light, and an imaging optical system arranged at a front end portion at a position different from the beam light emitting portion and receiving light from the subject, the measuring beam light being emitted obliquely relative to the optical axis of the imaging optical system; and a processor device having a processor, in the working method of the endoscope system, the processor comprising the following steps: acquiring a subject image obtained based on the light received by the imaging optical system, the subject image including a beam irradiation area that appears in the subject by irradiation with the measuring beam light; identifying the beam irradiation area from the subject image; and displaying a measuring image on a display in which a measuring mark for measuring the size of the subject is displayed on the subject image according to the beam irradiation area, the beam irradiation area in the subject image having a pattern of a specific shape, the pattern of the specific shape including a white central area and a peripheral area covering the periphery of the central area and having a characteristic amount based on the measuring beam light, the processor identifying the beam irradiation area having the pattern of the specific shape.

[0016] Effects of the Invention

[0017] According to the present invention, the irradiation area of ​​the measuring beam light can be stably detected by recognizing the beam irradiation area formed on the object by the measuring beam light as a pattern of a specific shape including a white central area and a peripheral area. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 This is an external view of the endoscope system.

[0019] Figure 2 It is a top view showing the front end portion of the endoscope.

[0020] Figure 3 It is a block diagram showing the functions of the endoscope system.

[0021] Figure 4 Graphs showing transmission distributions of the red filter RF, the green filter GF, and the blue filter BF.

[0022] Figure 5 This is a block diagram showing the function of the light beam emitting unit.

[0023] Figure 6 It is an image diagram showing the light spot SP and the measurement mark.

[0024] Figure 7 It is an explanatory diagram showing the relationship between the optical axis of the measuring light beam and the optical axis of the imaging optical system.

[0025] Figure 8 It is a block diagram showing the functions of the signal processing unit.

[0026] Fig. 9 1 and 2 are explanatory diagrams showing the central region CR1 and the peripheral region SR1 of the light spot SP.

[0027] Fig.10 It is a graph showing the distribution of pixel values ​​of the red image RP, the green image GP, and the blue image BP.

[0028] Fig.11 3 is a graph showing the transmittance of the measuring light beam in the wavelength region WMB.

[0029] Fig.12 This is a block diagram showing the functions of an irradiation region recognition unit having a learning model.

[0030] Fig.13 1 is an explanatory diagram showing a pattern of a light spot SP deformed from a circular shape.

[0031] Fig.14 This is a block diagram showing the functions of the irradiation region recognition unit including the pattern matching processing unit.

[0032] Fig.15 This is an image diagram showing the light spot and the first measurement mark when the observation distance is the near end Px.

[0033] Fig.16 This is an image diagram showing the light spot and the first measurement mark when the observation distance is near the center Py.

[0034] Fig.17 This is an image diagram showing the light spot and the first measurement mark when the observation distance is the far end Pz.

[0035] Fig.18 It is an explanatory diagram showing a first measuring mark of a cross, a cross with scale, a distorted cross, a circle and a cross, and a measuring point group type.

[0036] Fig.19 This is an image diagram showing three concentric circle marks of the same color.

[0037] Fig. 20 This is an image diagram showing three concentric circle marks of different colors.

[0038] Fig.21 This is an image diagram showing distorted concentric circle marks.

[0039] Fig. 22 This is a graphic diagram showing crosshairs and scales.

[0040] Fig.23 It is an explanatory diagram showing the center region CR2 and the peripheral region SR2 of the intersection line.

[0041] Fig.24 2 is an explanatory diagram showing the diameter DM of the central region CR1 of the light spot SP.

[0042] Fig.25 It is a flowchart showing a series of processes of the present invention. DETAILED DESCRIPTION

[0043] like Figure 1 As shown in FIG. 1 , the endoscope system 10 includes an endoscope 12, a light source device 14, a processor device 16, a display 18, and a user interface 19. The endoscope 12 includes an insertion portion 12a to be inserted into a subject, an operation portion 12b provided at a base end portion of the insertion portion 12a, and a universal cable 12c. The universal cable 12c is a light guide 28 (see FIG. 1 ) for guiding illumination light emitted by the light source device 14. Figure 3 ), a control line for transmitting control signals for controlling the endoscope 12, a signal line for transmitting image signals obtained by photographing the observation object, and a power line for supplying power to each part of the endoscope 12. A connector 29 connected to the light source device 14 is provided at the front end of the universal cable 12c.

[0044] The light source device 14 generates illumination light using, for example, a semiconductor light source such as an LED (Light Emitting Diode), an LD (Laser Diode), or a halogen lamp such as a xenon lamp. When the connector 29 is connected to the light source device 14, the illumination light is incident on the light guide 28 of the connector 29 (see Figure 3 ), and irradiates the observation object from the front end of the insertion part 12a.

[0045] Furthermore, the light source device 14 is electrically connected to the processor device 16, and the connector 29 of the endoscope 12 is connected to the processor device 16 via the light source device 14. The transmission and reception of control signals, image signals, etc. between the light source device 14 and the connector 29 is wireless communication. Therefore, the light source device 14 transmits the control signals, etc. transmitted and received with the connector 29 to the processor device 16 in a wireless manner. Furthermore, the light source device 14 supplies power for driving the endoscope 12 to the connector 29, but the supply of this power is also performed in a wireless manner.

[0046] The processor device 16 controls the light amount and light emission timing of the illumination light emitted by the light source device 14 and each part of the endoscope 12, and generates an endoscopic image by using an image signal obtained by photographing the observation object irradiated with the illumination light. In addition, the processor device 16 is electrically connected to a display 18 and a user interface 19. The display 18 displays the endoscopic image generated by the processor device 16, information related to the endoscopic image, etc. The user interface 19 has a function of receiving input operations such as function settings.

[0047] The endoscope 12 has a normal observation mode, a special observation mode, and a length measurement mode, and these three modes are switched by a mode switching switch 13a provided on the operating portion 12b of the endoscope 12. The normal observation mode is a mode in which the observation object is illuminated by illumination light. The special observation mode is a mode in which the observation object is illuminated by special light different from the illumination light. In the length measurement mode, the observation object is illuminated by illumination light and a measuring beam light, and a measuring mark for measuring the size of the observation object, etc. is displayed on the subject image obtained by photographing the observation object. In addition, the illumination light is light for giving brightness to the observation object as a whole to observe the observation object as a whole. The special light is light for emphasizing a specific area in the observation object. The measuring beam light is light for displaying a measuring mark.

[0048] Furthermore, the operation unit 12b of the endoscope 12 is provided with a freeze switch 13b for operating a still image acquisition command, which is a command for acquiring a still image of the subject image. When the user operates the freeze switch 13b, the screen of the display 18 is frozen, and a warning sound (e.g., "beep") is emitted to indicate that the still image acquisition is being performed. Then, the still image of the subject image obtained before and after the operation timing of the freeze switch 13b is stored in the still image storage unit 42 in the processor device 16 (refer to Figure 3 ). In addition, the still image storage unit 42 is a storage unit such as a hard disk or a USB (Universal Serial Bus) memory. When the processor device 16 can be connected to a network, a still image of the subject image can be stored in a still image storage server (not shown) connected to the network instead of or in addition to the still image storage unit 42.

[0049] In addition, a still image acquisition command can be issued using an operating device other than the freeze switch 13b. For example, a foot pedal can be connected to the processor device 16, and a still image acquisition command can be issued when the user operates the foot pedal (not shown) with his foot. The mode switching can be performed using the foot pedal. In addition, a gesture recognition unit (not shown) that recognizes the user's gesture can be connected to the processor device 16, and a still image acquisition command can be issued when the gesture recognition unit recognizes a specific gesture performed by the user. The mode switching can be performed using the gesture recognition unit.

[0050] Furthermore, a sight line input unit (not shown) provided near the display 18 may be connected to the processor device 16, and when the sight line input unit recognizes that the user's sight line stays within a specified area in the display 18 for more than a certain period of time, a still image acquisition command may be issued. Furthermore, a voice recognition unit (not shown) may be connected to the processor device 16, and when the voice recognition unit recognizes a specific voice uttered by the user, a still image acquisition command may be issued. Regarding mode switching, this may be performed using the voice recognition unit. Furthermore, an operation panel such as a touch panel (not shown) may be connected to the processor device 16, and when the user performs a specific operation on the operation panel, a still image acquisition command may be issued. Regarding mode switching, this may be performed using the operation panel.

[0051] like Figure 2 As shown, the front end portion 12d of the endoscope 12 is roughly circular and is provided with an imaging optical system 21 for receiving light from the subject, an illumination optical system 22 for irradiating illumination light to the subject, a light beam emitting portion 23 for radiating a measurement light beam to the subject, an opening 24 for allowing a treatment instrument to protrude toward the subject, and an air and water supply nozzle 25 for supplying air and water.

[0052] The optical axis Ax of the imaging optical system 21 extends in a direction perpendicular to the paper surface. The longitudinal first direction D1 is orthogonal to the optical axis Ax, and the lateral second direction D2 is orthogonal to the optical axis Ax and the first direction D1. The imaging optical system 21 and the light beam emitting unit 23 are respectively provided at different positions of the front end portion 12d and arranged along the first direction D1.

[0053] like Figure 3 As shown, the light source device 14 includes a light source unit 26 and a light source control unit 27. The light source unit 26 generates illumination light or special light for illuminating the subject. The illumination light or special light emitted from the light source unit 26 enters the light guide 28 and is irradiated to the subject through the illumination lens 22a. As the light source unit 26, as a light source of illumination light, a white light source emitting white light, or a plurality of light sources including a white light source and a light source emitting light of other colors (for example, a blue light source emitting blue light) can be used. In addition, as the light source unit 26, as a light source of special light, a light source emitting broadband light including a blue narrow-band light for emphasizing surface information such as surface blood vessels can be used. The light source control unit 27 is connected to the system control unit 41 of the processor device 16. In addition, as illumination light, a white mixed color light composed of a combination of blue light, green light, and red light can be used. In this case, it is preferable to perform the optical design of the illumination optical system 22 in such a way that the irradiation range of the green light is larger than the irradiation range of the red light.

[0054] The light source control unit 27 controls the light source unit 26 according to the command from the system control unit 41. In addition to issuing commands related to light source control to the light source control unit 27, the system control unit 41 also controls the light source 23a (refer to Figure 5 ). In the case of the normal observation mode, the system control unit 41 controls the lighting of the illumination light and the extinguishing of the measurement beam light. In the case of the special observation mode, the system control unit 41 controls the lighting of the special light and the extinguishing of the measurement beam light. In the case of the length measurement mode, the system control unit 41 controls the lighting of the illumination light and the extinguishing of the measurement beam light.

[0055] The illumination optical system 22 includes an illumination lens 22a, and light from the light guide 28 is irradiated to the observation object through the illumination lens 22a. The imaging optical system 21 includes an objective lens 21a, a zoom lens 21b, and an imaging element 32. Reflected light from the observation object is incident on the imaging element 32 through the objective lens 21a and the zoom lens 21b. As a result, a reflected image of the observation object is formed in the imaging element 32.

[0056] The zoom lens 21b has an optical zoom function of enlarging or reducing the subject as a zoom function by moving between the telephoto end and the wide-angle end. The optical zoom function can be turned on and off by the zoom operation unit 13c (see FIG. 14 ) provided on the operation unit 12b of the endoscope. Figure 1 ) is switched, and when the optical zoom function is turned on, the zoom operation unit 13c is further operated to enlarge or reduce the subject at a specific magnification.

[0057] The imaging element 32 is a color camera sensor that captures the reflected image of the subject and outputs an image signal. The imaging element 32 is preferably a CCD (Charge Coupled Device) camera sensor, a CMOS (Complementary Metal-Oxide Semiconductor) camera sensor, or the like. The imaging element 32 used in the present invention is a color camera sensor for obtaining red images, green images, and blue images of three colors, namely R (red), G (green), and B (blue). The red image is an image output from a red pixel having a red filter in the imaging element 32. The green image is an image output from a green pixel having a green filter in the imaging element 32. The blue image is an image output from a blue pixel having a blue filter in the imaging element 32. The imaging element 32 is controlled by the camera control unit 33.

[0058] like Figure 4As shown, the red filter RF has a high transmittance in the red band of 600 to 700 nm, so the red pixel has a high sensitivity in the red band. The green filter GF has a high transmittance in the green band of 500 to 600 nm, so the green pixel has a high sensitivity in the green band. The blue filter BF has a high transmittance in the blue band of 400 to 500 nm, so the blue pixel has a high sensitivity in the blue band. In addition, the red pixel also has sensitivity in the green band or the blue band. And, the green pixel also has sensitivity in the red band or the blue band. And, the blue pixel also has sensitivity in the red band or the green band.

[0059] The image signal output from the imaging element 32 is sent to the CDS / AGC circuit 34. The CDS / AGC circuit 34 performs correlated double sampling (CDS (Correlated Double Sampling)) and automatic gain control (AGC (Auto Gain Control)) on the image signal as an analog signal. The image signal passed through the CDS / AGC circuit 34 is converted into a digital image signal through an A / D converter (A / D (Analog / Digital: analog / digital) frequency converter) 35. The digital image signal after A / D conversion is input to the communication I / F (Interface) 37 of the light source device 14 via the communication I / F (Interface) 36.

[0060] Programs related to various processes are loaded into a program memory (not shown) in the processor device 16. The system control unit 41 composed of a processor executes the programs in the program memory, thereby realizing the functions of the receiving unit 38, the signal processing unit 39 and the display control unit 40.

[0061] The receiving unit 38 is connected to the communication I / F (Interface) 37 of the light source device 14. The receiving unit 38 receives the image signal transmitted from the communication I / F 37 and transmits it to the signal processing unit 39. The signal processing unit 39 has a built-in memory for temporarily storing the image signal received from the receiving unit 38, and processes the collection of image signals stored in the memory, that is, the image signal group, to generate a subject image. In addition, the receiving unit 38 can directly send the control signal associated with the light source control unit 27 to the system control unit 41. In addition, with respect to the processing unit or storage unit associated with the length measurement mode in the processor device 16 (for example, the irradiation area recognition unit 58 or the marking table 62), a length measurement processor (not shown) different from the processor device 16 can be provided. At this time, the length measurement processor and the processor device 16 are set in advance to be able to communicate with each other so that images or various information can be sent and received.

[0062] In the signal processing unit 39, when the normal observation mode is set, a signal distribution process is performed to distribute the blue image of the subject image to the B channel of the display 18, the green image of the subject image to the G channel of the display 18, and the red image of the subject image to the R channel of the display 18, thereby displaying the subject image in color on the display 18. The same signal distribution process as that in the normal observation mode is also performed in the length measurement mode. On the other hand, in the signal processing unit 39, when the special observation mode is set, the red image of the subject image is not used for display on the display 18, the blue image of the subject image is distributed to the B channel and the G channel of the display 18, and the green image of the subject image is distributed to the R channel of the display 18, thereby displaying the subject image in pseudo color on the display 18. In addition, in the signal processing unit 39, when the length measurement mode is set, a structure emphasis process for emphasizing the structure of blood vessels and the like, and a color difference emphasis process for expanding the color difference between the normal part and the lesion part in the observation object can be performed on the subject image.

[0063] The display control unit 40 displays the subject image generated by the signal processing unit 39 on the display 18. The system control unit 41 controls the imaging element 32 via the imaging control unit 33 provided in the endoscope 12. The imaging control unit 33 also controls the CDS / AGC circuit 34 and the A / D converter 35 based on the control of the imaging element 32.

[0064] like Figure 5 As shown, the light beam emitting section 23 is arranged relative to the optical axis Ax (reference Figure 7 ) obliquely emits the measuring light beam Lm. The light beam emitting unit 23 includes a light source 23a, a diffractive optical element DOE23b (Diffractive Optical Element), a prism 23c, and an emitting unit 23d. The light source 23a emits light of a color that can be detected by the pixels of the imaging element 32 (specifically, visible light), and includes a light emitting element such as a laser light source LD (Laser Diode) or an LED (Light Emitting Diode) and a focusing lens that collects the light emitted from the light emitting element. In addition, the light source 23a is provided on an observer electrical substrate (not shown). The observer electrical substrate is provided at the front end portion 12d of the endoscope, receives power supply from the light source device 14 or the processor device 16, and supplies power to the light source 23a.

[0065] In the present embodiment, the wavelength of the light emitted by the light source 23a is, for example, a red laser (color of the light beam) of 600 nm or more and 650 nm or less, but light of other wavelength bands, for example, green light or blue light of 495 nm or more and 570 nm or less, may be used. The light source 23a is controlled by the system control unit 41, and emits light according to a command from the system control unit 41. DOE 23b converts the light emitted from the light source into a measuring light beam for obtaining measurement information. In addition, with respect to the measuring light beam, it is preferred to adjust the light amount from the perspective of protecting the human body, eyes, and internal organs, and to adjust the light amount to a sufficient degree of whitening (pixel saturation) within the observation range of the endoscope 12.

[0066] The prism 23c is an optical component for changing the traveling direction of the measuring light beam converted by the DOE 23b. The prism 23c changes the traveling direction of the measuring light beam so as to intersect with the field of view of the imaging optical system 21 including the objective lens 21a. The details of the traveling direction of the measuring light beam will be described later. The measuring light beam Lm emitted from the prism 23c is irradiated to the subject through the emission portion 23d formed by the optical component.

[0067] By irradiating the measuring beam light to the subject, Figure 6 As shown in FIG. 1 , a light spot SP is formed in the subject as the light beam irradiation area. The position of the light spot SP is determined by the irradiation area recognition unit 58 (refer to FIG. 1 ). Figure 8 ) is recognized, and a measurement mark indicating the size of the object is set according to the position of the spot SP. The display control unit 40 displays a measurement image on the display 18 in which the set measurement mark is displayed on the object image.

[0068] The position of the light spot SP is related to the observation distance (the distance between the front end portion 12d of the endoscope and the object of observation). When the light spot SP is located at the bottom, the observation distance is small, and the higher the light spot SP is located, the larger the observation distance is. The set measurement mark Ma is displayed on the subject image. The radius of the light spot SP in the measurement mark Ma represents the actual size of the subject (for example, 5mm). In addition, the size (radius) of the measurement mark Ma changes according to the position of the light spot SP, that is, the observation distance. For example, when the light spot SP is located at the bottom, the size of the measurement mark Ma is large, and the higher the light spot SP is located, the smaller the size of the measurement mark Ma is.

[0069] As described later, the measurement mark includes a plurality of types such as a first measurement mark and a second measurement mark, and which measurement mark is displayed on the subject image can be selected by a user command. For example, the user interface 19 can be used as the user command.

[0070] In addition, instead of forming the emission portion 23d by an optical component, a measurement auxiliary slit formed at the front end portion 12d of the endoscope may be used. Furthermore, in the case where the emission portion 23d is formed by an optical component, an anti-reflection coating (AR (Anti-Reflection) coating) (anti-reflection portion) is preferably applied. The anti-reflection coating is provided in this way because: if the measurement light beam is not transmitted through the emission portion 23d but is reflected, thereby reducing the proportion of the measurement light beam irradiated to the subject, it is difficult for the irradiation area recognition unit 58 described later to recognize the position of the light spot SP formed on the subject by the measurement light beam.

[0071] In addition, the light beam emitting unit 23 only needs to be able to emit the measuring light beam toward the field of view of the imaging optical system 21. For example, the light source 23a can be provided in the light source device, and the light emitted from the light source 23a can be guided to the DOE 23b through the optical fiber. In addition, the structure can be set as follows: without using the prism 23c, the direction of the light source 23a and the DOE 23b is set to be inclined relative to the optical axis Ax of the imaging optical system 21, thereby emitting the measuring light beam Lm in the direction across the field of view of the imaging optical system 21.

[0072] Regarding the direction of travel of the measuring beam, Figure 7 As shown, the measuring light beam Lm is emitted in a state where the optical axis of the measuring light beam Lm intersects the optical axis Ax of the imaging optical system 21. It is known that if observation can be performed within the range of observation distance Rx, the position of the spot SP formed on the subject by the measuring light beam Lm within the imaging range (indicated by arrows Qx, Qy, Qz) of each point in the near end Px, the center vicinity Py, and the far end Pz of the range Rx (the point where each arrow Qx, Qy, Qz intersects with the optical axis of the measuring light beam Lm) is different. In addition, the photographic angle of view of the imaging optical system 21 is represented in the area sandwiched between the two solid lines 101, and the central area (the area sandwiched between the two dotted lines 102) with less aberration in this photographic angle of view is measured.

[0073] As described above, the measuring light beam Lm is emitted in a state where the optical axis of the measuring light beam Lm intersects the optical axis Ax, thereby enabling the size of the subject to be measured according to the movement of the light spot position relative to the change in the observation distance. Then, the subject illuminated by the measuring light beam is photographed by the imaging element 32, thereby obtaining a subject image including the light spot SP. In the subject image, the position of the light spot SP varies depending on the relationship between the optical axis Ax of the imaging optical system 21 and the optical axis of the measuring light beam Lm and the observation distance, but the closer the observation distance, the more pixels are displayed for the same actual size (e.g., 5 mm), and the farther the observation distance, the fewer pixels are displayed.

[0074] like Figure 8As shown in FIG. 1 , the signal processing unit 39 of the processor device 16 includes an irradiation area recognition unit 58, a measurement mark setting unit 61, and a mark table 62. In addition, in the signal processing unit 39, when the normal observation mode is set, the subject image of the subject illuminated by the illumination light is input. When the special observation mode is set, the subject image of the subject illuminated by the special light is input. When the length measurement mode is set, the subject image of the subject illuminated by the illumination light and the measurement beam light is input.

[0075] The irradiation area recognition unit 58 recognizes the light beam irradiation area having a specific shape pattern from the subject image. Specifically, the specific shape pattern includes a white central area CR1 and a peripheral area SR1 covering the periphery of the central area and having a characteristic amount based on the measurement light beam. If the light beam irradiation area is the above-mentioned spot SP, then Fig. 9 As shown, the pattern of the specific shape has a circular shape. At this time, the white center area CR1 is a circle, and the peripheral area SR1 is a ring.

[0076] Fig.10 1 shows the distribution of pixel values ​​of each color image in the subject image including a red image RP, a green image GP, and a blue image BP as a plurality of color images. When the pixel values ​​of the red image RP, the green image GP, and the blue image BP in the center region CR1 reach the maximum pixel value (e.g., 255), the center region CR1 becomes white. At this time, when the measuring light beam is incident on the imaging element 32, as shown in FIG. Fig.11 As shown, in the wavelength region WMB of the measuring light beam, not only the red filter RF of the imaging element 32, but also the green filter GF and the blue filter BF transmit the measuring light beam at a transmittance of 100%. On the other hand, in the peripheral region SR1, the pixel value of the red image RP becomes larger than the pixel value of the green image GP or the blue image BP. Therefore, the peripheral region SR1 has a red tint. In addition, in the light source unit 26, the light amount of the measuring light beam is emitted at a specific light amount, thereby setting the pixel values ​​of the red image RP, the green image GP, and the blue image BP in the center region CR1 to the maximum pixel value.

[0077] In the irradiation area recognition unit 58, the light spot SP having the specific shape and feature quantity as described above can be recognized. Fig.12As shown, the irradiation area recognition unit 58 preferably has a learning model 59, which recognizes the light spot SP by outputting the light spot SP as the light beam irradiation area for the input of the subject image. The learning model 59 performs machine learning based on a plurality of teacher data that establish a corresponding association between the subject image and the identified light beam irradiation area. As machine learning, CNN (Convolutional Neural Network) is preferably used.

[0078] By using the learning model 59 to identify the light spot SP, it is possible to identify not only the circular light spot SP composed of the circular central region CR1 and the annular peripheral region SR1 (refer to Fig. 9 ), it is also possible to identify a light spot SP of a pattern that is deformed from a circle of a specific shape. Fig.13 As shown in (A), the light spot SP deformed in the longitudinal direction can also be recognized. Fig.13 As shown in (B), a light spot SP that is deformed due to a missing part of a circle can also be identified. Furthermore, as feature quantities of the peripheral region SR1 that can be identified using the learning model 59, in addition to red, which is the color of the measuring light beam, there are blue and green. Furthermore, as feature quantities of the peripheral region SR1 that can be identified using the learning model 59, there are brightness, lightness, chroma, and hue of the measuring light beam. In addition, the brightness, lightness, chroma, and hue of the measuring light beam are preferably obtained by performing brightness conversion processing or brightness, chroma, and hue conversion processing on the peripheral region of the light spot SP included in the subject image.

[0079] And, if Fig.14 As shown, the irradiation area recognition unit 58 may include a pattern matching processing unit 60, which uses the template image of the light spot SP as the template image of the predetermined light beam irradiation area to perform pattern matching processing on the subject image, thereby recognizing the light spot SP. The template image of the light spot SP is stored in the template image storage unit 60a. The template image storage unit 60a stores not only the template image of the circular light spot SP composed of the circular center area CR1 and the annular peripheral area SR1, but also the template image of the light spot SP of the pattern deformed from the circle of a specific shape. In addition, in the pattern matching processing unit 60, it is also possible to perform pattern matching processing based on the pattern of the feature amount of the peripheral area SR1. As the feature amount that can be pattern matched, in addition to the distribution of red as the color of the measurement light beam, there are also distributions of blue and green. In addition, as the feature amount that can be pattern matched, there are distributions of brightness, lightness, chroma, and hue of the measurement light beam.

[0080] The measurement mark setting unit 61 sets a first measurement mark indicating the actual size (the same size as the actual size) of the subject according to the position of the light spot SP as a measurement mark for measuring the size of the subject. The measurement mark setting unit 61 sets a measurement mark image corresponding to the position of the light spot SP by referring to a mark table 62 in which measurement mark images that are displayed differently according to the irradiation position of the light spot SP are stored in association with the position of the light spot SP. In addition, in the mark table 62, in addition to the position of the light spot SP, measurement mark images that are displayed differently according to the mark display position of the first measurement mark displayed on the display 18 are stored in association with the position of the light spot SP and the mark display position.

[0081] The measurement mark image has a different size or shape, for example, depending on the irradiation position of the light spot SP. The display of the measurement mark image will be described later. Furthermore, the mark table 62 keeps the stored content even when the power of the processor device 16 is turned off. In addition, the mark table 62 stores the measurement mark image and the irradiation position in a corresponding relationship, but the distance to the subject (the distance between the distal end portion 12d of the endoscope 12 and the subject) corresponding to the irradiation position and the measurement mark image may also be stored in a corresponding relationship.

[0082] The display control unit 40 displays a measurement image in which a measurement mark is displayed on the subject image according to the position of the spot SP as the light beam irradiation area on the display 18. Specifically, the display control unit 40 displays a measurement image with a first measurement mark superimposed on the spot SP as the center on the display 18. As the first measurement mark, for example, a circular measurement mark is used. In this case, Fig.15 As shown, when the observation distance is close to the proximal end Px, the center of the spot SP1 formed on the tumor tm1 of the object is aligned, and a mark M1 indicating the actual size of 5 mm (horizontal and vertical directions of the object image) is displayed. In addition, when the measurement mark is displayed on the display 18, the observation distance can also be displayed on the display 18.

[0083] And, if Fig.16 As shown in FIG. 1 , when the observation distance is close to the central vicinity Py, the center of the spot SP2 formed on the tumor tm2 of the object is aligned, and a mark M2 indicating an actual size of 5 mm (horizontal and vertical directions of the object image) is displayed. The mark display position of the mark M2 is located at the center of the object image which is not easily affected by the deformation caused by the imaging optical system 21, so the mark M2 is not affected by the deformation and the like and has a circular shape.

[0084] And, if Fig.17As shown, the center of the spot SP3 formed on the tumor tm3 of the object is aligned, and a mark M3 indicating an actual size of 5 mm (in the horizontal and vertical directions of the object image) is displayed. Figure 15 to Figure 17 As shown, the size of the first measurement mark corresponding to the same actual size of 5 mm decreases as the observation distance increases. In addition, the shape of the first measurement mark also varies depending on the mark display position in response to the influence of the deformation caused by the imaging optical system 21.

[0085] In addition, Figure 15 to Figure 17 In the embodiment, the center of the spot SP is aligned with the center of the mark and displayed, but if there is no problem in measurement accuracy, the first measurement mark can be displayed at a position away from the spot SP. However, in this case, it is also preferable to display the first measurement mark near the spot.

[0086] And, in Figure 15 to Figure 17 In FIG. 1 , a first measurement mark corresponding to the actual size of the object, 5 mm, is displayed. However, the actual size of the object can be set to any value (e.g., 2 mm, 3 mm, 10 mm, etc.) according to the object to be observed and the purpose of observation. Figure 15 to Figure 17 In the example, the first measurement mark is set to be roughly circular, but Fig.18 As shown, it can also be set to a cross shape in which the vertical line and the horizontal line intersect. Furthermore, it can also be set to a cross shape with a scale Mx added to at least one of the vertical line and the horizontal line of the cross shape. Furthermore, as the first measuring mark, it can be set to a distorted cross shape in which at least one of the vertical line and the horizontal line is tilted. Furthermore, the first measuring mark can be set to a circle and a cross shape formed by combining a cross shape and a circle. In addition, the first measuring mark can be set to a measuring point group type formed by combining a plurality of measuring points EP corresponding to the actual size from the light spot. Furthermore, the number of the first measuring marks can be one or more, and the color of the first measuring mark can be changed according to the actual size.

[0087] In addition, as the first measurement mark, Fig.19 As shown, three concentric circular marks M4A, M4B, and M4C (with diameters of 2 mm, 5 mm, and 10 mm, respectively) of different sizes can be displayed on the subject image with the spot SP4 formed on the tumor tm4 as the center. With regard to the three concentric circular marks, since multiple marks are displayed, the labor and time of switching can be saved, and even when the subject is non-linear, measurement can be performed. In addition, when multiple concentric circular marks are displayed with the spot as the center, a combination of multiple conditions can be prepared in advance and selected from the combination, instead of specifying the size and color for each mark.

[0088] exist Fig.19In the example, all three concentric circle marks are displayed in the same color (black), but when multiple concentric circle marks are displayed, multiple colored concentric circle marks may be displayed with the colors changing according to the marks. Fig. 20 As shown, the mark M5A is represented by a dotted line representing red, the mark M5B is represented by a solid line representing blue, and the mark M5C is represented by a dashed line representing white. By changing the color of the mark in this way, the recognition is improved, so that measurement can be easily performed.

[0089] Furthermore, as the first measurement mark, in addition to a plurality of concentric circle marks, Fig.21 As shown in FIG. 1 , a plurality of distorted concentric circle marks formed by distorting each concentric circle may be used. In this case, the distorted concentric circle marks M6A, M6B, and M6C are displayed on the subject image with the spot SP5 formed on the tumor tm5 as the center.

[0090] In addition, as for the measuring beam light, light that forms a light spot when irradiated to the subject is used, but other light may be used. Fig. 22 As shown, a planar measuring light beam can be used which forms a cross line 80 on the subject when irradiated to the subject. By irradiating the planar measuring light beam to the subject, the cross line 80 as a linear beam irradiation area is formed on the subject. At this time, as a measuring mark, a second measuring mark consisting of the cross line 80 and the scale 82 is generated, and the scale 82 becomes an index of the size of the subject (for example, polyp P) on the cross line 80.

[0091] When a planar measuring light beam is used, the irradiation region recognition unit 58 recognizes the position of the cross line 80 (the irradiation position of the measuring light beam) as a light beam irradiation region having a pattern of a specific shape. Fig.23 As shown, the cross line 80 includes a white center area CR2 and a peripheral area SR2 covering the white center area R2 and having a characteristic amount based on the measurement beam light. The center area CR2 is linear, and the peripheral area SR2 is composed of two linear shapes separated by a specific interval. The irradiation area recognition unit 58 can recognize the cross line 80 having the above pattern and characteristic amount. In addition, the cross line 80 that is deformed from the linear shape, such as a part of the line being missing, can also be recognized by the irradiation area recognition unit 58.

[0092] As with the spot SP, the cross line 80 identified by the irradiation region identification unit 58 is located at a lower position, and the observation distance is closer, and the cross line 80 is located at an upper position, and the observation distance is farther. Therefore, when the second measurement mark is displayed on the measurement image, the interval between the scales 82 is larger when the cross line 80 is located at a lower position, and the interval between the scales 82 is smaller when the cross line 80 is located at an upper position.

[0093] In the above description, the measurement mark corresponding to the position of the beam irradiation area such as the spot SP or the cross line 80 is displayed on the measurement image, but instead, the measurement mark corresponding to the pattern of the beam irradiation area may be displayed on the measurement image. For example, in the case where the beam irradiation area is the spot SP, Fig.24 As shown in FIG. 1 , the diameter DM of the white center region CR1 in the spot SP is obtained, and a measurement mark corresponding to the diameter DM is displayed on the measurement image. It is known that when the observation distance is small, the diameter DM is large, and the larger the observation distance, the smaller the diameter. Using the relationship between the diameter DM and the observation distance, a measurement mark image with different sizes and shapes according to the diameter DM is determined.

[0094] Next, use Fig.25 The flowchart shown in FIG. 1 illustrates a series of processes of the present invention. If the mode switching switch 13a is operated to switch to the length measurement mode, the illumination light and the measuring light beam are irradiated on the subject. In the subject, a light spot SP is formed by the measuring light beam. By photographing the subject including the light spot SP, an image of the subject including the light spot SP can be obtained.

[0095] The receiving unit 38 receives the subject image output from the endoscope 12. The irradiation area recognition unit 58 recognizes the position of the light spot SP from the subject image. At this time, the irradiation area recognition unit 58 recognizes the position of the light spot SP including the white central area CR1 and the peripheral area SR1 covering the periphery of the central area CR1 and having the characteristic amount based on the measurement beam light. The display control unit 40 displays a measurement image on the display 18 in which a measurement mark for measuring the size of the subject is displayed on the subject image according to the position of the light spot SP recognized by the irradiation area recognition unit 58. The measurement mark corresponds to the position of the light spot SP, and the size and shape of the measurement mark change according to the position of the light spot SP.

[0096] In the above-mentioned embodiment, the hardware structure of the processing unit (processing unit) that performs various processes, such as the receiving unit 38, the signal processing unit 39, the display control unit 40, the system control unit 41, the still image storage unit 42, the irradiation area recognition unit 58, the learning model 59, the pattern matching processing unit 60, the template image storage unit 60a, the measurement mark setting unit 61, and the mark table 62, is various processors (processors) as shown below. The various processors include a general-purpose processor that executes software (programs) and functions as various processing units, namely a CPU (Central Processing Unit), a processor that can change the circuit structure after manufacturing, such as a FPGA (Field Programmable Gate Array), that is, a programmable logic device (PLD), and a processor that has a circuit structure specially designed to perform various processes, namely, a dedicated circuit, etc.

[0097] One processing unit may be composed of one of these various processors, or may be composed of a combination of two or more processors of the same type or different types (for example, a combination of multiple FPGAs, CPUs and FPGAs). Furthermore, multiple processing units may be composed of one processor. As an example of multiple processing units composed of one processor, first, there is a method in which a processor is composed of a combination of one or more CPUs and software, as represented by computers such as clients and servers, and the processor functions as multiple processing units. Secondly, there is a method in which a processor is used that implements the functions of the entire system including multiple processing units, as represented by a system on chip (SoC). In this way, as a hardware structure, various processing units are composed of one or more of the above-mentioned various processors.

[0098] More specifically, the hardware structure of these various processors is a circuit in the form of a combination of circuit elements such as semiconductor elements, and the hardware structure of the storage unit is a storage device such as a HDD (hard disc drive) or an SSD (solid state drive).

[0099] Another aspect of the present invention is as follows.

[0100] An endoscope system, comprising: an endoscope having a beam light emitting portion for irradiating a subject with a measuring beam light, and an imaging optical system arranged at a front end portion at a position different from the beam light emitting portion and receiving light from the subject, the measuring beam light being emitted obliquely relative to an optical axis of the imaging optical system; and a processor device, comprising: a receiving portion for acquiring a subject image obtained based on the light received by the imaging optical system, the subject image including a beam irradiation area that appears in the subject due to irradiation with the measuring beam light; an irradiation area recognition portion for recognizing the position of the beam irradiation area from the subject image; and a display control portion for displaying a measuring image on the display portion in which a measuring mark for measuring a size of the subject is displayed on the subject image according to the position of the beam irradiation area, wherein in the endoscope system,

[0101] The processor device recognizes the light beam irradiation area having a pattern of a specific shape through the irradiation area recognition unit, the pattern of the specific shape including a white central area and a peripheral area covering the periphery of the central area and having a characteristic amount based on the measurement light beam.

[0102] Explanation of symbols

[0103] 10-endoscope system, 12-endoscope, 12a-insertion portion, 12b-operation portion, 12c-bending portion, 12d-front end portion, 13a-mode switching switch, 13b-freeze switch, 13c-zoom operation portion, 14-light source device, 16-processor device, 18-display, 19-user interface, 21-imaging optical system, 21a-objective lens, 21b-zoom lens, 22-illumination optical system, 22a-illumination lens, 23-beam light Emitter, 23a-light source, 23b-DOE, 23c-prism, 23d-emitter, 24-opening, 25-air and water supply nozzle, 26-light source, 27-light source control unit, 28-light guide, 29-connector, 32-imaging element, 33-camera control unit, 34-CDS / AGC circuit, 35-A / D converter, 36-communication I / F, 37-communication I / F, 38-receiving unit, 39-signal processing unit, 40-display control unit , 41-system control unit, 42-still image storage unit, 58-irradiation area recognition unit, 59-learning model, 60-pattern matching processing unit, 60a-template image storage unit, 61-measurement mark setting unit, 62-mark table, 80-cross line, 82-scale, BF-blue filter, GF-green filter, RF-red filter, BP-blue image, GP-green image, RP-red image, CR1, CR2-center area, M 1. M2, M3-cross mark, Ma-measurement mark, tm1, tm2, tm3, tm4, tm5-tumor, SP-light spot, SP1, SP2, SP3, SP4, SP5-light spot, SR1, SR2-peripheral area, M4A, M4B, M4C, M5A, M5B, M5C-concentric circle mark, M6A, M6B, M6C-distortion concentric circle mark, P-polyp, WMB-wavelength area (measurement mark).

Claims

1. An endoscope system comprising: An endoscope having a light beam emitting portion for irradiating a measuring light beam to a subject, and an imaging optical system provided at a front end portion at a position different from the light beam emitting portion and receiving light from the subject, wherein the measuring light beam is emitted obliquely with respect to an optical axis of the imaging optical system; and a processor device, The processor device comprises a processor, and the processor performs the following processing: acquiring a subject image obtained based on the light received by the imaging optical system, the subject image including a beam irradiation area appearing in the subject by irradiation of the measuring beam light; identifying the light beam irradiation area from the subject image; and displaying, on a display, a measuring image in which a measuring mark for measuring the size of the object is displayed on the object image according to the light beam irradiation area; The light beam irradiation area in the subject image has a pattern of a specific shape, the pattern of the specific shape includes a white central area and a peripheral area covering the periphery of the central area and having a characteristic amount based on the measurement light beam, The processor identifies the light beam irradiation area having the pattern of the specific shape, The feature amount is at least any one of the color of the measuring beam light, a color other than the color of the measuring beam light, and the brightness, lightness, saturation, and hue of the measuring beam light.

2. The endoscope system according to claim 1, wherein: The processor recognizes the light beam irradiation area by outputting a learning model of the light beam irradiation area with respect to an input of the subject image.

3. The endoscope system according to claim 1, wherein: The processor performs pattern matching processing on the object image using a template image of the light beam irradiation area determined in advance, thereby identifying the light beam irradiation area.

4. The endoscope system according to claim 1, wherein: The processor is capable of recognizing the light beam irradiation area of ​​the pattern in which the specific shape is deformed.

5. The endoscope system according to claim 2, wherein: The processor is capable of recognizing the light beam irradiation area of ​​the pattern in which the specific shape is deformed.

6. The endoscope system according to claim 3, wherein: The processor is capable of recognizing the light beam irradiation area of ​​the pattern in which the specific shape is deformed.

7. The endoscope system according to any one of claims 1 to 6, wherein: The specific shape is a circle.

8. The endoscope system according to any one of claims 1 to 6, wherein: The specific shape is a line.

9. The endoscope system according to any one of claims 1 to 6, wherein: The processor receives the subject image including a plurality of color images, The light intensity of the measuring light beam is set to a specific light intensity, and the pixel value of each color image is set to a maximum pixel value, thereby making the central area of ​​the light beam irradiation area white.

10. The endoscope system according to any one of claims 1 to 6, wherein: The processor displays the measurement mark corresponding to the position of the light beam irradiation area on the measurement image.

11. The endoscope system according to claim 9, wherein: The processor displays the measurement mark corresponding to the position of the light beam irradiation area on the measurement image.

12. The endoscope system according to any one of claims 1 to 6, wherein: The processor displays the measurement mark corresponding to the pattern of the light beam irradiation area on the measurement image.

13. The endoscope system according to claim 9, wherein: The processor displays the measurement mark corresponding to the pattern of the light beam irradiation area on the measurement image.

14. A method for operating an endoscope system, the endoscope system comprising: an endoscope having a light beam emitting portion for irradiating a measuring light beam to a subject, and an imaging optical system disposed at a front end portion at a position different from the light beam emitting portion and receiving light from the subject, the measuring light beam being emitted obliquely with respect to an optical axis of the imaging optical system; and a processor device having a processor, wherein in the method for operating the endoscope system, The processor has the following steps: acquiring a subject image obtained based on the light received by the imaging optical system, the subject image including a beam irradiation area appearing in the subject by irradiation of the measuring beam light; identifying the light beam irradiation area from the subject image; and Displaying a measuring image on a display in which a measuring mark for measuring the size of the object is displayed on the object image according to the light beam irradiation area, The light beam irradiation area in the subject image has a pattern of a specific shape, the pattern of the specific shape includes a white central area and a peripheral area covering the periphery of the central area and having a characteristic amount based on the measurement light beam, The processor identifies the light beam irradiation area having the pattern of the specific shape, The feature amount is at least any one of the color of the measuring beam light, a color other than the color of the measuring beam light, and the brightness, lightness, saturation, and hue of the measuring beam light.

Images