Endoscope and endoscope system

Abstract

To suppress a dead angle region and an overlapping region when simultaneously capturing a direct view image of a biological tissue obtained through a front window facing the front side of a light reception surface of an imaging element and a side view image of the biological tissue obtained through a side window facing the lateral side in comparison to the front window as photographed images.SOLUTION: An endoscope which images a biological tissue in a body cavity comprises: an imaging element which is configured to capture an image of the biological tissue; an objective lens which simultaneously captures a direct view image of the biological tissue obtained through a front window facing the front side of a light reception surface of the imaging element and a side view image of the biological tissue obtained through a side window facing the lateral side in comparison to the front window as photographed images; and an optical element which is provided in at least one of the front window and the side window, and changes a visual field range of at least one of the direct view image and the side view image by adjusting the deflection characteristics of incident light with a control signal.SELECTED DRAWING: Figure 3

Description

[Technical Field]

[0001] The present invention relates to an endoscope and an endoscope system for imaging biological tissue inside a body cavity. [Background technology]

[0002] An endoscope is a device equipped with an imaging element that is inserted into a body cavity, such as a human body, to capture images of biological tissue on the inner surface of the body cavity. For example, an endoscope is inserted into the large intestine and displays the captured images on a monitor to determine whether or not the biological tissue has an abnormality, such as a lesion. When diagnosing biological tissue in the large intestine, the endoscope must be pulled in one direction while tilting the folds in one direction to determine whether or not a lesion exists at the base of the folds so that the folds protruding from the inner surface do not interfere with imaging. However, even when the folds are tilted in one direction, a lesion may exist in a portion hidden behind the folds. Furthermore, if the field of view of the captured image is narrow, the area between adjacent folds may not be captured. Therefore, in order to be able to capture images of the spaces between the folds from various directions without the folds collapsing, an objective lens with a wide viewing angle may be used in the objective optical system of the imaging device.

[0003] Also, an endoscopic system is known that includes an endoscope having an insertion section that is inserted into a tube to be observed, a direct-view observation section that has a field of view in the direction of the tip of the insertion section, a side-view observation section that has a field of view in the direction of the side of the insertion section, and a protrusion that protrudes from the insertion section and forms a blind spot in the field of view of the side-view observation section, and an image acquisition section that acquires direct-view observation images using the direct-view observation section and side-view observation images using the side-view observation section (Patent Document 1). [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Japanese Patent Application Laid-Open No. 2018-57799 Summary of the Invention [Problem to be solved by the invention]

[0005] In the above-described endoscope system, a direct-view observation image and a side-view observation image are simultaneously acquired, so that a wide range of the observation target can be displayed on the display unit. However, blind spots may occur between the direct-view observation image and the side-view observation image, where the blind spots are not visible in either image. That is, a lesion that was visible in the side-view observation image may disappear as the endoscope moves, and then suddenly appear in the direct-view observation image. Also, overlapping areas may occur between the direct-view observation image and the side-view observation image, where the lesion is visible in both images. That is, a lesion may exist in the overlapping area, and the lesion may be present in both the side-view observation image and the direct-view observation image. Simultaneous display of images with discontinuous field of view can be unnatural for the endoscope operator, and can lead to overlooking lesions in blind spots and miscounting the number of lesions in overlapping areas, leading to misdiagnosis.

[0006] Therefore, an object of the present invention is to provide an endoscope and an endoscope system that can achieve a smooth field of view by suppressing blind spots and overlapping areas when simultaneously capturing a direct-view image of biological tissue obtained through a front window facing forward from the light-receiving surface of the imaging element and a side-view image of biological tissue obtained through a side window facing to the side of the front window. [Means for solving the problem]

[0007] One aspect of the present invention is an endoscope for imaging biological tissue in a body cavity, comprising: an imaging element configured to capture an image of biological tissue; an objective lens that simultaneously forms, on the light-receiving surface as captured images, a direct-view image of the biological tissue obtained through a front window facing forward of the light-receiving surface of the imaging element and a side-view image of the biological tissue obtained through a side window facing laterally relative to the front window; The present invention is characterized by comprising an optical element provided in at least one of the front window and the side window, which changes the field of view of at least one of the direct-view image and the side-view image by adjusting the deflection characteristics of incident light using a control signal.

[0008] a control unit for generating the control signal; It is preferable that the control unit generates the control signal so as to achieve at least one of the following: a reduction in the overlapping area between the direct-view image and the side-view image in the captured image including the direct-view image and the side-view image; and a reduction in the blind spot area in the field of view of the captured image.

[0009] It is preferable that the control unit extracts the overlapping area between the direct-view image and the side-view image from pixel values ​​of the captured image, and generates the control signal by determining the deflection characteristics of the incident light according to the size of the overlapping area.

[0010] It is preferable that the control unit is provided with a prediction model that has previously been machine-learned to determine the relationship between the captured image, including the direct-view image and the side-view image, and the size of the overlapping area, and that the control unit determines the deflection characteristics of the incident light by using the prediction model to determine the presence or absence and size of the overlapping area from the pixel values ​​of the captured image of the biological tissue captured by the imaging element.

[0011] It is preferable that the control unit calculates the size of the blind spot between the direct-view image and the side-view image from the positional deviation amount at the end of the direct-view image or the side-view image of the linearly extending portion of the biological tissue captured by the imaging element and the image of the linearly extending portion in the side-view image, and generates the control signal by determining the deflection characteristics of the incident light according to the size.

[0012] It is preferable that the control unit is provided with a prediction model that has previously been machine-learned to determine the relationship between the position and amount of positional deviation of the linearly extending portion in the captured image and the size of the blind spot area, and that the control unit determines the amount of change by using the prediction model to determine the presence or absence and size of the blind spot area from the position and amount of positional deviation of the linearly extending portion obtained from the pixel values ​​of the captured image of the biological tissue captured by the imaging element.

[0013] The side direction the window is provided around the circumference of a cylindrical member surrounding the imaging element, It is preferable that the optical elements are provided at a plurality of locations along the circumferential direction so that the deflection characteristics have a distribution in the circumferential direction.

[0014] The maximum half angle of view of the objective lens is preferably 90 degrees or more.

[0015] Another aspect of the present invention is an endoscope system including an endoscope that captures images of biological tissue in a body cavity, and an endoscope processor that processes images of the biological tissue captured by the endoscope, The endoscope comprises: an imaging element configured to capture an image of biological tissue; an objective lens that simultaneously forms, on the light-receiving surface as captured images, a direct-view image of the biological tissue obtained through a front window provided in front of the light-receiving surface of the imaging element and a side-view image of the biological tissue obtained through a side window facing laterally relative to the front window; an optical element provided in at least one of the front window and the side window, which changes the field of view of at least one of the direct-view image and the side-view image by adjusting the deflection characteristics of incident light using a control signal; the endoscope processor includes an image processing unit that processes an image of a living tissue and generates a control signal for controlling a deflection characteristic of the incident light; The image processing unit is characterized in that it generates the control signal so as to satisfy at least one of the following: in the captured image including the direct-view image and the side-view image, the overlapping area between the direct-view image and the side-view image is reduced; and blind spots in the field of view of the captured image are reduced. [Effects of the Invention]

[0016] According to the above-described endoscope and endoscope system, when a direct-view image and a side-view image of biological tissue are simultaneously captured as captured images, blind spots and overlapping areas can be reduced to achieve a smooth field of view. [Brief explanation of the drawings]

[0017] [Figure 1] 1 is an external perspective view of an endoscope according to an embodiment of the present invention; [Figure 2] 1 is a block diagram showing a configuration of an endoscope system according to an embodiment; [Figure 3] FIG. 2 is a diagram illustrating an example of the configuration of a tip portion of an endoscope according to an embodiment. [Figure 4] 3A and 3B are diagrams illustrating an example of a deflection direction of incident light that is incident on an optical element used in an endoscope according to an embodiment. [Figure 5] 1 is a diagram illustrating a view range through a front window and a view range through a side window of an endoscope according to an embodiment of the present invention; [Figure 6] 1 is a diagram illustrating an example of a state in which a distal end portion of an endoscope according to an embodiment is inserted into an organ; [Figure 7] 7 is a diagram showing an example of an image displayed on a monitor in the inserted state of the distal end portion shown in FIG. 6. FIG. [Figure 8] 10A and 10B are diagrams illustrating another example of a state in which the distal end portion of the endoscope according to the embodiment is inserted into an organ. [Figure 9] 9 is a diagram showing an example of an image displayed on a monitor in the inserted state of the distal end portion shown in FIG. 8. FIG. [Figure 10] 10A and 10B are diagrams illustrating a change in the field of view obtained by adjusting the deflection direction of light incident on an optical element used in an endoscope according to an embodiment. [Figure 11] 10A and 10B are diagrams illustrating an example of an image displayed on a monitor obtained by adjusting the deflection characteristics of an optical element used in the endoscope of one embodiment. [Figure 12] 1 is a diagram illustrating an example of a configuration of a connector including a control unit of an endoscope according to an embodiment. FIG. DETAILED DESCRIPTION OF THE INVENTION

[0018] An endoscope and an endoscope system according to an embodiment of the present invention will be described below with reference to the drawings. Fig. 1 is a perspective view showing the appearance of an endoscope according to an embodiment. Fig. 2 is a block diagram showing the configuration of an endoscope system according to an embodiment. Fig. 3 is a diagram showing an example of the configuration of the tip portion of an endoscope according to an embodiment.

[0019] An endoscope (hereinafter referred to as an electronic scope) 100 shown in Fig. 1 is connected to a processor 200 for an electronic endoscope shown in Fig. 2 to form an endoscope system 1. The endoscope system 1 is a system specialized for medical use, and as shown in Fig. 2, mainly comprises the electronic scope 100, the processor 200 for an electronic endoscope, and a monitor 300. The electronic scope 100 and the monitor 300 are each connected to the processor 200.

[0020] 1, the electronic scope 100 mainly comprises a connector 110, an operation unit 120, and a tip end portion 132, and further comprises a flexible cable 130 that extends from the operation unit 120 toward the tip end portion 132 at the front and has flexibility, a bending tube 134 that is connected to the front of the flexible cable 130 via a connecting portion and can be bent freely, and a universal tube 128 that extends rearward from the operation unit 120. The connector 110 is fixed to the rear end of the universal tube 128 and is configured to be connected to a processor 200. A plurality of bending operation wires are inserted through the operation unit 120, the flexible cable 130, and the bending tube 134, with the tip of each bending operation wire connected to the rear end of the bending tube 134 and the rear end of each bending operation wire connected to the bending operation knob 122 of the operation unit 120. The bending tube 134 can be bent in any direction and by any angle in response to the operation of the bending operation knob 122.

[0021] Furthermore, the operation unit 120 includes a plurality of operation buttons 124. When the endoscope operator (surgeon or assistant) presses the operation button 124, the operation button 124 can instruct various functions, such as discharging water or gas from an air / water supply port (not shown) provided on the distal end surface of the distal end portion 132, suctioning liquid or gas in living tissue through a suction port, and discharging cleaning liquid from a cleaning liquid discharge nozzle for cleaning the objective lens.

[0022] The tip portion 132 at the tip of the bending tube 134 is made of a hard resin material (for example, ABS, modified PPO, PSU, etc.) that is substantially not elastically deformable. The tip portion 132 includes an LED light source 102 and an image sensor 108 located immediately behind the objective lens 106. The tip portion 132 is attached to the tip of the long flexible cable 130 and includes the LED light source 102, the objective lens 106, and the image sensor 108. The objective lens 106 is attached to the front of the image sensor 108 and forms an image of biological tissue on the light receiving surface of the image sensor 108 with a field of view of 180 degrees or more, preferably greater than 180 degrees. The tip portion 132 is provided with a front window facing forward of the light receiving surface of the image sensor 108 and a side window facing laterally relative to the front window, as described below. The image sensor 108 is configured to capture the image formed on the light receiving surface by the objective lens 106 through the front window and the side window. The front window and the side window are provided with optical elements capable of variably adjusting the deflection direction (deflection characteristics) of incident light, as described below. Here, a forward-facing front window refers to a window in which the center of the field of view seen through the front window faces forward. "Forward" refers to the direction in which the tip surface of the tip portion 132 faces. The orientation of a side window refers to the direction in which the center of the field of view seen through the side window faces. The side window may be oriented so that its field of view includes many areas to the side of the field of view of the front window, and may be tilted, for example, at an angle of 30 to 90 degrees relative to the direction of the front window. In the following explanation, the orientation of the side window will be described using an embodiment in which the side window faces laterally, perpendicular to the forward direction.

[0023] The flexible cable 130, the bending tube 134, and the tip portion 132 form an insertion portion 135 that is inserted into a body cavity. An image signal cable extending from the imaging element 108 provided in the tip portion 132 extends from the tip portion 132 through the bending tube 134, the flexible cable 130, and further through the inside of the operation unit 120 and the universal tube 128 to the inside of the connector 110. The connector 110 is connected to a processor 200. The processor 200 processes the image signal sent from the imaging element and controls the monitor 300 to display the image of the subject captured by the imaging element 108.

[0024] 2, the processor 200 of the endoscope system 1 includes a system controller 202 and a timing controller 206. The system controller 202 executes various programs stored in a memory 204 and comprehensively controls the entire electronic endoscope system 1. The system controller 202 also changes various settings of the electronic endoscope system 1 in response to instructions from an endoscope operator (surgeon or assistant) input to an operation panel 208. The timing controller 206 outputs clock pulses to each circuit in the electronic endoscope system 1 to adjust the timing of the operation of each part.

[0025] In addition to the imaging element 108, an LED light source 102 is provided at the tip 132 of the endoscope 100. The LED light source 102 emits illumination light that illuminates living tissue so that the imaging element 108 can capture an image. The LED light source 102 is driven to emit light by a drive signal generated by a light source control circuit 116 provided in the connector 110. Instead of the LED light source 102, a laser element may be used, or a high-intensity lamp such as a xenon lamp, a metal halide lamp, a mercury lamp, or a halogen lamp may be used. 2, the LED light source 102 is provided at the tip 132, but it may also be provided as a light source device in the connector 110 or the processor 200. In this case, illumination light is guided from the light source device to the tip 132 through a light guide made up of multiple bundled fiber cables.

[0026] The light emitted from the LED light source 102 is irradiated as illumination light onto the biological tissue, which is the subject, via the light distribution lens 104. The light reflected from the biological tissue passes through the front window 140, the side window 150 (see FIG. 3) and the objective lens 106 to form an optical image on the light receiving surface of the image sensor 108.

[0027] The imaging element 108 is, for example, a single-chip color CCD (Charge-Coupled Device) image sensor with various filters, such as an IR (Infrared) cut filter 108a and a Bayer-array color filter 108b, arranged on its light-receiving surface, and generates primary color signals of R (Red), G (Green), and B (Blue) corresponding to the optical image formed on the light-receiving surface. A single-chip color CCD image sensor may also be used as a single-chip color CMOS (Complementary Metal Oxide Semiconductor) image sensor. In this way, the electronic endoscope 100 uses the imaging element 108 to capture images of biological tissue inside organs and generate video.

[0028] A driver signal processing circuit 112 is provided inside the connector 110 of the electronic endoscope 100. The driver signal processing circuit 112 performs predetermined signal processing, such as color interpolation and matrix calculation, on the primary color signals input from the image sensor 108 to generate image signals (luminance signal Y and color difference signals Cb and Cr), and outputs the generated image signals to the image processing unit 220 of the electronic endoscope processor 200. The driver signal processing circuit 112 also accesses the memory 114 to read out unique information about the electronic endoscope 100. The unique information about the electronic endoscope 100 stored in the memory 114 includes, for example, the number of pixels and sensitivity of the image sensor 108, the operable frame rate, and the model number. The driver signal processing circuit 112 outputs the unique information read out from the memory 114 to the system controller 202.

[0029] The system controller 202 performs various calculations and generates control signals based on the information stored in the memory 204 and the unique information of the electronic endoscope 100. Using the generated control signals, the system controller 202 controls the operation and timing of each circuit within the electronic endoscope processor 200 so that processing appropriate for the electronic endoscope 100 connected to the electronic endoscope processor 200 is performed.

[0030] The timing controller 206 supplies clock pulses to the driver signal processing circuit 112, the image processing unit 220, and the light source section 230 in accordance with the timing control by the system controller 202. The driver signal processing circuit 112 drives and controls the image pickup element 108 in accordance with the clock pulses supplied from the timing controller 206, at a timing synchronized with the frame rate of the video processed on the electronic endoscope processor 200 side.

[0031] Under the control of the system controller 202, the image processing unit 220 generates a video signal for displaying an image or the like on the monitor based on the image signal input from the driver signal processing circuit 112, and outputs the video signal to the monitor 300. Furthermore, the image processing unit 220 is configured to generate a control signal for adjusting the deflection direction of incident light that enters the optical elements provided in the front window 140 and the side window 150 shown in Fig. 3 provided in the tip portion 132, in accordance with the content of the captured image, and adjust the deflection direction of the incident light. The image processing unit 220 may also perform a digitization process on the image of biological tissue obtained by the electronic scope 100, quantifying the feature amount of each pixel in the image that allows a lesion to be distinguished from a healthy area, evaluate the degree of progression of the lesion in the image, and generate a color map image in which the numerical values ​​of each pixel obtained by the digitization process are replaced with colors. In this case, the image processing unit 220 generates a video signal for displaying information on the results of the digitization process and the color map image on the monitor, and outputs the video signal to the monitor 300. This allows the endoscope operator to perform an accurate examination through the image displayed on the display screen of the monitor 300. The image processing unit 220 outputs the image, information on the results of the digitization process, and the color map image to the printer 400 as necessary.

[0032] The processor 200 is connected to a server 600 via a NIC (Network Interface Card) 210 and a network 500. The processor 200 can download information relating to an endoscopic examination (for example, electronic medical record information on the patient and information on the surgeon) from the server 600. The downloaded information is displayed, for example, on the display screen of the monitor 300 or on the operation panel 208. The processor 200 can also upload the results of an examination performed by the endoscope 100 to the server 600, thereby causing the server 600 to store the results of the examination.

[0033] In such an electronic scope 100, when a direct-view image of biological tissue captured through the front window 140 (see FIG. 3) and a side-view image of biological tissue captured through the side window 150 (see FIG. 3) are simultaneously captured as captured images and displayed on the monitor 300, it is undesirable for the same lesion to be captured simultaneously in both the side-view image and the direct-view image due to an overlapping area captured in both images. Also, displaying a captured image on the monitor 300 in which the direct-view image and the side-view image are discontinuous creates an uncomfortable feeling for the endoscope operator. Furthermore, if there is a blind spot between the side-view image and the direct-view image, it is undesirable for a lesion in the blind spot to be overlooked. For this reason, the front window 140 (see FIG. 3) and the side window 150 (see FIG. 3) of the tip 132 of the electronic scope 100 are provided with optical elements 142, 152 that change the field of view of at least one of the direct-view image and the side-view image by adjusting the deflection characteristics of the incident light with a control signal, as shown in FIG. This point will be explained below.

[0034] The tip portion 132 includes an objective lens 106, an image sensor 108, a front window 140, and a side window 150. The objective lens 106 and the image sensor 108 are disposed in a cylindrical member 133 formed of a hard resin material at the tip portion 132. The front window 140 and the side window 150 are provided with optical elements 142, 152. The front window 140 faces forward of the light receiving surface 108c of the imaging element 108. The side windows 150 face in a direction perpendicular to the forward direction. The objective lens 106 is composed of a group of lenses 106a to 106e, including meniscus lenses, convex lenses, and concave lenses, and has a half angle of view of 90 degrees or more, preferably greater than 90 degrees, and more preferably 110 degrees or more. Therefore, the objective lens 106 simultaneously forms a direct-view image of the biological tissue obtained through the front window 140 and a side-view image of the biological tissue obtained through the side window 150 as captured images on the light-receiving surface 108c. The subject-side surface of the lens 106a also serves as the front window 140.

[0035] The optical elements 142 and 152 are, for example, thin film elements that change the field of view of at least one of the direct-view image and the side-view image by adjusting the deflection direction of the incident light that is incident on the optical elements 142 and 152 using a control signal. The optical elements 142 and 152 may be, for example, liquid crystal lenses. Well-known liquid crystal lenses, such as those disclosed in Japanese Patent Nos. 5156999 and 6128719, may be used. A liquid crystal lens is, for example, a liquid crystal cell formed by sealing nematic liquid crystal in the gap between two tilted, non-parallel glass substrates. A voltage is applied to the liquid crystal cell, and the alignment of the liquid crystal is controlled by adjusting the voltage, thereby continuously changing the refractive index and continuously changing the polarization direction of incident light. The optical elements 142 and 152 may also be variable-focus lenses using electro-optic crystals whose refractive index changes with an applied voltage, such as potassium tantalate niobate. Alternatively, the optical elements 142 and 152 may be liquid lenses whose radius of curvature is changed to change the polarization direction of incident light.

[0036] 4 is a diagram illustrating an example of the deflection direction of incident light that has entered the optical elements 142 and 152. As shown in FIG. in By applying the control signal V, the emitted light L before control out , L out * That is, the optical elements 142 and 152 change the incident light L in Change the deflection direction. Therefore, by providing optical elements 142, 152 in front of the objective lens 106, the field of view of the image formed on the light receiving surface 108c through the objective lens 106 and the optical elements 142, 152 can be narrowed or widened by applying a control signal V.

[0037] 5 is a diagram schematically illustrating a field of view R1 through the front window 140 and a field of view R2 through the side window 150 of an embodiment of the electronic scope 100. The example shown in FIG. 5 illustrates the field of view when the deflection characteristics of the optical elements 142 and 152 are not adjusted. 5, the field of view of the image formed on the light receiving surface 108c includes an overlap region R3 that overlaps both the field of view R1 and the field of view R2. Also, there is a blind spot R4 that is not included in either the field of view R1 or the field of view R2. Therefore, when imaging biological tissue, the lesion may be in the blind spot R4 or in the overlap region R3.

[0038] FIG. 6 is a diagram showing an example of the state in which the distal end 132 of the electronic scope 100 is inserted into an organ. In FIG. 6, the distal end 132 is positioned to the right side of the organ, so part of the lesion X is located in the overlap region R3, and the remaining part is located in the field of view R1. FIG. 7 is a diagram showing an example of an image displayed on the monitor 300 when the distal end 132 is inserted as shown in FIG. 6. The monitor 300 displays a single image in which the field of view R2 is arranged around the field of view R1. If a lesion is present in the overlap region R3 between the field of view R1 and the field of view R2, the lesion X is displayed as a discontinuous overlap, as shown in FIG. 7. The circular dotted line in FIG. 7 indicates the boundary between the edge of the field of view visible through the front window 140 and the edge of the field of view visible through the side window 150. In FIG. 7, the edge of the field of view visible through the front window 140 and the edge of the field of view visible through the side window 150 overlap on the screen of the monitor 300. Therefore, in FIG. 7, the overlap region R3 exists both inside and outside the dotted line. In this way, the display of the lesion X of interest discontinuously and overlappingly is not desirable in terms of determining the size and progression of the lesion X while viewing the image. Furthermore, the display of the lesion X that is discontinuous and has overlapping portions is inconvenient for the endoscope operator and gives him or her a sense of discomfort.

[0039] FIG. 8 shows another example of the insertion state of the distal end portion 132 of the electronic scope 100 within an organ. In FIG. 8, the distal end portion 132 is positioned further to the right within the organ than in the example shown in FIG. 6, so part of the lesion X is located in a blind spot R4. FIG. 9 shows an example of an image displayed on the monitor 300 when the distal end portion 132 is inserted as shown in FIG. 8. The monitor 300 displays a single image in which a field of view R2 is positioned around a field of view R1. The two circular dotted lines in FIG. 9 indicate the edges of the field of view visible through the front window 140 and the side window 150. Therefore, the area between these two dotted lines corresponds to the blind spot R4. In this case, the portion of the lesion X within the blind spot R4 disappears as shown in FIG. 9 and is displayed discontinuously. In such a case, the endoscope operator may mistakenly determine that there are two lesions X, so it is undesirable to create a blind spot R4 in which the lesion X is partially obscured.

[0040] FIG. 10 is a diagram illustrating the change in the field of view obtained through the optical elements 142 and 152 and the objective lens by adjusting the deflection direction of the incident light incident on the optical elements 142 and 152. In FIG. By adjusting the optical element 142 with the control signal V, the range of the direct-view image formed on the light receiving surface 108c is changed from the visual field range R1 to the visual field range R1 * (narrower), or the field of view range R1 ** Similarly, by adjusting the optical element 152 with the control signal V, the range of the side-view image formed on the light-receiving surface 108c is changed from the viewing range R2 to the viewing range R2 * (narrower), or the field of view range R2 ** become (expand). Therefore, if an overlapping region R3 exists when the distal end 132 is inserted into an organ and imaging the biological tissue, the deflection characteristics of at least one of the optical elements 142 and 152 are adjusted to narrow at least one of the field of view R1 and R2. Furthermore, if a blind spot R4 exists when the distal end 132 is inserted into an organ and imaging the biological tissue, the deflection characteristics of at least one of the optical elements 142 and 152 are adjusted to widen at least one of the field of view R1 and R2. This eliminates overlapping or missing portions in the image of the lesion X. Figure 11 shows an example of an image displayed on the monitor 300 obtained by adjusting the deflection characteristics of the optical elements 142 and 152 used in the electronic scope 100 of one embodiment. The discontinuity in the display of the lesion X is eliminated, and no disappearance occurs.

[0041] In this way, optical elements 142, 152 that adjust the deflection characteristics (deflection direction) of the incident light are provided in the front window 140 and the side window 150 of the tip portion 132, and by adjusting the deflection characteristics of the incident light with the control signal V, the field of view of at least one of the direct view image and the side view image is changed. Therefore, when the direct view image and the side view image of the biological tissue are captured simultaneously as captured images, a smooth field of view can be achieved by suppressing blind spots and overlapping areas.

[0042] Optical elements 142, 152 are provided in both the front window 140 and the side window 150 of the tip portion 132, but it is also possible to provide an optical element in at least one of the front window 140 and the side window 150 and change the field of view of at least one of the direct-view image and the side-view image by adjusting the deflection characteristics of the incident light with a control signal.

[0043] According to one embodiment, the adjustment of the deflection direction of the incident light entering the optical elements 142, 152 is performed by a control signal V generated by the image processing unit 220 of the processor 200. According to one embodiment, the control signal V is generated by a control unit provided in the electronic scope 100. FIG. 12 is a diagram showing an example of the configuration of a connector 110 including a control unit of the electronic scope 100 of one embodiment. In the example shown in FIG. 12, a control unit 115 is provided in the connector 110. The presence or absence of an overlapping region R3 or a blind spot region R4 is determined using the image signal generated by the driver signal processing circuit 112, and if the overlapping region R3 or the blind spot region R4 exists, its size is calculated. Also, when the control signal V is generated by the image processing unit 220, the image signal obtained by the image processing unit 220 is used to determine whether or not there is an overlapping area R3 or a blind spot area R4, and if there is an overlapping area R3 or a blind spot area R4, its size is calculated. In this case, it is preferable that the control unit 115 or the image processing unit 220 generates the control signal V so as to achieve at least one of the following: a reduction in the overlapping area between the direct-view image and the side-view image in the captured image including the direct-view image and the side-view image; and a reduction in the blind spot area in the field of view of the captured image.

[0044] According to one embodiment, when the endoscope operator finds the presence of a lesion X in the captured image displayed on the monitor 300, the endoscope operator may adjust the deflection characteristics of the optical elements 142, 152 by operating the operation button 124 while viewing the image of the lesion X. In this case, when the endoscope operator presses the operation button 124, an instruction signal for adjusting the deflection characteristics of the optical elements 142, 152 is sent to the control unit 115 shown in FIG. 12. The control unit 115 generates a control signal V in accordance with the instruction signal to adjust the deflection characteristics of the optical elements 142, 152. The endoscope operator waits for the captured image after the adjustment of the deflection characteristics to be displayed on the monitor 300, and then repeatedly operates the operation button 12 to adjust the deflection characteristics.

[0045] According to one embodiment, the control unit 115 or the image processing unit 220 preferably extracts an overlap region R3 between the direct-view image and the side-view image from pixel values ​​of the captured image, and generates the control signal V by determining the deflection characteristics of the incident light according to the size of the overlap region R3. The presence or absence of the overlap region R3 and the size of the overlap region R3 can be determined, for example, even if no lesion X is present, by using an image of the surface of the biological tissue of the subject where the brightness has changed due to the unevenness of the surface, or an image where the color change appears on the surface of the biological tissue or a feature portion such as a blood vessel on the subject can be distinguished from other portions. The control signal V currently applied to the optical elements 142 and 152 is known, and the field of view R1, field of view R1, * Or field of view range R1 ** , and field of view range R2, field of view range R2 * Or field of view range R2 ** Since the location of the overlapping region R3 in the captured image is known, the range in which the overlapping region R3 may occur can be roughly identified in advance. Therefore, by examining the pixel values ​​of the captured image within the range that can be identified in advance, it is possible to efficiently determine the presence or absence of the overlapping region R3 and the size of the overlapping region R3.

[0046] According to one embodiment, the control unit 115 or the image processing unit 220 preferably calculates the size of a blind spot between the direct-view image and the side-view image from the positional deviation between the image of a linearly extending portion of the biological tissue in the direct-view image and the image of the same linearly extending portion in the side-view image captured by the image sensor 108, and determines the deflection direction of the incident light entering the optical elements 142, 152 based on the size of the blind spot, thereby generating the control signal V. Since the currently applied control signal V is known, the locations of the edges of the direct-view image and the side-view image can be roughly determined. Examples of linearly extending portions include the leading edge of a fold in the large intestine, blood vessels appearing on the surface of biological tissue, and a linear boundary portion separating a lesion X from a healthy area. While a linearly extending portion preferably extends in a substantially straight line, it may also be a curve whose direction changes smoothly. In the case of a smoothly changing curve, the amount of positional deviation at the end of the direct-view image or side-view image relative to the other linear image can be determined by extending the image of the smoothly changing curve from one linearly extending portion for the portion interrupted by the blind spot area R4.

[0047] According to one embodiment, the side viewing window 150 provided in the distal end portion 132 is provided around the circumference of the cylindrical member 133 that surrounds the image sensor 108. In this case, the optical elements 152 are preferably provided at multiple locations along the circumference so that the polarization characteristics, i.e., the polarization direction of incident light, are distributed around the circumference of the cylindrical member 133. By offsetting the position of the distal end portion 132 from the center of the organ canal, the presence or absence of the overlap region R3 and the blind spot region R4 varies around the circumference. Furthermore, the sizes of the overlap region R3 and the blind spot region R4 also vary around the circumference depending on the offset amount of the distal end portion 132. By providing the optical elements 152 at multiple locations along the circumference of the side window 150, the overlap region R3 and the blind spot region R4 can be reduced or even eliminated in the image displayed on the monitor 300.

[0048] According to one embodiment, the maximum half angle of view of the objective lens 106 is preferably equal to or greater than 90 degrees, and even more preferably greater than 90 degrees. The maximum half angle of view is more preferably equal to or greater than 100 degrees, even more preferably equal to or greater than 110 degrees, and particularly preferably equal to or greater than 112 degrees. By capturing images over a wide field of view, the electronic scope 100 can efficiently capture, in a side-view image, the presence of a lesion X that is located at the base of a fold in the large intestine or the like and that would be obscured by the collapse of the fold, for example, before the fold collapses.

[0049] The control unit 115 or the image processing unit 220 determines the presence and size of the overlap region R3 or the blind spot region R4 using the captured image. However, the presence and size of the overlap region R3 or the blind spot region R4 may be determined using a prediction model previously trained by machine learning. The control unit 115 or the image processing unit 220 includes a prediction model (AI model) previously trained by machine learning to determine the relationship between the captured image, including the direct-view image and the side-view image, and the size of the overlap region R3 or the blind spot region R4. The control unit 115 or the image processing unit 220 uses this prediction model to determine the presence and size of the overlap region R3 or the blind spot region R4 from the pixel values ​​of the captured image of the biological tissue captured by the image sensor 108, thereby determining the deflection characteristics (deflection direction) of the incident light incident on the optical elements 142 and 152. When the predictive model learns the relationship between the captured image and the size of the blind spot area R4 by machine learning, it learns in advance the relationship between the position of misalignment (the position where there is a misalignment) and the amount of misalignment between the direct view image and the side view image of a linearly extending portion in the captured image, and the size of the blind spot area R4, for example. When a neural network is used as a predictive model, it may include a deep neural network (DNN) model formed to be predictable through deep learning, a random forest model using a tree structure, a model using LASSO regression, or a nonlinear function using polynomials, kriging, RBF network (Radial Basis Function Network: RBFN), etc. When using a prediction model, information about the control signal V may also be used together with captured images including a direct-view image and a side-view image to perform machine learning in advance to determine the relationship between the overlap region R3 or the blind spot region R4. In this case, when the prediction model predicts and determines the size of the overlap region R3 or the blind spot region R4, information about the control signal V is also input to the prediction model as input data in addition to the captured images.

[0050] The endoscope and endoscopic system of the present invention have been described in detail above, but the present invention is not limited to the above-described embodiments, and various improvements and modifications may be made without departing from the spirit and scope of the present invention. [Explanation of symbols]

[0051] 1. Endoscopy system 100 Endoscope 102 LED light source 104 Orientation Lens 106 Objective Lens 106a~106e lenses 108 image sensor 108a Cut filter 108b Color Filter 110 Connector 112 Driver signal processing circuit 114 memory 115 Control Unit 116 Light source control circuit 120 Operation section 122 Curvature control knob 128 Universal Tube 130 flexible cable 132 Tip 133 Cylindrical member 134 Curved section 140 Front window 142,152 Optical elements 150 Side window 200 processors 202 System Controller 204 memory 206 Timing Controller 208 Operation Panel 210NIC 220 Image Processing Unit 300 monitors 400 printers 500 Network 600 servers

Claims

1. An endoscope for imaging biological tissue in a body cavity, an imaging element configured to capture an image of biological tissue; an objective lens that simultaneously forms, on the light-receiving surface as captured images, a direct-view image of the biological tissue obtained through a front window facing forward of the light-receiving surface of the imaging element and a side-view image of the biological tissue obtained through a side window facing laterally relative to the front window; an optical element provided in the front window and the side window, which adjusts the deflection characteristics of incident light using a control signal to change the field of view of the direct-view image and the side-view image so as to widen or narrow the field of view; An endoscope comprising:

2. a control unit for generating the control signal; 2. The endoscope according to claim 1, wherein the control unit generates the control signal so as to achieve at least one of reducing an overlapping area between the direct-view image and the side-view image in the captured image including the direct-view image and the side-view image, and reducing a blind spot area in the field of view of the captured image.

3. 3. The endoscope according to claim 2, wherein the control unit extracts the overlapping area between the direct-view image and the side-view image from pixel values ​​of the captured image, and generates the control signal by determining a deflection characteristic of the incident light in accordance with a size of the overlapping area.

4. 4. The endoscope according to claim 3, wherein the control unit includes a prediction model that has previously been machine-learned to determine a relationship between the captured image including the direct-view image and the side-view image and the size of the overlapping region, and the control unit determines the deflection characteristics of the incident light by using the prediction model to determine the presence or absence and size of the overlapping region from pixel values ​​of the captured image of the biological tissue captured by the imaging element.

5. The endoscope according to any one of claims 2 to 4, wherein the control unit calculates the size of the blind spot between the direct-view image and the side-view image from a positional deviation amount at an end of the direct-view image or the side-view image of the linearly extending portion of the biological tissue captured by the imaging element and the image of the linearly extending portion in the side-view image, and determines the deflection characteristics of the incident light in accordance with the size, thereby generating the control signal.

6. 6. The endoscope according to claim 5, wherein the control unit includes a prediction model that has been machine-learned in advance to determine a relationship between the position and amount of positional deviation of the linearly extending portion in the captured image and the size of the blind spot area, and the control unit determines the deflection characteristics of the incident light by using the prediction model to determine the presence or absence and size of the blind spot area from the position and amount of positional deviation of the linearly extending portion obtained from pixel values ​​of the captured image of the biological tissue imaged by the imaging element.

7. the side window is provided around the circumference of a cylindrical member that surrounds the imaging element, The endoscope according to any one of claims 1 to 6, wherein the optical elements are provided at a plurality of locations along the circumferential direction so that the deflection characteristics have a distribution in the circumferential direction.

8. 8. The endoscope according to claim 1, wherein the maximum half angle of view of the objective lens is 90 degrees or more.

9. An endoscope system including an endoscope that captures images of biological tissue in a body cavity, and an endoscope processor that processes images of the biological tissue captured by the endoscope, The endoscope comprises: an imaging element configured to capture an image of biological tissue; an objective lens that simultaneously forms, on the light-receiving surface as captured images, a direct-view image of the biological tissue obtained through a front window provided in front of the light-receiving surface of the imaging element and a side-view image of the biological tissue obtained through a side window facing laterally relative to the front window; an optical element provided in the front window and the side window, which adjusts the deflection characteristics of incident light using a control signal to change the field of view of the direct-view image and the side-view image so as to widen or narrow the field of view; the endoscope processor includes an image processing unit that processes an image of a living tissue and generates a control signal for controlling a deflection characteristic of the incident light; an endoscope system in which the image processing unit generates the control signal so as to satisfy the following conditions: in the captured image including the direct-view image and the side-view image, an overlapping area between the direct-view image and the side-view image is reduced; and a blind spot area in the field of view of the captured image is reduced.

Images