Endoscope and endoscope system

Abstract

An endoscope for photographing living tissue in a body cavity, comprising: an imaging element (108) configured to capture a living tissue image; an objective lens (106) configured to simultaneously image a direct-view image of the living tissue obtained through a front window (140) in front of a light-receiving surface of the imaging element and a side-view image of the living tissue obtained through a side window (150) more toward a side than the front window, on the light-receiving surface; and an optical element (142, 152) provided on at least any one of the front window and the side window, and configured to change a field of view range of at least any one of the direct-view image and the side-view image by adjusting a deflection characteristic of incident light using a control signal.

Description

Technical Field

[0001] This invention relates to an endoscope and endoscope system for imaging living tissue within body cavities. Background Technology

[0002] An endoscope is a device equipped with a camera that is inserted into a body cavity, such as the human body, to photograph living tissue on the inner surface of the cavity. For example, an endoscope is inserted into the large intestine, and the image is displayed on a monitor to determine if there are unhealthy areas, such as lesions, in the biopsy tissue. When diagnosing a biopsy in the large intestine, the endoscope must be pulled in one direction while simultaneously tilting the folds in the same direction, without obstructing the image, to determine if there are lesions at the base of the folds. However, even when the folds are tilted in one direction, lesions may still exist in areas hidden behind the folds. If the field of view for the image is narrow, the area between adjacent folds may not be captured.

[0003] Therefore, in order to capture images of the folds from all directions without them collapsing, a wide-angle objective lens can be used in the objective lens system of the imaging element.

[0004] Furthermore, a known endoscopic system includes: an endoscope having an insertion portion inserted into a tube for observation, a direct viewing portion having a field of view in the direction of the front end of the insertion portion, a lateral viewing portion having a field of view in the transverse direction of the insertion portion, a protrusion protruding from the insertion portion and forming a blind spot in the field of view of the lateral viewing portion; and an image acquisition unit that acquires a direct viewing image using the direct viewing portion and a lateral viewing image using the lateral viewing portion (Japanese Patent Application Laid-Open No. JP2018-57799). Summary of the Invention

[0005] In the aforementioned endoscope system, both direct and lateral observation images can be acquired simultaneously, thus displaying a wide range of objects on the display unit.

[0006] However, blind spots sometimes appear between direct and lateral viewing images, not visible in either image. That is, lesions shown in the lateral viewing image may disappear as the endoscope moves, only to suddenly reappear in the direct viewing image. Furthermore, overlapping areas sometimes occur between the direct and lateral viewing images. In other words, lesions may exist in the overlapping area, appearing simultaneously in both the lateral and direct viewing images.

[0007] Displaying images with such discontinuous fields of view can cause disorientation for the endoscopic operator. Furthermore, ignoring lesions in blind spots or incorrectly counting the number of lesions in overlapping areas can easily lead to misdiagnosis.

[0008] Therefore, the object of the present invention is to provide an endoscope and an endoscope system that, when simultaneously capturing a direct view image of a living tissue obtained through a front window facing the light-receiving front of the camera element and a side view image of the living tissue obtained through a side window facing more to the side than the front window, can suppress blind spots and overlapping areas and achieve a smooth field of view.

[0009] Solution for solving the problem

[0010] One embodiment of the present invention is an endoscope for imaging living tissue within a body cavity, characterized in that it comprises:

[0011] The imaging element is configured to capture images of living tissue.

[0012] The objective lens simultaneously images the live tissue as a camera image on the light-receiving surface through a front window that faces the front of the light-receiving surface of the imaging element and a side view of the live tissue that faces the side more towards the side than the front window.

[0013] And optical elements disposed on at least one of the front window and the side window, and by adjusting the deflection characteristics of the incident light using control signals, changing the field of view of at least one of the direct view image and the side view image.

[0014] Preferably, the system includes a control unit for generating the control signal.

[0015] The control unit generates the control signal to achieve at least one of the following: reducing the overlap area between the direct view image and the side view image in the camera image, which includes the direct view image and the side view image, and reducing the blind spot area in the field of view of the camera image.

[0016] Preferably, the control unit extracts the overlapping region between the front view image and the side view image based on the pixel values ​​of the camera image, and determines the deflection characteristics of the incident light based on the size of the overlapping region, thereby generating the control signal.

[0017] Preferably, the control unit has a prediction model for performing machine learning on the relationship between the size of the camera image, including the orthographic image and the side view image, and the overlapping region. By using the prediction model, the presence and size of the overlapping region are determined based on the pixel values ​​of the camera image of the living tissue captured by the camera element, thereby determining the deflection characteristics of the incident light.

[0018] Preferably, the control unit calculates the size of the dead zone region located between the direct view image and the side view image based on the positional offset of the linearly extending portion of the live tissue in the direct view image and the linearly extending portion in the side view image captured by the camera element at the ends of the direct view image or the side view image, and determines the deflection characteristics of the incident light based on the size, thereby generating the control signal.

[0019] Preferably, the control unit has a prediction model for performing machine learning on the relationship between the positional offset and positional offset of the linearly extending portion in the camera image and the size of the blind spot region. Using the prediction model, the positional offset and positional offset of the linearly extending portion are determined based on the pixel values ​​of the camera image of the living tissue captured by the camera element, thereby determining the presence and size of the blind spot region and thus determining the amount of change.

[0020] Preferably, the side window is configured to surround the circumference of the cylindrical member that surrounds the imaging element.

[0021] The optical element is disposed at each of a plurality of locations along the circumference such that the deflection characteristics are distributed along the circumference.

[0022] Preferably, the objective lens has a maximum half-angle of 90 degrees or more.

[0023] Another embodiment of the present invention is an endoscope system, specifically an endoscope system comprising an endoscope for imaging living tissue within a body cavity, and an endoscope processor for image processing of the images of the living tissue captured by the endoscope, wherein...

[0024] The endoscope has the following features:

[0025] The imaging element is configured to capture images of living tissue.

[0026] The objective lens simultaneously images the live tissue as a camera image on the light-receiving surface, using a front window located in front of the light-receiving surface of the camera element as a direct view image and a side view image of the live tissue obtained through a side window that is more lateral than the front window.

[0027] And optical elements, which are disposed on at least one of the front window and the side window, and change the field of view of at least one of the direct view image and the side view image by adjusting the deflection characteristics of the incident light using a control signal;

[0028] The endoscope processor includes an image processing unit that processes images of living tissue to generate control signals for controlling the deflection characteristics of the incident light.

[0029] The image processing unit generates the control signal to satisfy at least one of the following: reducing the overlap area between the direct view image and the side view image in the camera image, which includes the direct view image and the side view image, and reducing the blind spot area in the field of view of the camera image.

[0030] Invention Effects

[0031] According to the aforementioned endoscope and endoscope system, when simultaneously capturing direct and side views of living tissue as camera images, it is possible to suppress blind spots and overlapping areas and achieve a smooth field of view. Attached Figure Description

[0032] Figure 1 This is a perspective view of the exterior of an endoscope according to one embodiment.

[0033] Figure 2 This is a block diagram illustrating the structure of an endoscope system in one embodiment.

[0034] Figure 3 This is a diagram illustrating an example of the configuration of the front end of an endoscope according to one embodiment.

[0035] Figure 4 This is a diagram illustrating an example of the deflection direction of incident light incident on an optical element used in an endoscope according to one embodiment.

[0036] Figure 5 This diagram schematically illustrates the field of view through the front window and the field of view through the side window of an endoscope according to one embodiment.

[0037] Figure 6 This is a diagram illustrating an example of the insertion state of the anterior endoscope of one embodiment into an organ.

[0038] Figure 7 It is shown Figure 6 The image shown is an example of an image displayed on the monitor when the front end is in the inserted state.

[0039] Figure 8This is another example of the insertion state of the anterior endoscope of one embodiment into an organ.

[0040] Figure 9 It is shown Figure 8 The image shown is an example of an image displayed on the monitor when the front end is in the inserted state.

[0041] Figure 10 This is a diagram illustrating the change in field of view obtained by adjusting the deflection direction of incident light incident on the optical element used in an endoscope according to one embodiment.

[0042] Figure 11 This is a diagram illustrating an example of an image displayed on a monitor obtained by adjusting the deflection characteristics of an optical element used in an endoscope according to one embodiment.

[0043] Figure 12 This is an example diagram illustrating the configuration of a connector for a control unit of an endoscope having one embodiment. Detailed Implementation

[0044] The following description of embodiments of the endoscope and endoscope system of the present invention will be based on the accompanying drawings. Figure 1 This is a perspective view of the exterior of an endoscope according to one embodiment. Figure 2 This is a block diagram illustrating the structure of an endoscope system in one embodiment. Figure 3 This is a diagram illustrating an example of the configuration of the front end of an endoscope according to one embodiment.

[0045] Figure 1 The endoscope (hereinafter referred to as the electronic endoscope) 100 shown is connected to Figure 2 The electronic endoscope processor 200 shown forms the endoscope system 1. The endoscope system 1 is a system specifically designed for medical use and mainly consists of an electronic endoscope 100, an electronic endoscope processor 200, and a display 300, such as... Figure 2 As shown. The electronic endoscope 100 and the display 300 are respectively connected to the processor 200.

[0046] like Figure 1 As shown, the electronic endoscope 100 mainly includes a connector 110, an operating section 120, and a front end section 132. It also includes a flexible cable 130 extending forward from the operating section 120 to the front end section 132 and having flexibility; a bent tube 134 connected to the front of the flexible cable 130 via a connecting section and freely bending; and a universal tube 128 extending rearward from the operating section 120. The connector 110 is fixed to the rear end of the universal tube 128 and is configured to connect to the processor 200.

[0047] Multiple bending operation lines are inserted into the operating unit 120, the flexible cable 130, and the bend 134. The front end of each bending operation line is connected to the rear end of the bend 134, and the rear end of each bending operation line is connected to the bending operation knob 122 of the operating unit 120. The bend 134 can be bent in any direction and at any angle according to the operation of the bending operation knob 122.

[0048] Furthermore, the operating unit 120 is equipped with a plurality of operating buttons 124. The operating buttons 124 allow the endoscopic operator (surgeon or assistant) to indicate the following functions by pressing the operating buttons 124: discharging water or gas from the air or water supply port (not shown) provided on the front end face of the front end 132, aspirating liquid or gas from the living tissue through the suction port, and discharging cleaning fluid from the cleaning fluid discharge nozzle to clean the objective lens.

[0049] The front end 132 of the bend 134 is made of a rigid resin material (such as ABS, modified PPO, PSU, etc.) and is basically not elastically deformable.

[0050] An LED light source 102 and an imaging element 108 are disposed inside the front end portion 132, located immediately after the objective lens 106. That is, the front end portion 132, located at the front end of the long flexible cable 130, includes the LED light source 102, the objective lens 106, and the imaging element 108. The objective lens 106 is positioned in front of the imaging element 108, and performs image imaging of living tissue on the light-receiving surface of the imaging element 108 with a viewing angle greater than or equal to 180 degrees, preferably with a field of view greater than 180 degrees. As described later, a front window facing the light-receiving surface of the imaging element 108 and a side window facing further to the side are provided in the front end portion 132. The imaging element 108 is configured to capture the image formed on the light-receiving surface by the objective lens 106 through the front window and the side window. Optical elements are provided on the front window and the side window for variablely adjusting the deflection direction (deflection characteristics) of the incident light (described later). Here, the term "front window facing forward" refers to the window whose center of view, as seen from the front window, points forward. "Forward" refers to the orientation of the front face of the front end 132. The orientation of the side window is the orientation of the center of view, as seen from the side window. The side window only needs to include more lateral areas in its field of view compared to the front window, and its orientation relative to the front window only needs to be within, for example, a range of 30 to 90 degrees. In the following explanation, the orientation of the side window will be described using a lateral direction orthogonal to the front.

[0051] A flexible cable 130, a bend 134, and a front end 132 form an insertion portion 135 that is inserted into the body cavity. An image signal cable extending from the imaging element 108 located at the front end 132 passes through the interior of the bend 134, the flexible cable 130, the operating portion 120, and the general-purpose tube 128 to the interior of the connector 110. The connector 110 is connected to the processor 200. The processor 200 processes the image signals sent from the imaging element and controls the display of the image of the subject captured by the imaging element 108 on the display 300.

[0052] like Figure 2 The processor 200 of the illustrated endoscope system 1 includes a system controller 202 and a timing controller 206. The system controller 202 executes various programs stored in the memory 204 and integrates control of the entire electronic endoscope system 1. Furthermore, the system controller 202 changes various settings of the electronic endoscope system 1 according to instructions input to the operation panel 208 by the endoscope operator (surgeon or assistant). The timing controller 206 outputs clock pulses to each circuit within the electronic endoscope system 1 to adjust the operation time of each component.

[0053] In addition to the imaging element 108, an LED light source 102 is provided in the front end 132 of the electronic endoscope 100. The LED light source 102 emits illumination light to illuminate the living tissue for the imaging element 108 to take pictures.

[0054] The LED light source 102 is driven by a drive signal generated by the light source control circuit 116 located in the connector 110, thereby emitting light. A laser element can also be used instead of the LED light source 102, or a high-brightness lamp, such as a xenon lamp, metal halide lamp, mercury lamp, or halogen lamp, can also be used.

[0055] exist Figure 2 In the example shown, the LED light source 102 is located at the front end 132, but it can also be located as a light source device at the connector 110 or the processor 200. In this case, the illumination light from the light source device to the front end 132 is guided to the front end 132 by a beam of optical fiber with multiple optical cables.

[0056] The light emitted from the LED light source 102 serves as illumination light, and is projected onto the living tissue, which is the subject, via the light distribution lens 104. Reflected light from the living tissue passes through the front window 140 and the side window 150 (see...). Figure 3 The objective lens 106 forms an optical image on the light-receiving surface of the imaging element 108.

[0057] The imaging element 108 is, for example, a monolithic color CCD (Charge-Coupled Device) image sensor that arranges various filters such as an IR (Infrared) cutoff filter 108a and a color filter 108b using a Bayer array on the light-receiving surface, and generates R (Red), G (Green), and B (Blue) primary color signals based on the optical image image formed on the light-receiving surface. Instead of a monolithic color CCD image sensor, a monolithic color CMOS (Complementary Metal Oxide Semiconductor) image sensor can also be used. In this way, the electronic endoscope 100 uses the imaging element 108 to image living tissue inside an organ and generate moving images.

[0058] A driver signal processing circuit 112 is disposed inside the connector 110 of the electronic endoscope 100. The driver signal processing circuit 112 performs predetermined signal processing such as color interpolation and matrix operations on the primary color signal input from the imaging element 108, generates an image signal (brightness signal Y, chromatic aberration signals Cb, Cr), and outputs the generated image signal to the image processing unit 220 of the electronic endoscope processor 200. Furthermore, the driver signal processing circuit 112 accesses the memory 114 and reads specific information of the electronic endoscope 100. The specific information of the electronic endoscope 100 recorded in the memory 114 includes, for example, the number of pixels, sensitivity, operable frame rate, and model of the imaging element 108. The driver signal processing circuit 112 outputs the specific information read from the memory 114 to the system controller 202.

[0059] The system controller 202 performs various operations and generates control signals based on information stored in the memory 204 and specific information of the electronic endoscope 100. The system controller 202 uses the generated control signals to control the operation and timing of various circuits within the electronic endoscope processor 200 in order to perform processing adapted to the electronic endoscope 100 connected to the electronic endoscope processor 200.

[0060] The timing controller 206 supplies clock pulses to the driver signal processing circuit 112, the image processing unit 220, and the light source unit 230 according to the timing control executed by the system controller 202. The driver signal processing circuit 112 drives and controls the imaging element 108 according to the clock pulses supplied from the timing controller 206, in a timing sequence synchronized with the image frame rate processed on the electronic endoscope processor 200 side.

[0061] Under the control of the system controller 202, the image processing unit 220 generates a video signal for displaying images on the display, etc., based on the image signal input from the driver signal processing circuit 112, and outputs it to the display 300. Further, the image processing unit 220 is configured to generate a control signal for adjusting the deflection direction of incident light incident on the optical element based on the content of the captured image, and adjust the deflection direction of the incident light; wherein the optical element is provided on the front end portion 132. Figure 3 The image processing unit 220 can also quantize the images of living tissue obtained by the electronic endoscope 100, quantizing the feature values ​​of each pixel in the image that can distinguish between lesions and healthy areas, assessing the development of lesions in the image, and further replacing the values ​​of each pixel obtained by quantization with color to generate a color map image. In this case, the image processing unit 220 generates a video signal for displaying the information generated by quantization and the color map image on the monitor, and outputs it to the monitor 300. Thus, the endoscopic operator can perform high-precision examinations through the images displayed on the screen of the monitor 300. The image processing unit 220 outputs the image, the information of the quantization results, and the color map image to the printer 400 as needed.

[0062] Processor 200 is connected to server 600 via NIC (Network Interface Card) 210 and network 500. Processor 200 can download information about endoscopic examinations (e.g., patient electronic medical record information and surgeon information) from server 600. The downloaded information is displayed, for example, on the display screen of monitor 300 or on operation panel 208. In addition, processor 200 can also upload the examination results of electronic endoscopy 100 to server 600, thereby storing the examination results on server 600.

[0063] In such an electronic endoscope 100, the anterior window 140 (refer to...) will be used to... Figure 3 Orthogonal images of living tissue taken and viewed through a side window at 150° (reference). Figure 3 When a side view image of a biopsy is captured simultaneously as a camera image and displayed on the monitor 300, it is undesirable for the same lesion to appear in both the side view and the frontal view images due to overlapping areas in the two images. Furthermore, displaying discontinuous camera images of the frontal and side views on the monitor 300 can cause disorientation for the endoscopic operator. Moreover, when there are blind spots between the side view and the frontal view images, it is undesirable to overlook lesions located in the blind spots.

[0064] Therefore, as Figure 3 As shown, at the anterior window 140 of the anterior endoscope 132 (refer to...) Figure 3) and side window 150 (reference) Figure 3 The image is provided with optical elements 142 and 152, which adjust the deflection characteristics of the incident light by means of a control signal, thereby changing the field of view of at least one of the direct view image and the side view image. This will be explained below.

[0065] The front end portion 132 includes an objective lens 106, an imaging element 108, a front window 140, and a side window 150. The objective lens 106 and the imaging element 108 are located within a cylindrical member 133 made of rigid resin material at the front end portion 132. Optical elements 142 and 152 are disposed in the front window 140 and the side window 150.

[0066] The front window 140 faces the front of the light-receiving surface 108c of the camera element 108. The side window 150 faces a direction perpendicular to the front.

[0067] Objective lens 106 includes a set of lenses 106a-106e, including a meniscus lens, a convex lens, and a concave lens, with a half-angle of 90 degrees or more, preferably exceeding 90 degrees, and more preferably greater than or equal to 110 degrees. Therefore, as a photographic image, objective lens 106 simultaneously forms on the light-receiving surface 108c a direct view image of the living tissue obtained through the front window 140 and a side view image of the living tissue obtained through the side window 150. The subject-side surface of lens 106a also serves as the front window 140.

[0068] Optical elements 142 and 152 are, for example, thin-film elements, which change the field of view of at least one of the orthographic and side-view images by adjusting the deflection direction of incident light incident on optical elements 142 and 152 using control signals.

[0069] For example, liquid crystal lenses are used as optical elements 142 and 152. The liquid crystal lens can be a lens known, for example, from Japanese Patent No. 5156999 and Japanese Patent No. 6128719. A liquid crystal lens is, for example, an element in which a liquid crystal cell is formed to seal a nematic liquid crystal in a gap where glass substrates are arranged at non-parallel angles; a voltage is applied to the liquid crystal cell, and the alignment of the liquid crystal is controlled by adjusting the voltage, thereby continuously changing the refractive index and the deflection direction of the incident light. Optical elements 142 and 152 can also use electro-optic crystals whose refractive index changes with the applied voltage, such as variable focus lenses using potassium tantalate or niobate. Furthermore, optical elements 142 and 152 can also use liquid lenses that change the deflection direction of the incident light by changing the radius of curvature.

[0070] Alternatively, diffractive optical elements can be used as optical elements 142 and 152. For example, the deflection direction of incident light can be changed by applying an active diffractive optical element with a movable grating structure to optical elements 142 and 152. The movable grating structure uses an electrostatic force corresponding to the applied voltage to change the displacement of the fringes of the reflection grating.

[0071] Figure 4 This is a diagram illustrating an example of the deflection direction of incident light incident on optical elements 142 and 152. For example... Figure 4 As shown, the incident light L in By assigning a control signal V, the emitted light L before control is controlled. out Change to L out * That is, optical elements 142 and 152 change the incident light L through control signal V. in The direction of deflection.

[0072] Therefore, by placing optical elements 142 and 152 in front of the objective lens 106 and by providing a control signal V, the field of view of the image formed on the light-receiving surface 108c by the objective lens 106 and optical elements 142 and 152 can be narrowed or widened.

[0073] Figure 5 This diagram schematically illustrates the field of view R1 through the anterior window 140 and the field of view R2 through the side window 150 of an electronic endoscope 100 according to one embodiment. Figure 5 The example shown illustrates the field of view without adjusting the deflection characteristics of optical elements 142 and 152.

[0074] like Figure 5 As shown, the field of view of the image imaged on the light-receiving surface 108c includes an overlapping region R3 that overlaps with both the field of view R1 and the field of view R2. Furthermore, there is a blind spot region R4 that is not included in either the field of view R1 or the field of view R2. Therefore, when photographing a biopsy, the lesion may sometimes be located in the blind spot region R4 and sometimes in the overlapping region R3.

[0075] Figure 6 This is a diagram illustrating an example of the intra-organ insertion state of the anterior endoscope 132 of the electronic endoscope 100. Figure 6 In the middle, because the anterior end 132 is biased to the right side of the organ, part of the lesion site X is located in the overlapping area R3, and the rest is located in the field of vision R1. Figure 7 It is shown Figure 6The diagram shows an example of an image displayed on the display 300 when the anterior end 132 is in the inserted state. In the display 300, the field of view R2 is positioned around the field of view R1 and displayed as a single image. When a lesion is present in the overlapping area R3 between the field of view R1 and the field of view R2, then... Figure 7 As shown, the lesion sites X are displayed discontinuously overlapping. Figure 7 The circular dashed line in the diagram indicates the boundary between the end of the field of view visible from the front window 140 and the end of the field of view visible from the side window 150. Figure 7 In this case, the ends of the field of view visible from the front window 140 and the ends of the field of view visible from the side window 150 overlap on the screen of the display 300. Therefore, in Figure 7 In the diagram, the overlapping region R3 exists both inside and outside the dashed line.

[0076] Therefore, when observing images and judging the size and progression of lesion X, it is undesirable for the lesion X of interest to be displayed discontinuously and overlappingly. Furthermore, displaying discontinuous and overlapping lesion X is inconvenient for the endoscopic operator and creates a sense of disharmony.

[0077] Figure 8 This is another example of an organ insertion configuration showing the tip 132 of the electronic endoscope 100. Figure 8 In the middle, due to the front end 132 compared to Figure 6 The example shown is more biased towards the right side of the organ, so part of the lesion site X is located in the blind spot region R4. Figure 9 It is shown Figure 8 The diagram shows an example of an image displayed on the display 300 when the front end 132 is in the inserted state. In the display 300, the field of view R2 is arranged around the field of view R1 and displayed as a single image. Figure 9 The two circular dashed lines indicate the ends of the field of vision visible from the front window 140 and the ends of the field of vision visible from the side window 150. Therefore, the area between these two dashed lines corresponds to the blind spot area R4. In this case, as... Figure 9 As shown, lesion X partially disappears within the blind spot region R4 and is displayed discontinuously. In such cases, endoscopists may sometimes misjudge that there are two lesion sites X, and therefore do not want to create a blind spot region R4 where lesion X partially disappears.

[0078] Figure 10 This is a diagram illustrating the change in the field of view obtained by adjusting the deflection direction of the incident light incident on optical elements 142 and 152, and the objective lens.

[0079] By adjusting the optical element 142 using the control signal V, the range of the orthographic image formed on the light-receiving surface 108c is changed from the field of view R1 to the field of view R2. * (Narrowing), or becoming the field of view R1 ** (Wideened). Similarly, by adjusting the optical element 152 using the control signal V, the range of the side view image image formed on the light-receiving surface 108c is changed from the field of view R2 to the field of view R2. * (Narrowing), or becoming the field of view R2 ** (Wide).

[0080] Therefore, if an overlapping region R3 exists when the tip 132 is inserted into an organ to photograph the living tissue, the deflection characteristics of at least one of optical elements 142 and 152 are adjusted to narrow at least one of the field of view R1 and field of view R2. Furthermore, if a blind spot R4 exists when the tip 132 is inserted into an organ to photograph the living tissue, the deflection characteristics of at least one of optical elements 142 and 152 are adjusted to expand at least one of the field of view R1 and field of view R2. Thus, overlapping or missing portions can be eliminated in the image of the lesion site X. Figure 11 This diagram illustrates an example of an image displayed on a display 300 obtained by adjusting the deflection characteristics of optical elements 142 and 152 used in an electronic endoscope 100 according to one embodiment. The discontinuity in the display of the lesion site X is resolved and does not disappear.

[0081] In this way, optical elements 142 and 152 for adjusting the deflection characteristics (deflection direction) of incident light are provided on the front window 140 and side window 150 of the front end 132. By adjusting the deflection characteristics of the incident light using the control signal V, the field of view of at least one of the orthographic and side images can be changed. Therefore, when the orthographic and side images of living tissue are captured simultaneously as video images, blind spots and overlapping areas can be suppressed and a smooth field of view can be achieved.

[0082] In addition, optical elements 142 and 152 are provided in both the front window 140 and the side window 150 of the front end 132. However, optical elements may also be provided in at least one of the front window 140 and the side window 150. By adjusting the deflection characteristics of the incident light using a control signal, the field of view of at least one of the direct view image and the side view image can be changed.

[0083] According to one embodiment, the deflection direction of incident light incident on such optical elements 142, 152 is adjusted using a control signal V generated by the image processing unit 220 of the processor 200. According to one embodiment, the control signal V is generated by a control unit provided in the electronic endoscope 100. Figure 12 This is an example diagram illustrating the configuration of the connector 110 of the control unit of an electronic endoscope 100 having one embodiment. Figure 12 In the example shown, a control unit 115 is provided on connector 110. The image signal generated by driver signal processing circuit 112 is used to determine whether there is an overlapping area R3 or a dead zone R4, and if there is an overlapping area R3 or a dead zone R4, its size is calculated.

[0084] Furthermore, when the image processing unit 220 generates the control signal V, the image signal obtained by the image processing unit 220 is used to determine whether there is an overlapping region R3 or a dead zone region R4, and if there is an overlapping region R3 or a dead zone region R4, its size is calculated.

[0085] Preferably, in this case, the control unit 115 or the image processing unit 220 generates a control signal V to achieve at least one of reducing the overlap area between the direct view image and the side view image in the camera image, which includes the direct view image and the side view image, and reducing the blind spot area in the field of view of the camera image.

[0086] According to one embodiment, when a lesion X is detected in the image displayed on the monitor 300, the endoscopic operator can adjust the deflection characteristics of the optical elements 142 and 152 by operating the operation button 124 while observing the image of the lesion X. In this case, the endoscopic operator sends an indication signal for adjusting the deflection characteristics of the optical elements 142 and 152 to the operation button 124. Figure 12 In the control unit 115 shown, the control unit 115 generates a control signal V based on the instruction signal to adjust the deflection characteristics of the optical elements 142 and 152. The endoscope operator displays the image after adjusting the deflection characteristics on the display 300, and then repeatedly operates the operation button 12 to adjust the deflection characteristics.

[0087] Preferably, according to one embodiment, the control unit 115 or the image processing unit 220 extracts the overlapping region R3 between the front view image and the side view image based on the pixel values ​​of the captured image, and determines the deflection characteristics of the incident light based on the size of the overlapping region R3, thereby generating a control signal V. Regarding the presence or absence of the overlapping region R3, for example, even in the absence of a lesion X, the presence or absence of the overlapping region R3 and its size can be determined using a surface image where the brightness changes due to the unevenness of the living tissue surface of the subject, an image showing color changes on the living tissue surface, or an image that can be distinguished from other parts of the subject, such as blood vessels. The control signal V currently applied to the optical elements 142 and 152 is known, as is the field of view R1. * Or field of view R1 ** and field of view R2, field of view R2 * Or field of view R2 ** Since the location within the captured image is known, the approximate range within which the overlapping region R3 might occur can be determined in advance. Therefore, by examining the pixel values ​​of the captured image within this predetermined range, the presence and size of the overlapping region R3 can be effectively determined.

[0088] Preferably, according to one embodiment, the control unit 115 or the image processing unit 220 calculates the size of the dead zone region located between the direct view image and the side view image based on the positional offset of the linearly extending portion in the direct view image and the similarly linearly extending portion in the side view image of the living tissue captured by the imaging element 108 at the ends of the direct view image or the side view image, and determines the deflection direction of the incident light incident on the optical elements 142, 152 based on the size, thereby generating a control signal V. Since the currently applied control signal V is known, the positions of the ends of the direct view image and the side view image can be roughly understood. Examples of linearly extending portions include, for example, the leading edge of a fold in the large intestine, blood vessels appearing on the surface of the living tissue, or a boundary portion that extends linearly to distinguish between a lesion site X and a healthy site. The linearly extending portion is preferably generally straight, but it can also be a curve with a smooth change in direction. When the curve is a smooth curve, for the part interrupted by the dead angle region R4, the positional offset of the end of the orthographic image relative to another linear image can be determined by extending the smooth curve image from a part that extends in a linear shape.

[0089] According to one embodiment, the side window 150 provided at the front end 132 is configured to surround the circumference of the cylindrical member 133 for surrounding the imaging element 108. In this case, the optical element 152 is preferably provided at each of a plurality of positions along the circumference, such that the deflection characteristics, i.e., the deflection direction of the incident light, are distributed circumferentially along the cylindrical member 133. By shifting the position of the front end 132 away from the center of the organ, the presence or absence of the overlapping region R3 and the dead angle region R4 changes circumferentially, and the sizes of the overlapping region R3 and the dead angle region R4 are also distributed circumferentially according to the amount of shift in the position of the front end 132. By providing the optical element 152 at each of a plurality of positions along the circumference of the side window 150, the overlapping region R3 and the dead angle region R4 in the image displayed on the display 300 can be reduced or even eliminated.

[0090] According to one embodiment, the maximum half-angle of view of the objective lens 106 is preferably 90 degrees or more, or even greater than 90 degrees. More preferably, the maximum half-angle of view is 100 degrees or more, further preferably 110 degrees or more, and particularly preferably 112 degrees or more. Using the electronic endoscope 100 in this way allows for imaging within a wide field of view; for example, it enables effective imaging of the presence of lesion sites X located at the base of folds in the large intestine or the like, which are obscured by fold collapse, in a side view image before fold collapse.

[0091] The control unit 115 or image processing unit 220 uses the camera image to determine the presence and size of the overlapping region R3 or the blind spot region R4. However, the presence and size of the overlapping region R3 or the blind spot region R4 can also be determined using a pre-built machine learning prediction model. The control unit 115 or image processing unit 220 has a prediction model (AI model) for pre-built machine learning of the relationship between the camera image, including the orthographic and side view images, and the size of the overlapping region R3 or the blind spot region R4. Using the prediction model, the control unit 115 or image processing unit 220 determines the presence and size of the overlapping region R3 or the blind spot region R4 based on the pixel values ​​of the camera image of the living tissue captured by the camera element 108, thereby determining the deflection characteristics (deflection direction) of the incident light incident on the optical elements 142 and 152. When the prediction model performs machine learning on the relationship between the camera image and the size of the blind spot region R4, for example, it performs machine learning in advance on the relationship between the positional offset position (the position where the positional offset exists) between the orthographic and side views of the linearly extending portion within the camera image and the positional offset amount and the size of the blind spot region R4.

[0092] Predictive models, when using neural networks, include models of deep neural networks (DNNs) that can be predictably formed through deep learning, models based on random forest methods using tree structures, models using LASSO regression, or nonlinear functions such as multinomials, Kriging models, and Radial Basis Function Networks (RBFNs).

[0093] When using a predictive model, information about the control signal V can be used in conjunction with camera images, including both front and side views, and machine learning can be performed beforehand to assess its relationship with the overlapping region R3 or the blind spot region R4. In this case, when the predictive model predicts and calculates the size of the overlapping region R3 or the blind spot region R4, information about the control signal V is input into the predictive model in addition to the camera images used as input data.

[0094] The endoscope and endoscope system provided by the present invention have been described in detail above. However, the present invention is not limited to the described embodiments, and various improvements and changes can be made without departing from the spirit of the present invention.

Claims

1. An endoscope used for imaging living tissue within a body cavity, wherein, The endoscope has the following features: The imaging element is configured to capture images of living tissue. The objective lens simultaneously images the live tissue as a camera image on the light-receiving surface through a front window that faces the front of the light-receiving surface of the imaging element and a side view of the live tissue that faces the side more towards the side than the front window. An optical element is disposed on at least one of the front window and the side window, and the field of view of at least one of the direct view image and the side view image is changed by adjusting the deflection characteristics of the incident light using a control signal. as well as The control unit is used to generate the control signal. The control unit generates the control signal to achieve at least one of the following: reducing the overlap area between the direct view image and the side view image in the camera image, which includes the direct view image and the side view image, and reducing the blind spot area in the field of view of the camera image.

2. The endoscope according to claim 1, wherein, The control unit extracts the overlapping region between the front view image and the side view image based on the pixel values ​​of the camera image, and determines the deflection characteristics of the incident light based on the size of the overlapping region, thereby generating the control signal.

3. The endoscope according to claim 2, wherein, The control unit has a predictive model for performing machine learning on the relationship between the size of the camera image, including the orthographic image and the side view image, and the overlapping region. By using the predictive model, the presence and size of the overlapping region are determined based on the pixel values ​​of the camera image of the living tissue captured by the camera element, thereby determining the deflection characteristics of the incident light.

4. The endoscope according to any one of claims 1 to 3, wherein, The control unit calculates the size of the dead zone region between the direct view image and the side view image based on the positional offset of the linearly extending portion of the live tissue in the direct view image and the linearly extending portion in the side view image captured by the camera element at the ends of the direct view image or the side view image, and determines the deflection characteristics of the incident light based on the size, thereby generating the control signal.

5. The endoscope according to claim 4, wherein, The control unit has a prediction model for performing machine learning on the relationship between the positional offset and positional offset of the linearly extending portion in the camera image and the size of the blind spot region. Using the prediction model, the positional offset and positional offset of the linearly extending portion are determined based on the pixel values ​​of the camera image of the living tissue captured by the camera element, thereby determining the presence and size of the blind spot region and thus determining the amount of change.

6. The endoscope according to any one of claims 1 to 3, wherein, The side window is configured to surround the circumference of the cylindrical member that surrounds the imaging element. The optical element is disposed at each of a plurality of locations along the circumference such that the deflection characteristics are distributed along the circumference.

7. The endoscope according to any one of claims 1 to 3, wherein, The objective lens has a maximum half-angle of 90 degrees or more.

8. An endoscope system comprising an endoscope for imaging living tissue within a body cavity, and an endoscope processor for image processing of images of the living tissue captured by the endoscope, wherein, The endoscope has the following features: The imaging element is configured to capture images of living tissue. The objective lens simultaneously images the live tissue as a camera image on the light-receiving surface, using a front window located in front of the light-receiving surface of the camera element as a direct view image and a side view image of the live tissue obtained through a side window that is more lateral than the front window. as well as An optical element, disposed on at least one of the front window and the side window, alters the field of view of at least one of the direct view image and the side view image by adjusting the deflection characteristics of the incident light using a control signal. The endoscopic processor includes an image processing unit that processes images of living tissue to generate control signals for controlling the deflection characteristics of the incident light. The image processing unit generates the control signal to satisfy at least one of the following: reducing the overlap area between the direct view image and the side view image in the camera image, which includes the direct view image and the side view image, and reducing the blind spot area in the field of view of the camera image.

Images