Endoscope system and method of operation thereof

Abstract

The present application provides an endoscope system which miniaturizes an optical system and obtains an in-focus image, and a working method thereof. A lens drive processor (44) drives a zoom lens (42) in accordance with a press of a freeze button (12h). An image acquisition processor selects, as an optimal image, a post-press evaluation frame image (84, 224) having the highest evaluation value from among post-press evaluation frame images (84, 85, 224, 225) in which evaluation values are calculated from examination images acquired in conjunction with the operation of the zoom lens (42) during a predetermined period of time after the press of the freeze button (12h), and displays the optimal image (100c).

Description

Technical Field

[0001] This invention relates to an endoscope system for displaying evaluated images and its operating method. Background Technology

[0002] After an endoscopic examination, diagnosis or postoperative evaluation is performed. Therefore, when lesions are found, specific processing is required to obtain static images. Sometimes, images captured during the examination may be out of focus on the observed area. One reason for this is that the distance between the endoscope and the observed area may be too close or too far during image capture, resulting in defocusing. In such cases, the endoscope can be moved back and forth to adjust the focus. However, the depth of field becomes shallower, especially when zooming in to magnify the observed area, making manual focusing by the doctor or other user difficult.

[0003] As a technology for automatically focusing on the subject, the autofocus mechanism of the endoscope disclosed in Patent Document 1 can be cited as an example. The oral endoscope in Patent Document 2 does not use an autofocus mechanism but acquires an image while the focusing lens automatically moves upon inputting a camera instruction, thereby achieving miniaturization and light weight. Furthermore, the endoscope in Patent Document 3 evaluates the brightness or jitter of the captured image and automatically selects an appropriate still image, reducing the burden on the user in selecting a suitable still image.

[0004] Previous technical documents

[0005] Patent documents

[0006] Patent Document 1: International Publication No. 2019 / 220584

[0007] Patent Document 2: Japanese Patent Application Publication No. 2016-209232

[0008] Patent Document 3: Japanese Patent Application Publication No. 2020-168164 Summary of the Invention

[0009] The technical problem to be solved by the invention

[0010] Problems such as the optical system used for imaging becoming larger and heavier have emerged in autofocus mechanisms. Therefore, there is a need for a technology that can miniaturize the optical system of the endoscope as much as possible without assembling an autofocus mechanism, while maintaining the narrow diameter of the front end, to obtain an image with the correct focus, and to provide this information to the user.

[0011] The purpose of this invention is to provide an endoscope system and its operating method that enables miniaturization of the optical system and obtains an image with a focused focal point.

[0012] means for solving technical problems

[0013] The endoscope system of the present invention illuminates a subject and captures light from the subject. The endoscope system includes an endoscope and a processor. The endoscope captures reflected light from the subject and includes a zoom lens and a lens driving processor for driving the zoom lens. The processor includes an image acquisition processor. The lens driving processor drives the zoom lens upon pressing a freeze button. The image acquisition processor performs the following processing: before pressing the freeze button, it acquires images in a time-series sequence, i.e., inspection images; during a predetermined time period before and after pressing the freeze button, i.e., a comprehensive evaluation time, and during the time period after pressing the freeze button, i.e., a post-press evaluation time period, it acquires inspection images in conjunction with the movement of the zoom lens driven by the lens driving processor; it calculates the evaluation value of the inspection images selected as post-press evaluation frames from the inspection images acquired during the post-press evaluation time period; it selects the post-press evaluation frame image with the highest evaluation value among the post-press evaluation frame images with calculated evaluation values ​​as the best image; and it displays the best image.

[0014] The preferred image acquisition processor performs the following processing: from the inspection images acquired within the comprehensive evaluation time, save the inspection images with an evaluation hold frame amount as evaluation hold frame images; and calculate the evaluation value of the saved evaluation hold frame images.

[0015] The preferred comprehensive evaluation time is the time that is the sum of the evaluation time after pressing and the time before pressing the freeze button, i.e., the evaluation time before pressing. The evaluation holding frame image consists of the evaluation frame image after pressing and the evaluation frame image before pressing, which includes the inspection image acquired during the evaluation time before pressing. The image acquisition is processed by the processor as follows: the evaluation values ​​of the evaluation frame image after pressing and the evaluation frame image before pressing are calculated.

[0016] The preferred image acquisition processor performs the following processing: within the evaluation hold frame range, during the pre-press evaluation time, the inspection image with a portion of the frames removed is saved as the pre-press evaluation frame image, and during the post-press evaluation time, the inspection images of all frames are saved as the post-press evaluation frame image.

[0017] The preferred image acquisition processor performs the following processing: within the evaluation hold frame range, a portion of the frames of the inspection images acquired within the comprehensive evaluation time are removed and saved as evaluation hold frame images.

[0018] The overall evaluation time is preferably at least 0.5 seconds. The post-press evaluation time is preferably at least 0.3 seconds. The evaluation value is preferably the contrast value. The evaluation hold frame is preferably at least 15 frames.

[0019] The image acquisition processor performs the following processing: selects the image with the highest evaluation value from the pre-press evaluation frame image as the best image before pressing; selects the image with the highest evaluation value from the post-press evaluation frame image as the best image after pressing; and selects and displays the image with the highest evaluation value between the best image before pressing and the best image after pressing as the best image.

[0020] The preferred image acquisition processor performs the following processing: when an evaluation hold-frame image with an evaluation value above the threshold is obtained, the evaluation hold-frame image with an evaluation value above the threshold is displayed as the best image.

[0021] The preferred image acquisition processor performs the following processing: after the freeze button is pressed, when the distance between the subject and the front end of the endoscope is within the range of 2mm to 6mm, the zoom lens is driven.

[0022] In the operating method of the endoscope system of the present invention, the endoscope system illuminates a subject and captures light from the subject. The endoscope system includes: an endoscope having a zoom lens and a lens driving processor for driving the zoom lens; and a processor device having an image acquisition processor. The operating method of the endoscope system includes the following steps: the lens driving processor drives the zoom lens according to the pressing of a freeze button; the image acquisition processor acquires images in a time sequence before the freeze button is pressed, i.e., inspection images; during a predetermined time before and after the freeze button is pressed, i.e., a comprehensive evaluation time, and during the time after the freeze button is pressed, i.e., a post-press evaluation time period, the inspection images are acquired in conjunction with the action of the zoom lens driven by the lens driving processor; an evaluation value is calculated for the inspection images selected as post-press evaluation frame images from the inspection images acquired during the post-press evaluation time period; the post-press evaluation frame image with the highest evaluation value among the post-press evaluation frame images for which the evaluation value is calculated is selected as the best image; and the best image is displayed.

[0023] Invention Effects

[0024] According to the endoscope system and its operating method of the present invention, an endoscope system and its operating method that can miniaturize the optical system and obtain an image with a focused focal point can be provided. Attached Figure Description

[0025] Figure 1 This is a structural diagram of an endoscope system.

[0026] Figure 2 It is a graph representing the spectrum of white light.

[0027] Figure 3 It is a block diagram representing the function of an endoscope system.

[0028] Figure 4 This is an explanatory diagram showing the driving mechanism of a zoom lens.

[0029] Figure 5 This is an explanatory diagram showing the depth of field.

[0030] Figure 6 It is an explanatory diagram showing the relationship between the position of the zoom lens and the distance between the subject and the depth of field.

[0031] Figure 7 This is an explanatory diagram representing the comprehensive evaluation time, the evaluation time before pressing, the evaluation time after pressing, the evaluation frame image before pressing, and the evaluation frame image after pressing.

[0032] Figure 8 It is an image of the setting screen.

[0033] Figure 9 It is an image of a custom screen.

[0034] Figure 10 This is an explanatory diagram showing the evaluation section and selection section when no threshold is set.

[0035] Figure 11 This is an explanatory diagram showing the evaluation section and selection section when the threshold is set to 350.

[0036] Figure 12 This is an explanatory diagram showing the evaluation section and selection section when the threshold is set to 300.

[0037] Figure 13 It is an image displayed on the screen.

[0038] Figure 14 This is a flowchart illustrating the functions of the first embodiment.

[0039] Figure 15 This is a block diagram illustrating the function of the evaluation unit in the second embodiment.

[0040] Figure 16 This is a block diagram illustrating the function of the selection unit in the second embodiment.

[0041] Figure 17 This is an explanatory diagram showing the functions of the pre-press evaluation section, the post-press evaluation section, the pre-press selection section, the post-press selection section, and the comprehensive selection section.

[0042] Figure 18 This is a flowchart illustrating the functions of the second embodiment. Detailed Implementation

[0043] [First Implementation]

[0044] exist Figure 1The endoscope system 10 includes an endoscope 12, a light source device 13, a processor device 14, a display 15, and a user interface 16. The endoscope 12 is optically connected to the light source device 13 and electrically connected to the processor device 14. The endoscope 12 has an insertion part 12a that is inserted into the body of the object being observed, an operation part 12b provided at the base of the insertion part 12a, and a bending part 12c and a front end part 12d provided at the front end side of the insertion part 12a. By operating the bending knob 12e of the operation part 12b, the bending part 12c is bent. The front end part 12d is oriented in the desired direction by the bending action of the bending part 12c.

[0045] In addition to the angle knob 12e, the operation unit 12b is also equipped with a freeze button 12h used for the acquisition of still images of the subject and a zoom operation unit 12i used for the operation of the zoom lens 42.

[0046] Endoscope 12 acquires a sequential series of images, i.e., examination images. In this specification, examination images include still images, evaluation hold frames, pre-press evaluation frames, and post-press evaluation frames. During the acquisition of examination images, when it is desired to acquire one frame of an examination image, i.e., a still image, the freeze button 12h is pressed, and a still image acquisition instruction is sent to endoscope 12 and processor device 14. A memory (not shown) for storing still images can be connected to processor device 14.

[0047] The processor device 14 is electrically connected to the display 15 and the user interface 16. The display 15 outputs and displays inspection images and information attached to the inspection images. The user interface 16 has a keyboard, mouse, touchpad, and microphone, and has input operation functions such as receiving function settings.

[0048] The light source unit 20 of the light source device 13 emits illumination light to illuminate the subject. The light source unit 20 includes, for example, a laser diode, an LED (Light Emitting Diode), a xenon lamp, or a halogen lamp, and emits at least... Figure 2 Illumination light (white light) in the spectrum shown. White light includes light that is substantially equivalent to white in the photographing of the subject using endoscope 12. Figure 2 The so-called simulated white light shown is a mixture of violet light (V), blue light (B), green light (G), or red light (R). Furthermore, the light source unit 20 includes filters, etc., to adjust the wavelength band, spectrum, or light intensity of the illumination light, as needed. The light source control unit 22 controls the lighting and extinguishing of each light source constituting the light source unit 20, as well as the amount of light emitted.

[0049] like Figure 3As shown, illumination light emitted by the light source 20 is incident on the endoscope 12 via the light guide 23. An illumination optical system 30 and an imaging optical system 40 are provided at the front end 12d of the endoscope 12. The illumination optical system 30 has an illumination lens 31, through which illumination light propagating via the light guide 23 illuminates the object being observed. The imaging optical system 40 has an objective lens 41, a zoom lens 42, and an image sensor 43. Light from the object being observed, based on the illumination light, is incident on the image sensor 43 via the objective lens 41 and the zoom lens 42. Thus, an image of the object being observed is formed in the image sensor 43.

[0050] Zoom lens 42 is a lens group used to magnify or reduce the observed object (subject). Specifically, such as... Figure 4 As shown, the endoscope 12 comprises four lens groups, G1, G2, G3, and G4, sequentially from the front end 12d. When observing structures such as capillaries or performing surgical procedures, or when zooming in to magnify the subject (i.e., performing close-up photography), the zoom operation unit 12i is operated to move the zoom lens groups G2 and G3 in the direction with a larger Step number to focus on the close-up. Conversely, when confirming the penetration range of a larger lesion or when observing the subject over a wider area (i.e., performing long-range photography), the zoom operation unit 12i is operated to move the zoom lens groups G2 and G3 in the direction with a smaller Step number to focus on the distant object.

[0051] The lens drive processor 44, according to the operation of the zoom operation unit 12i, causes the zoom lens groups G2 and G3 to move as follows: Figure 4 Move between the farthest point (Step 1) and the nearest point (Step 5) as shown. Figure 4 The diagram uses dotted lines to illustrate the approximate positions of zoom lens groups G2 and G3 from Step 1 to Step 5. Zoom lens groups G2 and G3 move towards the front end 12d or towards the side opposite to the front end 12d between positions Step 1 and Step 5. Meanwhile, zoom lens groups G1 and G4 are fixed. By moving zoom lens groups G2 and G3, the apparent focus range, i.e., the depth of field, changes. However, the movement of the lens groups is not limited to this.

[0052] like Figure 5 As shown, the depth of field changes depending on the position of zoom lens groups G2 and G3. When zoom lens groups G2 and G3 are located in Step 1 (reference...) Figure 4 When zoom lens groups G2 and G3 are located in Step 2 to Step 5 (see reference...), the depth of field increases (the area at focus widens). On the other hand, when zoom lens groups G2 and G3 are located in Step 2 to Step 5 (see reference...), the depth of field increases (the area at focus widens). Figure 4When zoom lens groups G2 and G3 are located in Step 1, the depth of field becomes shallower (the area for focusing becomes narrower). The shallower the depth of field, the more difficult it is to adjust the focus. Especially when... Figure 6 As shown, when taking close-up photographs of the subject (Ob) at high magnification, in addition to a shallower depth of field, the effects of subject-side movements such as creeping motion become more pronounced, making it more difficult to focus.

[0053] Figure 6 Examples illustrating the relationship between the positions of zoom lens groups G2 and G3, the distance between them and the subject Ob, and the depth of field are shown. In Example 1, zoom lens groups G2 and G3 are located at Step 2, suitable for observing a wide-range distant view, but the depth of field is near the front of the subject Ob, so the focus is off. Similarly, in Example 2, zoom lens groups G2 and G3 are located at Step 3, suitable for magnifying and observing a smaller close-up view, but the depth of field covers the inside of the subject Ob, resulting in a lack of focus. On the other hand, Example 3 is an example of close-up photography similar to Example 2, but zoom lens groups G2 and G3 are located between Step 2 and Step 3, with the depth of field precisely at the surface of the subject Ob, thus achieving focus. To obtain a still image with focus, as in Example 3, it is necessary to move zoom lens groups G2 and G3 so that the depth of field is at the surface of the subject Ob.

[0054] In addition to operating the zoom operation unit 12i, the lens drive processor 44 also drives the zoom lens groups G2 and G3 when the freeze button 12h is pressed. When the freeze button 12h is pressed, the zoom lens groups G2 and G3 move at a predetermined speed toward the front end 12d or the opposite side, and acquire inspection images in conjunction with the movement of the zoom lens groups G2 and G3 driven by the lens drive processor 44.

[0055] After pressing the freeze button for 12 hours, it is preferable to drive the zoom lens groups G2 and G3 when the distance between the subject and the tip of the endoscope 12 is within a range of 2mm to 6mm. This is because when the zoom lens groups G2 and G3 are in positions from Step 2 to Step 3, the distance between the subject and the tip of the endoscope 12 is within a range of 2mm to 6mm. Furthermore, this is a position where the depth of field changes significantly, making manual focus adjustment more difficult; therefore, this range is also preferable for assisting with automatic photography. Additionally, the range of movement for the zoom lens groups G2 and G3 is not limited to the range from Step 2 to Step 3 (where the distance between the subject and the tip of the endoscope 12 is 2mm to 6mm). The range of movement for the zoom lens 42 (zoom lens groups G1, G2, G3, and G4) can be arbitrarily set.

[0056] When driving the zoom lens groups G2 and G3, they can move from their current position towards the front end or the opposite side. Alternatively, they can temporarily move the zoom lens groups G2 and G3 to a starting position (e.g., position Step 2 or Step 3) and then move towards the front end or the opposite side. According to this structure, when capturing still images, the lenses can be automatically zoomed while the focus position changes, allowing for image capture simultaneously. Furthermore, by using a single actuator for the focusing mechanism instead of an autofocus mechanism, the narrow diameter of the endoscope can be maintained.

[0057] The camera sensor 43, which images the reflected light from the subject through the objective lens 41 and the zoom lens 42, is a CMOS (Complementary Metal Oxide Semiconductor) sensor, a CCD (Charge-Coupled Device) sensor, or the like.

[0058] In addition to a color camera sensor 43 that includes a color filter (such as a Bayer filter) that converts sensed light into a color image signal, a monochrome camera sensor 43 may also be included that does not include a color filter that converts sensed light into a monochrome image signal. Furthermore, the color camera sensor can convert sensed light into a CMY signal instead of an RGB signal.

[0059] When acquiring a color image, the image signal includes a B image signal output from the B pixel of the image sensor 43, a G image signal output from the G pixel of the image sensor 43, and an R image signal output from the R pixel of the image sensor 43. The image signal detected by the image sensor 43 is sent to the inspection image acquisition unit 51 of the processor device 14, where various signal processing and image processing, such as noise reduction, tone correction, and tone adjustment, are performed based on the image signal to generate a monochrome image or a color image, i.e., an inspection image.

[0060] The processor device 14 includes a central control unit 50, an image acquisition unit 51, a setting unit 60, an evaluation unit 80, a selection unit 90, and a display control unit 100. Furthermore, a temporary storage memory 17 is included in the processor device 14. In the processor device 14, the central control unit 50, which is composed of an image acquisition processor, executes the program stored in the memory, thereby realizing the functions of the image acquisition unit 51, the setting unit 60, the evaluation unit 80, the selection unit 90, and the display control unit 100.

[0061] In the setting unit 60, when the freeze button is pressed for 12 hours and one frame of inspection image (i.e., a still image) is acquired, a selection method and evaluation method for the candidate inspection image (i.e., the best image) that becomes the best focus point can be set from the inspection image (still image). During inspection, the inspection image is sent from the inspection image acquisition unit 51 to the setting unit 60, and the inspection image is sent to the evaluation unit 80 according to the settings of the setting unit 60. Hereinafter, the setting unit 60 will be described in detail.

[0062] Before or during the inspection, users can select which inspection image to evaluate before or after pressing the freeze button for 12 hours. Specifically, in addition to Figure 7 In addition to the overall evaluation time, pre-press evaluation time, and post-press evaluation time shown, thresholds for evaluation frames and evaluation values ​​are also set. An evaluation frame refers to the frame of the inspection image that is used to calculate the evaluation value.

[0063] refer to Figure 7 The method for selecting inspection images to be evaluated will be explained. The comprehensive evaluation time refers to the predetermined time before and after the freeze button is pressed for 12 hours, during which the inspection images SI are acquired and sent to the evaluation unit 80. If the comprehensive evaluation time is too long, the number of inspection images SI to be evaluated increases, putting pressure on the processing power of the processor device 14 and the capacity of the temporary storage memory 17. If the comprehensive evaluation time is too short, the number of inspection images SI that become the best image candidates decreases. Therefore, the comprehensive evaluation time is preferably at least 0.5 seconds, and more preferably 1.0 second. Furthermore, a comprehensive evaluation time of 1.0 second is appropriate when the frame rate of the camera sensor 43 is 60fps.

[0064] Additionally, one frame refers to a time period, including at least the period from the moment the image signal is received in the camera sensor 43 until the signal readout is complete (the period during which one inspection image can be obtained). The overall evaluation time can be arbitrarily set in the setting screen 70 described later.

[0065] In the total evaluation time, the time before pressing the freeze button for 12 hours is the pre-press evaluation time, and the time after pressing the freeze button for 12 hours is the post-press evaluation time. During the post-press evaluation time, the inspection image is acquired in conjunction with the operation of the zoom lens groups G2 and G3 driven by the lens drive processor 44. The post-press evaluation time is preferably the time required for the zoom lens groups G2 and G3 to move between each Step. For example, when the zoom lens groups G2 and G3 move from Step 2 (reference...) Figure 4 , Figure 5 Move the position of ) to Step 3 (refer to) Figure 4 , Figure 5 When the time required for the position is 0.33 seconds, the evaluation time after pressing is preferably at least 0.3 seconds.

[0066] The pre-press evaluation time is the time obtained by subtracting the post-press evaluation time from the overall evaluation time. For example, when the overall evaluation time is 1.0 second and the post-press evaluation time is 0.3 seconds, the pre-press evaluation time is 0.7 seconds. Furthermore, the pre-press evaluation time and post-press evaluation time can be arbitrarily set in the setting screen 70 described later. It is also possible to set either the pre-press evaluation time or the post-press evaluation time to 0 seconds.

[0067] The image of the frame selected as the evaluation object by the frame setting described later from the inspection images acquired during the pre-press evaluation time is set as the pre-press evaluation frame image. Similarly, the image of the frame selected as the evaluation object by the frame setting from the inspection images acquired during the post-press evaluation time is set as the post-press evaluation frame image. The pre-press evaluation frame image and the post-press evaluation frame image are stored in the temporary storage memory 17 within the processor device 14 and sent to the evaluation unit 80.

[0068] From such Figure 8 The setup screen 70 shown allows users to set evaluation times, including overall evaluation time, pre-press evaluation time, and post-press evaluation time, via the user interface 16. The overall evaluation time, pre-press evaluation time, and post-press evaluation time can be input from the input box 72. The input box 72 is not limited to this; it can also be used for tab input or voice input. Based on this structure, the inspection image to be evaluated can be set with reference to the evaluation time, thus avoiding pressure on the processor's processing power and the temporary storage 17.

[0069] In the setting screen 70, in addition to the evaluation time, the evaluation frame can also be set. In the evaluation frame setting, the number of inspection images (frames) stored in the temporary storage memory 17 during the comprehensive evaluation time, i.e., the evaluation holding frames, can be set. If there are too many evaluation holding frames, the number of inspection images to be evaluated increases, putting pressure on the processing power of the processor device 14 and the capacity of the temporary storage memory 17. If there are too few evaluation holding frames, the number of inspection images that become the best image candidates decreases. Therefore, the evaluation holding frames are preferably at least 15 frames. In the setting screen 70, the evaluation holding frames can be input from the input box 72.

[0070] In the settings screen 70, you can also input the number of evaluation frames before and after pressing the freeze button (12 hours) from the input box 72. Adding the evaluation frames before and after pressing the freeze button yields the number of evaluation frames held. Furthermore, when the custom button 76 is selected, you can also... Figure 9 Configure it in the custom screen 76a shown.

[0071] In such Figure 9In the custom screen 76a shown, the evaluation frame pattern (hereinafter referred to as the pattern) of the inspection image that is to be evaluated can be arbitrarily set. In the setting screen 70, the evaluation time and evaluation frame settings input from the input box 72 can be displayed.

[0072] exist Figure 9 In the specific examples represented by patterns 1 to 5, frame 76b with a diagonal line indicates the frame to be evaluated, and blank frame 76c indicates a frame that is not to be evaluated. In the custom screen 76a, frames can be selected by alternating between selecting a frame via user interface 16 (which makes it an evaluation target frame) and selecting it again (which deselects it), but the method of frame selection is not limited to this. A template of the evaluation frame pattern represented by patterns 1 to 5 can also be selected via radio button 76d. Additionally, Figure 9 A custom frame 76a is shown when the endoscope 12 is at a frame rate of 60fps.

[0073] Figure 9 Pattern 1 is an example where, when the overall evaluation time is 1.0 second, the post-press evaluation time is 0.3 seconds, and the evaluation hold frame is 30 frames, during the pre-press evaluation time, an inspection image is saved as the pre-press evaluation frame image every other frame, and during the post-press evaluation time, inspection images from all frames are saved as the post-press evaluation frame images. Pattern 1 is an evaluation frame pattern intended to prioritize the post-press evaluation frame image over the pre-press evaluation frame image within the limited overall evaluation time and evaluation hold frame.

[0074] Figure 9 Pattern 2 is an example where, when the overall evaluation time is 1.0 second, the post-press evaluation time is 0.3 seconds, and the evaluation hold frame is 30 frames, the inspection image acquired within the overall evaluation time is saved as the evaluation hold frame image every other frame. The evaluation hold frame image is the inspection image acquired within the evaluation hold frame. In other words, Pattern 2 is an evaluation frame pattern that saves the inspection image every other frame during both the pre-press evaluation time and the post-press evaluation time. Pattern 2 is an evaluation frame pattern designed to acquire both the post-press evaluation frame image and the pre-press evaluation frame image in a balanced manner within the limited overall evaluation time and evaluation hold frame.

[0075] Figure 9 Pattern 3 is an example as follows: when the overall evaluation time is 1.0 seconds, the post-press evaluation time is 0.4 seconds, and the evaluation hold frame is 30 frames, during the pre-press evaluation time, the inspection image is saved as the pre-press evaluation frame image every 1 frame, and during the post-press evaluation time, the inspection images of all frames are saved as the post-press evaluation frame images.

[0076] Figure 9Pattern 4 is an example as follows: when the overall evaluation time is 1.0 seconds, the post-press evaluation time is 0.4 seconds, and the evaluation hold frame is 30 frames, the inspection image obtained during the overall evaluation time is saved as the evaluation hold frame image every 1 frame.

[0077] Figure 9 Pattern 5 is an example as follows: when the overall evaluation time is 1.0 second, the post-press evaluation time is 0.4 seconds, and the evaluation hold frame is 30 frames, during the pre-press evaluation time, the inspection image is saved as the pre-press evaluation frame image every 1 frame. During the post-press evaluation time, 4 frames after the freeze button is pressed are saved as the post-press evaluation frame image. 1 frame is skipped, and the next 3 frames are saved as the post-press evaluation frame image. 2 frames are skipped, and the next 2 frames are saved as the post-press evaluation frame image. 3 frames are skipped, and then the post-press evaluation frame image is saved every 3 frames.

[0078] Pattern 5 is an evaluation frame pattern designed to accommodate situations where the user presses the freeze button 12h while the focus is aligned, and it is desired to add a post-press evaluation frame image during the initial phase of the post-press evaluation time. Alternatively, an inspection image can be acquired at the moment the freeze button 12h is pressed and added to either the pre-press evaluation frame image or the post-press evaluation frame image.

[0079] It is possible to create evaluation frame patterns arbitrarily. For example, in Pattern 1, the inspection image can be saved as the pre-press evaluation frame image every 2 or 3 frames or more during the pre-press evaluation time. Furthermore, in Pattern 2, the inspection image acquired during the comprehensive evaluation time can be saved as the evaluation hold frame image every 2 or 3 frames or more.

[0080] According to the above settings, the inspection images are stored in the temporary storage memory 17 within the processor device 14. When the freeze button 12h is pressed, firstly, the inspection images of the evaluation holding frames during the pre-press evaluation time are stored in the temporary storage memory 17. Next, when the inspection images are acquired in conjunction with the driving of the zoom lens groups G2 and G3 during the post-press evaluation time and stored in the temporary storage memory 17, the earliest pre-press evaluation frame images in the time sequence are deleted sequentially. In this way, the post-press evaluation frame images are stored in the temporary storage memory 17 in a manner that limits them to the evaluation holding frames, and the inspection images are stored without putting pressure on the temporary storage memory 17.

[0081] Furthermore, when the evaluation hold frame is not limited to the overall evaluation time, either the overall evaluation time or the evaluation hold frame can be prioritized. The choice between prioritizing the overall evaluation time and the evaluation hold frame can be arbitrarily set. Based on the above structure, the inspection image to be evaluated can be set through a frame selection method, thus avoiding pressure on the processor and memory.

[0082] In setting screen 70, a threshold for the evaluation value can be set. The evaluation value is the pixel value of the evaluation hold frame image calculated by the evaluation unit 80. When a threshold is set for the evaluation value, if an evaluation hold frame image with an evaluation value above the threshold is obtained, that evaluation hold frame image can be set as the best image (see below). Figure 11 and Figure 12 The threshold for the evaluation value can be entered from input box 72.

[0083] According to the settings of the setting unit 60, the pre-press evaluation frame image and the post-press evaluation frame image are sent to the evaluation unit 80. The evaluation unit 80 calculates the evaluation value based on the evaluation hold frame image. The evaluation hold frame image is one or more images that combine the pre-press evaluation frame image and the post-press evaluation frame image, and is an inspection image stored in the temporary storage memory 17 within the scope of the evaluation hold frame. The evaluation value is preferably a contrast value. In addition, the evaluation value may also use edge amount, frequency, smoothness, and brightness, etc.

[0084] The evaluation unit 80 calculates the evaluation value and sends the pre-press evaluation frame image and the post-press evaluation frame image sequentially to the selection unit 90. In the selection unit 90, as... Figure 10 As shown, among the evaluation frame image before the press and the evaluation frame image after the press (evaluation hold frame image) for which the evaluation value is calculated, the inspection image with the highest evaluation value is selected as the best image. The evaluation hold frame image selected as the best image is sent to the display control unit 100.

[0085] right Figure 10 The functions of the evaluation unit and selection unit when no threshold is set will be explained using a specific example. Inspection image A81 is the pre-press evaluation frame image with a calculated evaluation value of 10, inspection image B82 is the pre-press evaluation frame image with a calculated evaluation value of 50, inspection image C83 is the pre-press evaluation frame image with a calculated evaluation value of 300, inspection image D84 is the pre-press evaluation frame image with a calculated evaluation value of 390, and inspection image E85 is the pre-press evaluation frame image with a calculated evaluation value of 200. Furthermore, the inspection images are acquired in the order of inspection images A81, B82, C83, D84, and E85. Inspection images A81 (evaluation value 10), B82 (evaluation value 50), C83 (evaluation value 300), D84 (evaluation value 390), and E85 (evaluation value 200) are sent to the selection unit 90. The selection unit 90 selects the inspection image D84 (evaluation value 390) with the highest evaluation value as the optimal image and sends it to the display control unit 100.

[0086] When a threshold value for the evaluation value is set via the setting unit 60, such as Figure 11 , Figure 12 For example, select the best image. Figure 11A specific example is shown where the threshold for the evaluation value is 350. In this case, the evaluation value of the inspected image D84 is calculated to be 390, and at the point when it is sent to the selection unit 90, the selection unit 90 selects the inspected image D84 (evaluation value 390) with an evaluation value above the threshold (350) as the best image. Figure 10 Unlike other examples, when the optimal image is selected, a signal is sent to the lens drive processor 44 to indicate this, stopping the operation of zoom lens groups G2 and G3 even during the evaluation period after the image is pressed. That is, in Figure 11 In a specific example, inspection images A81 (evaluation value 10), B82 (evaluation value 50), C83 (evaluation value 300), and D84 (evaluation value 390) can be obtained, but inspection image E85 will not be obtained.

[0087] Figure 12 A specific example is shown where the threshold for the evaluation value is 300. In this case, the evaluation value of the previous evaluation frame image, i.e., the inspection image C83, is calculated to be 300. At the time of sending the value to the selection unit 90, the selection unit 90 selects the inspection image C83 (evaluation value 300) marked with an evaluation value above the threshold (300) as the best image. In this case, a signal transmitting this situation is sent to the lens drive processor 44 at the time the best image is selected. Even if the freeze button is pressed for 12 hours, the zoom lens groups G2 and G3 will not be driven, and the inspection image C83 will be displayed as the best image. That is, in Figure 12 In specific examples, with Figure 10 The examples differ; inspection images A81 (evaluation value 10), B82 (evaluation value 50), and C83 (evaluation value 300) can be acquired, but inspection images D84 and E85 will not be acquired. Based on this structure, when an image is acquired that is in focus, the zoom lens is not moved, thereby shortening the time required to present the best image to the user.

[0088] The display control unit 100 generates, for example, Figure 13 The display screen 100a, as shown, is output to the monitor 15. The display screen 100a displays the current inspection image 100b and the optimal image 100c. If the save button 100d on the display screen 100a is selected via the user interface 16, the optimal image 100c is saved to a recording medium external to the processor device 14, such as a report memory (not shown). Furthermore, if the cancel button 100e on the display screen 100a is selected, the display of the optimal image is canceled, and the system returns to its normal display state, showing only the current inspection image 100b. With this structure, the optimal image can be checked and saved during inspection.

[0089] according to Figure 14The flowchart shown illustrates a series of processes in this embodiment from acquiring the inspection image to displaying the display screen 100a. First, the inspection image acquisition unit 51 acquires the inspection image (step S101) and sends it to the evaluation unit 80 according to the settings of the setting unit 60 (step S102). Next, the evaluation value is calculated (step S103). The selection unit 90 selects the best image (step S104), the display control unit 100 generates the display screen 100a, and displays it on the display 15 (step S105).

[0090] [Second Implementation]

[0091] In the second embodiment, such as Figure 15 As shown, the evaluation unit 80 is provided with a pre-press evaluation unit 201 and a post-press evaluation unit 202. Furthermore, as... Figure 16 As shown, the selection unit 90 is provided with a pre-press selection unit 211, a post-press selection unit 212, and a comprehensive selection unit 213. Other structures are the same as in the first embodiment.

[0092] In this embodiment, the process continues as in the first embodiment until the inspection image is acquired and sent to the evaluation unit 80 according to the settings of the setting unit 60, and the selection unit 90 sends the best image to the display control unit 100. Therefore, the description is omitted.

[0093] In this embodiment, such as Figure 17 As illustrated, if an inspection image is sent to the evaluation unit 80, the pre-press evaluation frame image is sent to the pre-press evaluation unit 201, and the post-press evaluation frame image is sent to the post-press evaluation unit 202. Evaluation values ​​for both the pre-press and post-press evaluation frame images are then calculated. The pre-press evaluation frame image with the calculated evaluation value is sent to the pre-press selection unit 211, and the post-press evaluation frame image with the calculated evaluation value is sent to the post-press selection unit 212.

[0094] In the pre-press selection unit 211, the image with the highest evaluation value is selected from the pre-press evaluation frame images as the best image before pressing, and it is sent to the comprehensive selection unit 213. Similarly, in the post-press selection unit 212, the image with the highest evaluation value is selected from the post-press evaluation frame images as the best image after pressing, and it is sent to the comprehensive selection unit 213. In the comprehensive selection unit 213, the image with the highest evaluation value is selected as the best image from both the pre-press best image and the post-press best image, and it is sent to the display control unit 100.

[0095] right Figure 17A specific example will be described in detail. Inspection image F221 is the pre-press evaluation frame image with a calculated evaluation value of 10, inspection image G222 is the pre-press evaluation frame image with a calculated evaluation value of 50, and inspection image H223 is the pre-press evaluation frame image with a calculated evaluation value of 300. Furthermore, the inspection images are acquired in the order of inspection image F221, inspection image G222, and inspection image H223. Inspection image F221 (evaluation value 10), inspection image G222 (evaluation value 50), and inspection image H223 (evaluation value 300) are sent to the pre-press selection unit 211. The pre-press selection unit 211 selects the inspection image H223 (evaluation value 300) with the highest evaluation value from inspection image F221 (evaluation value 10), inspection image G222 (evaluation value 50), and inspection image H223 (evaluation value 300) as the pre-press best image and sends it to the comprehensive selection unit 213.

[0096] Inspection image I224 is the post-press evaluation frame image with a calculated evaluation value of 390, and inspection image J225 is the post-press evaluation frame image with a calculated evaluation value of 200. Furthermore, the inspection images are acquired in the order of inspection images I224 and J225. Inspection images I224 (evaluation value 390) and J225 (evaluation value 200) are sent to the post-press selection unit 212. The post-press selection unit 212 selects the inspection image I224 (evaluation value 390) with the highest evaluation value from inspection images I224 (evaluation value 390) and J225 (evaluation value 200) as the best post-press image and sends it to the comprehensive selection unit 213.

[0097] The comprehensive selection unit 213 compares the evaluation values ​​of the best image before pressing, i.e., inspection image H223 (evaluation value 300), and the best image after pressing, i.e., inspection image I224 (evaluation value 390). Here, the evaluation value of inspection image H223 is 300, and the evaluation value of inspection image I224 is 390. Therefore, the comprehensive selection unit 213 selects inspection image I224 (evaluation value 390) with the higher evaluation value as the best image and sends it to the display control unit 100. The display screen 100a generated by the display control unit 100 is the same as in the first embodiment, so the description is omitted.

[0098] The setting unit 60 can be used to set the optimal image from the optimal image before pressing and the optimal image after pressing. Furthermore, when selecting the optimal image using the method of this second embodiment, no threshold is set. According to the above structure, the optimal image can be selected from as many inspection images as possible.

[0099] according to Figure 18The flowchart shown illustrates a series of processes in this embodiment from acquiring the inspection image to displaying the display screen 100a. The pre-press evaluation frame image is sent to the pre-press evaluation unit 201, and an evaluation value is calculated (step S201). The post-press evaluation frame image is sent to the post-press evaluation unit 202, and an evaluation value is calculated (step S202). The pre-press selection unit 211 selects the image with the highest evaluation value from the pre-press evaluation frame image as the best image before pressing, and sends it to the comprehensive selection unit 213 (step S203). The post-press selection unit 212 selects the image with the highest evaluation value from the post-press evaluation frame image as the best image after pressing, and sends it to the comprehensive selection unit 213 (step S204). The comprehensive selection unit 213 selects the image with the higher evaluation value from the best pre-press image and the best post-press image as the best image, and sends it to the display control unit 100 (step S205).

[0100] In the first and second embodiments, the example described is of a processor device 14 installed in an endoscope system 10; however, the present invention is not limited to this, and other medical devices may be used. Furthermore, the endoscope 12 may be a rigid or flexible endoscope. Additionally, some or all of the central control unit 50, image acquisition unit 51, setting unit 60, evaluation unit 80, selection unit 90, and display control unit 100 in the endoscope system 10 may be installed, for example, in a medical image processing device that communicates with the processor device 14 and cooperates with the endoscope system 10. For example, it may be installed in a diagnostic support device that acquires images captured by the endoscope 12 directly from the endoscope system 10 or indirectly from a PACS (Picture Archiving and Communication Systems). Furthermore, a portion or all of the central control unit 50, examination image acquisition unit 51, setting unit 60, evaluation unit 80, selection unit 90, and display control unit 100 of the endoscope system 10 can be installed in a medical service support device that is connected via a network to various examination devices, including the first examination device, the second examination device, ..., the Nth examination device of the endoscope system 10.

[0101] In this embodiment, the hardware structure of the processing units that perform various processes, such as the central control unit 50, the image acquisition unit 51, the setting unit 60, the evaluation unit 80, the selection unit 90, and the display control unit 100, is as shown below. These processors include general-purpose processors that execute software (programs) and function as various processing units, such as CPUs (Central Processing Units), FPGAs (Field Programmable Gate Arrays), and programmable logic devices (PLDs) whose circuit structures can be modified after manufacturing, as well as dedicated circuits with circuit structures specifically designed for performing various processes.

[0102] A processing unit can be composed of one of these various processors, or it can be composed of a combination of two or more processors of the same or different types (e.g., multiple FPGAs or a combination of a CPU and an FPGA). Furthermore, a single processor can also constitute multiple processing units. As examples of a single processor constituting multiple processing units, firstly, in computers such as client or server computers, a processor is composed of a combination of one or more CPUs and software, and this processor functions as multiple processing units. Secondly, in systems on a chip (SoC), a processor is used to implement the functionality of the entire system, including multiple processing units, using a single IC (Integrated Circuit) chip. Thus, various processing units are constructed as hardware structures using one or more of the aforementioned processors.

[0103] Furthermore, more specifically, the hardware architecture of these various processors is a circuit that combines circuit elements such as semiconductor components. And the hardware architecture of the storage section is a storage device such as an HDD (hard disk drive) or an SSD (solid state drive).

[0104] Symbol Explanation

[0105] 10-Endoscope system, 12-Endoscope, 12a-Insertion section, 12b-Operating section, 12c-Bend section, 12d-Front end section, 12e-Angle knob, 12h-Freeze button, 12i-Zoom operating section, 13-Light source device, 14-Processor device, 15-Display, 16-User interface, 17-Temporary storage memory, 20-Light source unit, 22-Light source control unit, 23-Light guide, 30-Illumination optical system, 31-Illumination lens, 40-Camera optical system, 41-Objective lens, 42-Zoom lens, 43-Camera sensor, 44-Lens drive processor, 50-Central control unit, 51-Inspection image acquisition unit, 60-Setting unit, 70-Setting screen, 72-Input box, 76-Custom button, 76a-Auto Define the screen: 76b - Frame with diagonal lines, 76c - Blank frame, 76d - Radio button, 80 - Evaluation section, 81 - Check image A, 82 - Check image B, 83 - Check image C, 84 - Check image D, 85 - Check image E, 90 - Selection section, 100 - Display control section, 100a - Display screen, 100b - Current check image, 100c - Best image, 100d - Save button, 100e - Cancel button, 201 - Evaluation section before pressing, 202 - Evaluation section after pressing, 211 - Selection section before pressing, 212 - Selection section after pressing, 213 - Comprehensive selection section, 221 - Check image F, 222 - Check image G, 223 - Check image H, 224 - Check image I, 225 - Check image J.

Claims

1. An endoscope system that illuminates a subject and captures light from the subject, the endoscope system comprising: An endoscope having a zoom lens and a lens driving processor for driving the zoom lens; and Processor device, having an image acquisition processor, The processor for driving the lens performs the following processing: Pressing the freeze button activates the zoom lens. The image acquisition processor performs the following processing: Before pressing the freeze button, acquire images in a time-series sequence, i.e., inspect the images; The inspection image is obtained in conjunction with the action of the zoom lens driven by the lens drive processor during the specified time before and after pressing the freeze button, i.e., the comprehensive evaluation time before and after pressing the freeze button, i.e., the time after pressing the freeze button. Calculate the evaluation value of the inspection image selected as the post-press evaluation frame image from the inspection images acquired within the post-press evaluation time. The image with the highest calculated evaluation value among the post-press evaluation frame images is selected as the best image; and Display the optimal image. The image acquisition processor performs the following processing: From the inspection images acquired within the comprehensive evaluation time period, the inspection images with an evaluation hold frame count are saved as evaluation hold frame images; and Calculate the evaluation value of the saved evaluation-preserving frame image. The comprehensive evaluation time is the sum of the evaluation time after pressing the button and the evaluation time before pressing the freeze button. The evaluation hold frame image is composed of the post-press evaluation frame image and the pre-press evaluation frame image, which includes the inspection image acquired during the pre-press evaluation time. The image acquisition processor performs the following processing: Calculate the evaluation values ​​of the post-press evaluation frame image and the pre-press evaluation frame image. The image acquisition processor performs the following processing: Within the range of the evaluation hold frame, The inspection image with a portion of frames removed is saved as the pre-press evaluation frame image during the pre-press evaluation time, and the inspection image with all frames saved is saved as the post-press evaluation frame image during the post-press evaluation time.

2. The endoscope system according to claim 1, wherein, The image acquisition processor performs the following processing: Within the range of the evaluation hold frame, A portion of the frames from the inspection images acquired within the comprehensive evaluation time period are removed and saved as the evaluation hold frame images.

3. The endoscope system according to claim 1, wherein, The comprehensive evaluation time is at least 0.5 seconds.

4. The endoscope system according to claim 2, wherein, The comprehensive evaluation time is at least 0.5 seconds.

5. The endoscopic system according to any one of claims 1 to 4, wherein, The evaluation time after pressing is at least 0.3 seconds.

6. The endoscope system according to any one of claims 1 to 4, wherein, The evaluation value is the contrast value.

7. The endoscope system according to claim 5, wherein, The evaluation value is the contrast value.

8. The endoscope system according to claim 1 or 2, wherein, The evaluation frame rate is at least 15 frames.

9. The endoscope system according to claim 1, wherein, The image acquisition processor performs the following processing: Select the image with the highest evaluation value from the pre-press evaluation frame images as the best image before pressing; Select the image with the highest evaluation value from the post-press evaluation frame images as the best post-press image; and The image with the higher evaluation value among the best image before pressing and the best image after pressing is selected as the best image.

10. The endoscope system according to claim 1 or 2, wherein, The image acquisition processor performs the following processing: When an evaluation hold-frame image with an evaluation value above the threshold is obtained, the evaluation hold-frame image with an evaluation value above the threshold is displayed as the best image.

11. The endoscope system according to any one of claims 1 to 4, wherein, The image acquisition processor performs the following processing: After the freeze button is pressed, the zoom lens is activated when the distance between the subject and the front end of the endoscope is between 2mm and 6mm.

12. A method of operating an endoscope system, the endoscope system illuminating a subject and capturing light from the subject, the endoscope system comprising: An endoscope having a zoom lens and a lens driving processor for driving the zoom lens; and The processor device includes an image acquisition processor, and the method of operating the endoscope system includes the following steps: The lens driving processor drives the zoom lens when the freeze button is pressed; Before the freeze button is pressed, the image acquisition processor acquires images in a time sequence, i.e., inspection images; The inspection image is obtained in conjunction with the action of the zoom lens driven by the lens drive processor during the specified time before and after pressing the freeze button, i.e., the comprehensive evaluation time before and after pressing the freeze button, i.e., the time after pressing the freeze button. Calculate the evaluation value of the inspection image selected as the post-press evaluation frame image from the inspection images acquired within the post-press evaluation time. The image with the highest calculated evaluation value among the post-press evaluation frame images is selected as the best image; and Display the optimal image. The image acquisition processor also performs the following steps: From the inspection images acquired within the comprehensive evaluation time period, the inspection images with an evaluation hold frame count are saved as evaluation hold frame images; and Calculate the evaluation value of the saved evaluation-preserving frame image. The comprehensive evaluation time is the sum of the evaluation time after pressing the button and the evaluation time before pressing the freeze button. The evaluation hold frame image is composed of the post-press evaluation frame image and the pre-press evaluation frame image, which includes the inspection image acquired during the pre-press evaluation time. The image acquisition processor also performs the following steps: Calculate the evaluation values ​​of the post-press evaluation frame image and the pre-press evaluation frame image. The image acquisition processor also performs the following steps: Within the range of the evaluation holding frames, the inspection image with a portion of the frames removed is saved as the pre-press evaluation frame image during the pre-press evaluation time, and the inspection image with all frames saved is saved as the post-press evaluation frame image during the post-press evaluation time.

Images