Information processing device, operation method, program, and endoscope system

By detecting and adjusting for vignetting in endoscopic images, the device ensures accurate focus, exposure, and white balance, addressing the challenges of varying endoscope types and improving surgical image quality.

JP2024166295AActive Publication Date: 2024-11-28SONY GROUP CORP
View PDF 12 Cites 0 Cited by

Patent Information

Application Number
JP2024154938
Authority / Receiving Office
JP · JP
Patent Type
Applications
Current Assignee / Owner
Filing Date
2024-09-09
Publication Date
2024-11-28
Estimated Expiration
2038-03-20

AI Technical Summary

Technical Problem

Endoscopic images often have an extremely dark mask region near the edge due to optical shadow (vignetting), which complicates automatic focus adjustment, and the position and size of this region change depending on the type of endoscope used, making fixed focus evaluation areas ineffective.

Method used

An information processing device that detects the vignetting area in the endoscopic image and calculates an evaluation value to determine the effective area for focus, exposure, and white balance processing, adjusting these parameters based on the detected area to ensure accurate image capture.

Benefits of technology

This approach allows for the capture of endoscopic images suitable for surgery by reliably focusing on the surgical site and adjusting exposure and white balance, improving image quality.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure 2024166295000001_ABST
    Figure 2024166295000001_ABST
Patent Text Reader

Abstract

To capture an endoscopic image suitable for surgery.SOLUTION: An information processing device calculates a first evaluation value by performing detection of a detection frame set for a captured endoscopic image, detects an effective region of the captured image, which is a region free of vignetting by a scope of the endoscope, on the basis of the first evaluation value, and performs at least one of automatic focusing processing, automatic exposure processing, and automatic white balance processing on the basis of a second evaluation value with an evaluation value calculation target region set in the effective region as a target. This technology can be applied to an endoscope system.SELECTED DRAWING: Figure 12
Need to check novelty before this filing date? Find Prior Art

Description

[Technical field]

[0001] The present technology relates to an information processing device, an operating method, a program, and an endoscope system, and more particularly to an information processing device, an operating method, a program, and an endoscope system that are capable of capturing endoscopic images suitable for surgery. [Background technology]

[0002] In recent years, minimally invasive endoscopic surgery has been attracting attention. In order to obtain endoscopic images suitable for surgery, it has been proposed to equip endoscopes with AF (Auto Focus), AE (Automatic Exposure), and AWB (Auto White Balance) functions.

[0003] For example, Patent Document 1 discloses a technique for calculating a focus evaluation value based on an endoscopic image and performing AF for the endoscope. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] JP 2013-80108 A Summary of the Invention [Problem to be solved by the invention]

[0005] Normally, extremely dark masked areas are generated near the edges of an endoscopic image due to optical shadows (vignetting) caused by the scope. If AF is performed based on such an endoscopic image, the focus may be adjusted based on the masked area.

[0006] Furthermore, when the endoscope is a rigid scope, the position and size of the mask area change depending on the type of scope attached, so the area to be used for calculating the evaluation value cannot be fixed in advance.

[0007] The present technology has been made in consideration of such circumstances, and makes it possible to capture endoscopic images suitable for surgery. [Means for solving the problem]

[0008] An information processing device according to one aspect of the present technology includes a detection unit that performs detection within a detection frame set on an image captured by an endoscope and calculates a first evaluation value, a detection unit that detects an effective area of ​​the captured image, which is an area free of vignetting caused by a scope possessed by the endoscope, based on the first evaluation value, and a control unit that executes at least one of an automatic focusing process, an automatic exposure process, and an automatic white balance process based on a second evaluation value targeted at an evaluation value calculation target area set within the effective area.

[0009] An endoscope system according to another aspect of the present technology includes a light source device that irradiates light onto an operative field region, an endoscope that images the operative field region, and an information processing device connected to the image capturing device and the light source device. The information processing device includes a detection unit that performs detection within a detection frame set in an image captured by the endoscope and calculates a first evaluation value, a detection unit that detects an effective area of ​​the captured image, which is an area without vignetting due to a scope of the endoscope, based on the first evaluation value, and a control unit that performs at least one of an automatic focusing process, an automatic exposure process, and an automatic white balance process based on a second evaluation value targeted at an evaluation value calculation target area set within the effective area.

[0010] In this technology, a first evaluation value is calculated by performing detection within a detection frame set on an image captured by an endoscope, an effective area of ​​the captured image, which is an area without vignetting due to the scope possessed by the endoscope, is detected based on the first evaluation value, and at least one of automatic focusing processing, automatic exposure processing, and automatic white balance processing is executed based on a second evaluation value targeted at an evaluation value calculation target area set within the effective area. Effect of the Invention

[0011] According to the present technology, it is possible to capture endoscopic images suitable for surgery.

[0012] Note that the effects described herein are not necessarily limited to those described herein, and may be any of the effects described in this disclosure. [Brief description of the drawings]

[0013] [Figure 1] FIG. 1 is a diagram illustrating an example of the configuration of an endoscope system to which the present technology is applied. [Diagram 2] FIG. 1 is a perspective view showing the appearance of an endoscope. [Diagram 3] FIG. 2 is a diagram showing an example of a captured image. [Figure 4] FIG. 2 is a diagram showing an example of a captured image. [Diagram 5] FIG. 2 is a block diagram showing a configuration example of a CCU and an endoscope. [Figure 6] FIG. 13 is a diagram illustrating an example of detection frame setting. [Figure 7] 13A and 13B are diagrams illustrating an example of detection of edges of a mask region. [Figure 8] 13A and 13B are diagrams illustrating an example of detection of edges of a mask region. [Figure 9] 13A and 13B are diagrams illustrating an example of setting a detection frame when a mask region is detected based on an AF evaluation value. [Figure 10] 13A and 13B are diagrams illustrating another example of setting the detection frame when detecting a mask area based on an AF evaluation value. [Figure 11] 13 is a diagram showing yet another example of setting the detection frame when detecting a mask region based on an AF evaluation value. FIG. [Figure 12] FIG. 13 is a diagram showing an example of setting an evaluation value calculation target region. [Figure 13] FIG. 13 is a diagram showing another example of setting the evaluation value calculation target region. [Figure 14] 11 is a flowchart illustrating an AF process of a CCU. [Figure 15] 15 is a flowchart illustrating a mask area detection process performed in step S3 of FIG. 14. [Figure 16]15 is a flowchart illustrating another mask area detection process performed in step S3 of FIG. 14. [Figure 17] FIG. 2 is a block diagram showing an example of the configuration of a CCU that performs AE. [Figure 18] 11 is a diagram showing an example of setting an evaluation value calculation target region of an AE evaluation value. FIG. [Figure 19] 13A and 13B are diagrams illustrating another example of setting the evaluation value calculation target region of the AE evaluation value. [Figure 20] FIG. 2 is a block diagram showing an example of the configuration of a computer. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0014] Hereinafter, an embodiment of the present technology will be described in the following order. 1. Endoscope System Configuration 2. First embodiment: Example of detecting a mask area based on detection values 3. Second embodiment: Example of detecting a mask area based on an AF evaluation value 4. Third embodiment: Example 1 of setting evaluation value calculation target area 5. Fourth embodiment: Example 2 of setting evaluation value calculation target area 6. Fifth embodiment: Example of automatic focusing process 7. CCU Operation 8. Other Examples

[0015] <<Endoscope system configuration>> FIG. 1 is a diagram showing an example of the configuration of an endoscope system to which the present technology is applied.

[0016] The endoscope system 1 in FIG. 1 includes a display device 11, a CCU (camera control unit) 12, a light source device 13, a treatment tool device 14, an insufflation device 15, a recorder 16, and a cart 18 on which a printer 17 is mounted.

[0017] The endoscope system 1 also includes an endoscope 19, an energy treatment tool 20, forceps 21, trocars 22 to 25, a foot switch 26, and a patient bed 27. The endoscope system 1 is installed, for example, in an operating room, and supports a surgeon performing laparoscopic surgery on an affected area in the abdomen 31 of a patient lying on the patient bed 27.

[0018] The display device 11 is configured with a stationary 2D display, a head-mounted display, etc. The display device 11 displays an image of the operative part (operative field area) supplied from the CCU 12, etc.

[0019] The CCU 12 is connected to each device including the light source device 13 and the endoscope 19. The CCU 12 receives an image of the operation site captured by the endoscope 19 and transmitted via a camera cable, and displays the image on the display device 11. The CCU 12 outputs the image captured by the endoscope 19 to the recorder 16 or the printer 17 as necessary. The CCU 12 and the endoscope 19 may be connected by wireless communication.

[0020] Furthermore, the CCU 12 performs an automatic focusing process, which is a process for performing AF of the endoscope 19. That is, in the endoscope system 1, the focus adjustment of the endoscope 19 is performed automatically under the control of the CCU 12, without being dependent on the operation of the surgeon.

[0021] The light source device 13 is connected to the endoscope 19 via a light guide cable. The light source device 13 outputs light of various wavelengths to the endoscope 19 by switching between them.

[0022] The treatment tool device 14, which is a high-frequency output device, is connected to the energy treatment tool 20 and the foot switch 26 via a cable. The treatment tool device 14 outputs a high-frequency current to the energy treatment tool 20 in response to an operation signal supplied from the foot switch 26.

[0023] The insufflation device 15 includes an air supply means and an air intake means. The insufflation device 15 supplies air into the inside of the abdomen 31 through a hole in a trocar 24, which is a hole opening instrument attached to the abdominal wall of the abdomen 31.

[0024] The recorder 16 records the captured images supplied from the CCU 12 .

[0025] The printer 17 prints the captured images supplied from the CCU.

[0026] The endoscope 19 is inserted into the abdomen 31 through a hole in a trocar 22 attached to the abdominal wall of the abdomen 31. The endoscope 19 irradiates the inside of the abdomen 31 with light emitted from the light source device 13, and captures an image of the inside of the abdomen 31. The endoscope 19 outputs a captured image obtained by capturing an image of the inside of the abdomen 31 to the CCU 12.

[0027] The energy treatment tool 20 is composed of an electric scalpel or the like. The energy treatment tool 20 is inserted into the inside of the abdomen 31 through a hole in a trocar 23 attached to the abdominal wall of the abdomen 31. The energy treatment tool 20 modifies or cuts the inside of the abdomen 31 using electric heat.

[0028] The forceps 21 are inserted into the abdomen 31 through a hole in a trocar 25 attached to the abdominal wall of the abdomen 31. The forceps 21 grasps the inside of the abdomen 31. The endoscope 19, the energy treatment tool 20, and the forceps 21 are grasped by an operator, an assistant, a scopist, a robot, or the like.

[0029] The foot switch 26 receives an operation performed by the foot of an operator, an assistant, etc. The foot switch 26 outputs an operation signal indicating the content of the received operation to the CCU 12 and the treatment tool device 14.

[0030] By using the endoscope system 1, a surgeon can excise an affected area in the abdomen 31 while viewing the captured image displayed on the display device 11.

[0031] FIG. 2 is a perspective view showing the appearance of the endoscope 19. As shown in FIG.

[0032] As shown in Fig. 2, endoscope 19, which is a rigid endoscope, is composed of a camera head 51 and a scope 52 with a long, thin lens barrel. There are flexible endoscopes, in which the portion inserted into the body is bendable, and rigid endoscopes, in which the portion is not bendable. Endoscope 19 is the latter type. The surgeon will hold camera head 51 and insert scope 52 into the patient's body to proceed with surgery.

[0033] An imaging element that performs photoelectric conversion of light from inside the body guided by the lens in the scope 52, a drive unit for driving the lens in the scope 52, and the like are provided inside the camera head 51. The camera head 51 guides light emitted from the light source device 13 through the scope 52 to irradiate the inside of the abdomen 31 and capture an image of the operative area. The camera head 51 outputs the captured image obtained by capturing the image to the CCU 12 via a camera cable.

[0034] The scope 52 is detachable from the camera head 51. There are multiple types of scopes 52 with different specifications, such as scopes with different diameters (scope diameters) and scopes with different lens F-numbers. The type of scope 52 to be used is appropriately selected depending on the content of the surgery, the condition of the operative area, and the like, and is attached to the camera head 51.

[0035] FIG. 3 is a diagram showing an example of a captured image.

[0036] As shown in FIG. 3, a circular effective area is formed substantially in the center of a horizontally long rectangular captured image, and a mask area, which is a dark area, is formed outside the effective area.

[0037] The masked area is an area formed by the optical shadow (vignetting) of the scope 52. Since the shape of the scope 52 is an elongated cylinder, a pitch black area is reflected outside the image circle. The state of the surgical site is displayed within the range of the effective area, which is an area without vignetting. Note that vignetting by the scope 52 refers to an optical shadow that occurs when the optical path is physically blocked by, for example, the side wall of the scope 52.

[0038] As described above, the scope 52 is switched depending on the content of the surgery, etc. The position, size, and shape of the effective area in the captured image change depending on the type of scope 52 attached to the camera head 51.

[0039] Fig. 4 is a diagram showing an example of a captured image, in which the state of the operation site within the effective area is omitted.

[0040] The captured image shown in A of Fig. 4 is captured when a scope 52 is attached whose scope diameter is shorter than the vertical length of the captured image. The captured image in A of Fig. 4 includes the entire circular effective area, similar to the captured image in Fig. 3.

[0041] On the other hand, the captured image shown in Fig. 4B is an image captured when a scope 52 is attached whose diameter is longer than the vertical length of the captured image. The effective area in Fig. 4B is a circle with the top and bottom cut off.

[0042] In this way, an effective area having a diameter corresponding to the diameter of the scope is formed in the captured image. It is possible to identify the diameter of the scope 52 attached to the camera head 51 from the diameter of the effective area included in the captured image.

[0043] As described above, in the endoscope system 1, the focus adjustment of the endoscope 19 is automatically performed by AF.

[0044] If the AF evaluation value, which is an evaluation value for performing AF, is calculated for the entire captured image, the AF evaluation values for each region, including the mask region, will be calculated, which is not preferable. When calculating the AF evaluation value including the mask region, for example, an operation such as focusing on the edge (boundary) of the mask region is performed.

[0045] Also, if the region for calculating the AF evaluation value is fixed and set within the effective region, it can only be set in a narrow range near the center of the captured image that becomes the effective region regardless of which scope 52 is attached. As shown in FIG. 4, the range of the effective region changes depending on the attached scope 52.

[0046] In the CCU 12, based on the captured image, the scope diameter of the scope 52 attached to the endoscope 19, the center position of the effective region, etc. are specified, and based on the specified information, the evaluation value calculation target region, which is the region for calculating the AF evaluation value, is set within the effective region. Also, the AF evaluation value is calculated for the evaluation value calculation target region, and automatic focusing processing is performed.

[0047] Since the AF evaluation value is calculated for the evaluation value calculation target region set within the effective region and automatic focusing processing is performed, the CCU 12 can surely focus on the surgical site shown in the effective region.

[0048] In this way, a series of processes of the CCU 12 that sets the evaluation value calculation target region according to the scope 52 attached to the endoscope 19 and performs automatic focusing processing will be described later.

[0049] <<Example of Detecting Mask Region Based on Detection Value in the First Embodiment>> <Configuration Example of CCU and Endoscope> FIG. 5 is a block diagram showing a configuration example of the CCU 12 and the endoscope 19.

[0050] As shown on the left side of FIG. 5, camera head 51 is made up of an image sensor 111, an image sensor driver 112, a lens driver 113, a zoom lens driving section 114, and a focus lens driving section 115.

[0051] The imaging element 111 is configured by, for example, a CMOS image sensor or a CCD image sensor. The imaging element 111 converts an optical image formed on an imaging surface into an electric signal by photoelectric conversion, and outputs the electric signal to the CCU 12 as an imaging signal.

[0052] The imaging element driver 112 is a driver for driving the imaging element 111. The imaging element driver 112 outputs a drive signal to cause the imaging element 111 to perform a predetermined operation such as an imaging operation or a reset operation. For example, the shutter speed of the imaging element 111 is controlled by the drive signal output by the imaging element driver 112.

[0053] The lens driver 113 is configured with a processor such as a CPU (Central Processing Unit), a DSP (Digital Signal Processor), etc. The lens driver 113 controls the operations of a zoom lens driving unit 114 and a focus lens driving unit 115 in accordance with a control signal supplied from the CCU 12.

[0054] The zoom lens driver 114 moves the zoom lens 101 in the scope 52 along the optical axis to adjust the imaging magnification.

[0055] The focus lens driver 115 moves the focus lens 102 in the scope 52 along the optical axis to adjust the focus.

[0056] As shown on the right side of FIG. 5, the CCU 12 includes a camera signal processing unit 131, a detection frame gate 132, a detection unit 133, a mask detection unit 134, an AF detection unit 135, and a lens controller 136.

[0057] The camera signal processor 131 performs various types of signal processing, such as white balance processing and gamma correction processing, on the imaging signal supplied from the imaging element 111 .

[0058] The camera signal processing unit 131 outputs the imaging signal obtained by performing the signal processing as a video signal to the display device 11. An image of the operative area is displayed on the display device 11 based on the video signal output from the camera signal processing unit 131. The imaging signal output from the camera signal processing unit 131 is also supplied to the detection frame gate 132.

[0059] The detection frame gate 132 sets a detection frame in a predetermined area on the captured image in accordance with the control of the lens controller 136. For example, before detection of a mask area, the detection frame is set in a predetermined area of ​​the captured image including the mask area. Also, after detection of the mask area, the detection frame is set in the valid area.

[0060] The detection frame gate 132 outputs the imaging signals of pixels within the detection frame out of the imaging signals supplied from the camera signal processing unit 131. The imaging signals output from the detection frame gate 132 are supplied to a detection unit 133 and an AF detection unit 135.

[0061] The detection unit 133 detects the imaging signal supplied from the detection frame gate 132, and outputs the detection value of each detection frame to the mask detection unit 134. For example, the detection unit 133 integrates the luminance values ​​of the pixels in each detection frame, and obtains the integration result as a detection value.

[0062] The mask detection unit 134 detects the edge of the mask area based on the detection value supplied from the detection unit 133. Although the mask detection unit 134 will be described as detecting the edge of the mask area, detecting the edge of the mask area also corresponds to detecting the edge of the effective area.

[0063] Based on the position of the edge of the detected mask area, the mask detection unit 134 identifies the diameter of the scope 52 attached to the camera head 51. Based on the position of the edge of the mask area, the mask detection unit 134 also identifies the center position of the effective area and the position (range) of the mask area.

[0064] The mask detection unit 134 outputs information regarding the scope diameter, the center position of the effective area, and the position of the mask area as mask detection results to the lens controller 136. As will be described later, the detection of the mask area by the mask detection unit 134 may be performed based on an AF evaluation value obtained by the AF detection unit 135.

[0065] The AF detection unit 135 calculates an AF evaluation value based on the imaging signal supplied from the detection frame gate 132. For example, the AF detection unit 135 calculates an AF evaluation value that indicates contrast by second-order differentiation using the luminance signals of all pixels in the AF detection frame. Generally, when the subject is in focus, the difference in luminance signals between adjacent pixels is larger than when the subject is not in focus, and the contrast is greater.

[0066] The AF evaluation value calculated by the AF detection unit 135 is supplied to a lens controller 136 and is also supplied to a mask detection unit 134 as appropriate.

[0067] The lens controller 136 outputs a control signal to the lens driver 113 of the camera head 51 to adjust the position of the zoom lens 101 .

[0068] Furthermore, lens controller 136 outputs a control signal to lens driver 113 of camera head 51 to adjust the position of focus lens 102. The process of adjusting the position of focus lens 102 by outputting the control signal becomes an automatic focusing process for performing AF of endoscope 19. The control signal at this time is a signal that includes at least the position of focus lens 102 of lens driver 113 or the amount of movement of the position of focus lens 102 as an AF control parameter.

[0069] The lens controller 136 adjusts the shutter speed and ISO sensitivity of the image sensor 111 by appropriately outputting a control signal to the image sensor driver 112 of the camera head 51. The process of adjusting the shutter speed and ISO sensitivity by outputting the control signal and controlling the exposure without the operation of the surgeon is the automatic exposure process for performing AE. As will be described later, the AE function is realized based on the detection result of the mask area. The control signal at this time is a signal that includes at least the speed value of the shutter speed or the sensitivity value of the ISO sensitivity as an AE control parameter.

[0070] A focus instruction signal, a zoom instruction signal, a manual / autofocus switching signal, and the like are input to the lens controller 136 in response to an operation by the surgeon. The zoom instruction signal is a signal that indicates the content of zoom adjustment by the surgeon, and the focus instruction signal is a signal that indicates the content of focus adjustment by the surgeon.

[0071] The manual / autofocus switching signal is a signal for switching between manual mode and autofocus mode for focus adjustment. The focus instruction signal is input when manual mode for focus adjustment is selected.

[0072] <Mask area detection> FIG. 6 is a diagram showing an example of detection frame setting.

[0073] The detection frame is set, for example, over the entire area of ​​the captured image. In the example of Fig. 6, 30 detection frames are set in the column direction (vertical direction) and 40 detection frames are set in the row direction (horizontal direction) in a tiled pattern over the entire area of ​​the captured image. The size of the entire detection frame in Fig. 6 is the same as the size of the captured image.

[0074] In the mask detection section 134, the edge of the mask area is detected based on the detection value of 1200 obtained based on the image pickup signals of the pixels within each detection frame.

[0075] In this way, a detection frame having a resolution sufficient to detect the edge of the mask area is set for the captured image.

[0076] In the example of FIG. 6, the size of the entire detection frame is the same as the size of the captured image, but the detection frame may be set in an area smaller than the size of the captured image. Furthermore, the number of detection frames is also arbitrary.

[0077] FIG. 7 is a diagram showing an example of detection of the edge of a mask region.

[0078] The sequence of "x" characters shown inside each detection frame represents a specific detection value. A detection frame containing the number "0" indicates that the detection value is 0, that is, the detection frame contains black pixels.

[0079] When such detection values ​​are obtained in detection section 133, mask detection section 134 in FIG. 5 obtains the maximum detection value among the detection values ​​in the detection frames of each column and the maximum detection value among the detection values ​​in the detection frames of each row.

[0080] Furthermore, as indicated by the tip of the white arrow #1, the mask detection unit 134 obtains the difference between the detection values ​​in every other column in the array of the maximum detection values ​​in each column. Similarly, in the row direction, the mask detection unit 134 obtains the difference between the detection values ​​in every other row in the array of the maximum detection values ​​in each row.

[0081] For example, the mask detection unit 134 focuses on the differences in the detection values ​​of every other column, starting from the difference on the left side of the image, and detects the position of the column where the difference is equal to or greater than the threshold as the position of the left edge of the mask area. The mask detection unit 134 also focuses on the differences in the detection values ​​of every other column, starting from the difference on the right side of the image, and detects the position of the column where the difference is equal to or greater than the threshold as the position of the right edge of the mask area.

[0082] Similarly, in the row direction, the mask detection unit 134 focuses on the differences in the detection values ​​of every other row, starting from the difference at the top of the image, and detects the position of the row where a difference equal to or greater than the threshold is found as the position of the upper edge of the mask area.Furthermore, the mask detection unit 134 focuses on the differences in the detection values ​​of every other row, starting from the difference at the bottom of the image, and detects the position of the row where a difference equal to or greater than the threshold is found as the position of the lower edge of the mask area.

[0083] The mask detection unit 134 detects all edges surrounding the effective area, as shown in FIG. 8, based on the positions of the top, bottom, left and right edges and the fact that the effective area has a circular shape.

[0084] 8, the detection frame indicated by hatching is the detection frame at the position detected as the edge of the mask area. The outside of the detection frame indicated by hatching is the mask area, and the inside (including the position of the detection frame indicated by hatching) is the valid area.

[0085] If the effective area has a large diameter and is cut off at the top and bottom, the left and right edges are detected based on the difference between the detection values ​​of every other row, and the mask area is specified. If no edges are found, the area is specified as not having a mask.

[0086] When the mask area and the effective area are detected, the mask detection unit 134 determines, for example, the average edge width based on the edge width of the effective area as the diameter of the effective area, that is, the scope diameter. The mask detection unit 134 also determines the position indicated by the average median value of the edge width as the center position of the effective area. Note that the range of the average edge width and the corresponding scope diameter may be stored in advance, and the scope diameter may be determined by referring to a table based on the determined average edge width.

[0087] The information indicating the thus determined scope diameter and the center position of the effective area, together with information regarding the position of the mask area, is supplied to the lens controller 136. In the lens controller 136, the setting of a detection frame used for calculating an AF evaluation value and the like are performed based on the detection result of the mask area.

[0088] <<Second embodiment: Example of detecting a mask area based on an AF evaluation value>> It is also possible to detect the mask region based on the AF evaluation value obtained by the AF detection section 135, rather than based on the detection value obtained by the detection section 133.

[0089] First setting example FIG. 9 is a diagram showing an example of setting a detection frame when detecting a mask region based on an AF evaluation value.

[0090] For example, the detection frame gate 132 sets thin strip-shaped detection frames at the top, bottom, left, and right edges of the captured image, as shown as detection frames F1 to F4 on the captured image on the left side of Fig. 9. The detection frames F1 to F4 are set according to the control of the lens controller 136, for example.

[0091] Detection frames F1 and F2 indicated by dashed lines are detection frames for detecting the upper and lower edges of the mask area, respectively. Detection frames F3 and F4 indicated by dashed lines are detection frames for detecting the left and right edges of the mask area, respectively.

[0092] Moreover, the detection frame gate 132 moves the positions of the detection frames F1 to F4 toward the center of the captured image, as indicated by the white arrows.

[0093] In this way, the AF evaluation value is calculated in sequence in the AF detection section 135 based on the imaging signals corresponding to the pixels within the detection frame that is set by changing the position.

[0094] When a change equal to or greater than a threshold occurs in the AF evaluation value calculated by the AF detection unit 135, the mask detection unit 134 specifies the position where the change is detected as the edge position of the mask area. For example, as indicated by the tip of arrow #2, when detection frames F1 to F4 are set at the edge positions of the mask area, a change in the AF evaluation value equal to or greater than the threshold is detected.

[0095] Based on the distance L1 between detection frame F1 and detection frame F2 and the distance L2 between detection frame F3 and detection frame F4, the mask detection unit 134 identifies the diameter of the scope attached to the camera head 51, the center position of the effective area, and the position of the mask area. For example, the average of the distances L1 and L2 is identified as the scope diameter, and the position indicated by the average of the median values ​​of the edge widths is identified as the center position of the effective area.

[0096] Second setting example FIG. 10 is a diagram showing another example of setting the detection frame when detecting a mask region based on the AF evaluation value.

[0097] The detection frame setting method shown in Fig. 10 differs from the setting method in Fig. 9 in that the detection frames F1 to F4 are set so as to gradually increase in width rather than changing the positions of the detection frames F1 to F4. The widths of the detection frames F1 to F4 are gradually increased by moving the inner edges of the narrow band-shaped detection frames F1 to F4 set on the top, bottom, left, and right edges of the captured image toward the center of the captured image as indicated by the white arrows.

[0098] The AF evaluation value is calculated in sequence in the AF detection section 135 based on the imaging signals corresponding to the pixels within the detection frame set with varying widths.

[0099] When a change equal to or greater than a threshold occurs in the AF evaluation value calculated by the AF detection unit 135, the mask detection unit 134 specifies the position of the inner edge of the detection frame when the change is detected as the position of the edge of the mask area. For example, as indicated by the tip of arrow #3, when the inner edges of detection frames F1 to F4 are set to the positions of the edges of the mask areas, a change in the AF evaluation value equal to or greater than the threshold is detected.

[0100] The mask detection unit 134 determines the diameter of the scope attached to the camera head 51, the center position of the effective area, and the position of the mask area based on the distance L1 between detection frames F1 and F2, and the distance L2 between detection frames F3 and F4.

[0101] Third setting example FIG. 11 is a diagram showing yet another setting example of the detection frame when detecting a mask region based on the AF evaluation value.

[0102] The detection frame setting method shown in FIG. 11 differs from the setting method in FIG. 9 etc. in that a plurality of substantially square small detection frames are set in the initial state instead of a narrow band detection frame.

[0103] That is, the detection frame gate 132 sets substantially square detection frames F1-1 to F1-3 on the upper edge of the captured image, and sets substantially square detection frames F2-1 to F2-3 on the lower edge, as shown on the left side of Fig. 11. The detection frame gate 132 sets substantially square detection frames F3-1 to F3-3 on the left edge of the captured image, and sets substantially square detection frames F4-1 to F4-3 on the right edge.

[0104] Furthermore, the detection frame gate 132 moves the position of each detection frame toward the center of the captured image, or widens the width of the detection frame toward the center of the captured image.

[0105] The AF evaluation value is calculated in sequence in the AF detection section 135 based on the imaging signals corresponding to the pixels within the detection frame that is set with the position and width changed.

[0106] When a change equal to or greater than a threshold occurs in the AF evaluation value calculated by the AF detection section 135, the mask detection section 134 specifies the position where the change is detected as the position of the edge of the mask area.

[0107] For example, as indicated by the tip of arrow #4, when approximately square detection frames F1-1 to F1-3 and detection frames F2-1 to F2-3 are each set at the edge positions of the mask area, a change in the AF evaluation value equal to or greater than the threshold is detected.

[0108] Furthermore, when the inner edges of the detection frames F3-1 to F3-3 and the detection frames F4-1 to F4-3 reach the edge positions of the mask regions, respectively, a change in the AF evaluation value equal to or greater than the threshold is detected.

[0109] Mask detection unit 134 identifies the diameter of the scope attached to camera head 51, the center position of the effective area, and the position of the mask area, based on the distance between the opposing detection frames.

[0110] 11, the detection frames F1-1 to F1-3 and F2-1 to F2-3 set at the top and bottom edges of the captured image are set at different positions without changing the shape, but they may be set at different widths. Also, the detection frames F3-1 to F3-3 and F4-1 to F4-3 set at the left and right edges of the captured image are set at different widths, but they may be set at different positions without changing the shape.

[0111] The method of changing the shape of the detection frame and the method of changing the width may be used in combination. By changing the position and width of a small detection frame, it becomes possible to detect the detailed shape of the edge of the mask area.

[0112] If the effective area has a large diameter and is cut off at the top and bottom, the mask area is detected based only on the AF evaluation value calculated based on the detection frames set on the left and right. If no edge is found, it is determined that there is no mask area.

[0113] Information indicating the scope diameter and the center position of the effective area specified based on the AF evaluation value, together with information regarding the position of the mask area, is supplied to the lens controller 136. In the lens controller 136, setting of a detection frame used for calculating the AF evaluation value and the like are performed based on the detection result of the mask area.

[0114] <<Third embodiment: Example 1 of setting evaluation value calculation target area>> FIG. 12 is a diagram showing an example of setting an evaluation value calculation target region.

[0115] 12, the lens controller 136 sets the evaluation value calculation target area, which is the area for calculating the AF evaluation value, inside the effective area so as not to overlap with the mask area. For example, the evaluation value calculation target area is set by enlarging or reducing the size of the default area and shifting the default center position based on the detection result of the mask area.

[0116] A horizontally long rectangular area A1 shown in A of Fig. 12 is an evaluation value calculation target area set within an effective area having a diameter shorter than the vertical length of a captured image. A region A2 shown in B of Fig. 12 is an evaluation value calculation target area set within an effective area having a diameter longer than the vertical length of a captured image.

[0117] Each evaluation value calculation target region is set so that it fits entirely within the effective region. In A and B of Figures 12, two horizontally long rectangles representing the evaluation value calculation target regions are shown within the effective region, which indicates that the size of the evaluation value calculation target region can be set arbitrarily as long as it fits within the effective region.

[0118] In the example of FIG. 12, the shape of the evaluation value calculation target region is a horizontally long rectangle, but it may be another shape, such as a square or a circle.

[0119] Information regarding such an evaluation value calculation target area is supplied from the lens controller 136 to the detection frame gate 132, and a detection frame for calculating an AF evaluation value is set within the evaluation value calculation target area.

[0120] The AF detection unit 135 calculates an AF evaluation value based on the image pickup signals of pixels within a detection frame set in the evaluation value calculation target area. Furthermore, the lens controller 136 performs an automatic focusing process based on the calculated AF evaluation value.

[0121] This enables the CCU 12 to reliably focus on the surgical site that is captured in the effective area.

[0122] <<Fourth embodiment: Example 2 of setting evaluation value calculation target area>> FIG. 13 is a diagram showing another example of setting the evaluation value calculation target region.

[0123] Each square shown on the captured image in Fig. 13 represents a detection frame. In the example of Fig. 13, a plurality of detection frames are set in a tiled pattern around the center of the captured image, eight vertically and eight horizontally. In this example, the position of the detection frame is fixed regardless of the range of the effective area. The detection frame gate 132 supplies the AF detection unit 135 with an image signal of the pixels in each detection frame.

[0124] The lens controller 136 selects, from among the detection frames thus fixedly set, a detection frame that does not overlap the mask area and is entirely within the effective area as an evaluation value calculation target area for calculating an AF evaluation value. The detection frames shown in color in Fig. 13 are detection frames selected as the evaluation value calculation target area. On the other hand, a detection frame that overlaps the mask area is treated as an invalid (weight 0) detection frame.

[0125] The lens controller 136 outputs information about the detection frame selected as the evaluation value calculation target region to the AF detection unit 135, and causes the AF detection unit 135 to calculate an AF evaluation value for the detection frame selected as the evaluation value calculation target region.

[0126] That is, in the example of Figure 12, the detection frame for calculating the AF evaluation value is set based on the detection result of the mask area, whereas in the example of Figure 13, the detection frame for calculating the AF evaluation value is selected from pre-set detection frames.

[0127] In the example of A in Fig. 13, a part of the preset detection frames that falls within the effective area is set as the evaluation value calculation target area. In the example of B in Fig. 13, all of the preset detection frames fall within the effective area, so all of the detection frames are set as the evaluation value calculation target area.

[0128] In the example of FIG. 13, all the detection frames within the effective area are set as the evaluation value calculation target area, but a part of the detection frames within the effective area may be set as the evaluation value calculation target area.

[0129] Information regarding such an evaluation value calculation target area is supplied from the lens controller 136 to the AF detection unit 135, and an AF evaluation value is calculated based on the pixel signals of the pixels in the detection frame set as the evaluation value calculation target area. Also, based on the calculated AF evaluation value, the lens controller 136 performs autofocus processing.

[0130] Also by this, the CCU 12 can surely focus on the surgical part imaged in the effective area.

[0131] <<Example of Autofocus Processing in the Fifth Embodiment>> Based on the scope diameter, the center position of the effective area, and the position of the mask area specified as described above, the F value of the scope 52 may be estimated, and the depth of focus may be obtained based on the estimated F value.

[0132] The F value and the depth of focus of the scope 52 are used by the lens controller 136, for example, to set parameters that define the content of autofocus processing, such as the AF speed, the focusing accuracy, and the wobbling amplitude amount. The lens controller 136 calculates the AF speed, the focusing accuracy, and the wobbling amplitude amount respectively based on the F value and the depth of focus of the scope 52, and performs autofocus processing based on the calculation results.

[0133] Thereby, the lens controller 136 can perform autofocus processing with higher accuracy. Note that a table of the F value and the depth of focus corresponding to the scope diameter may be stored in advance, and the F value and the depth of focus may be obtained by referring to the table from the specified scope diameter.

[0134] <<Operation of CCU>> <AF Processing> The AF process of the CCU 12 will be described with reference to the flowchart of FIG.

[0135] The process of FIG. 14 is performed, for example, when the endoscope 19 captures an image of the operation site and an image capturing signal is supplied from the imaging element 111.

[0136] In step S1, the camera signal processing unit 131 performs various types of signal processing, such as white balance processing, on the imaging signal supplied from the imaging element 111.

[0137] In step S2, the detection frame gate 132 outputs image pickup signals of pixels within each detection frame.

[0138] For example, when detection of the mask area is performed as described with reference to Figures 7 and 8, the image signals of the pixels within each detection frame set over the entire captured image are output from the detection frame gate 132 and supplied to the detection unit 133.

[0139] Furthermore, when detection of the mask area is performed as described with reference to Figures 9, 10, and 11, the image signals of the pixels within each detection frame set at a predetermined position in the captured image are output from the detection frame gate 132 and supplied to the AF detection unit 135.

[0140] In step S3, a mask area detection process is performed. Information on the scope diameter, the center position of the effective area, and the position of the mask area, which are identified by the mask area detection process, is supplied from the mask detection unit 134 to the lens controller 136. Details of the mask area detection process will be described later with reference to the flowcharts of FIGS. 15 and 16.

[0141] In step S4, the lens controller 136 sets an evaluation value calculation target area based on the detection result of the mask area.

[0142] When the evaluation value calculation target area is set as described with reference to Figure 12, the lens controller 136 outputs information regarding the evaluation value calculation target area to the detection frame gate 132, causing a detection frame to be set within the evaluation value calculation target area.

[0143] Furthermore, when the evaluation value calculation target area is set as described with reference to FIG. 13, the lens controller 136 outputs information regarding the evaluation value calculation target area to the AF detection unit 135, and causes the AF evaluation value to be calculated for the detection frame selected as the evaluation value calculation target area.

[0144] In step S5, the AF detection section 135 calculates an AF evaluation value based on the imaging signal supplied from the detection frame gate 132, with the detection frame of the evaluation value calculation target region as a target.

[0145] In step S6, the lens controller 136 performs an automatic focusing process based on the AF evaluation value calculated by the AF detection unit 135. A control signal is output from the lens controller 136 to the lens driver 113 of the camera head 51, and the position of the focus lens 102 is adjusted, thereby achieving AF.

[0146] The above process is repeated while the imaging signal is being supplied from the imaging element 111.

[0147] <Mask area detection process> Next, the mask area detection process performed in step S3 in FIG. 14 will be described with reference to the flowchart in FIG.

[0148] 15 is a process in which detection of a mask area is performed using detection values ​​obtained by the detection unit 133. The detection frame gate 132 supplies the detection unit 133 with imaging signals of pixels within each detection frame set over the entire captured image as described with reference to FIGS.

[0149] In step S11, the detection unit 133 detects the imaging signal supplied from the detection frame gate 132, and outputs the detection value of each detection frame.

[0150] In step S12, the mask detection unit 134 detects the maximum detection value among the detection values ​​in the detection frame of each column, and also detects the maximum detection value among the detection values ​​in the detection frame of each row.

[0151] In step S13, the mask detection unit 134 finds the difference between the detection values ​​for every other column in the array of the maximum detection values ​​for each column. The mask detection unit 134 also finds the difference between the detection values ​​for every other row in the array of the maximum detection values ​​for each row.

[0152] In step S14, the mask detection unit 134 sequentially focuses on the differences in the detection values ​​of every other column, and detects the positions where the difference is equal to or greater than the threshold as the positions of the left and right edges of the mask area. The mask detection unit 134 also sequentially focuses on the differences in the detection values ​​of every other row, and detects the positions where the difference is equal to or greater than the threshold as the positions of the top and bottom edges of the mask area.

[0153] In step S15, the mask detection unit 134 identifies the scope diameter and the center position of the effective area based on the edge width of the effective area.

[0154] In step S16, the mask detection unit 134 outputs information on the scope diameter, the center position of the effective area, and the position of the mask area as the detection result of the mask area to the lens controller 136. After that, the process returns to step S3 in Fig. 14, and the subsequent processes are performed.

[0155] Next, another mask area detection process performed in step S3 of FIG. 14 will be described with reference to the flowchart of FIG.

[0156] In the process of FIG. 16, detection of a mask region is performed using an AF evaluation value obtained by the AF detection unit 135.

[0157] In step S21, the lens controller 136 outputs information regarding the position of the detection frame to the detection frame gate 132 to set the detection frame. For example, information for setting the detection frame at each position of the captured image is supplied from the lens controller 136 to the detection frame gate 132 as described with reference to Figures 9, 10, and 11. When the detection frame is set, an image signal corresponding to a pixel within the detection frame is supplied from the detection frame gate 132 to the AF detection unit 135.

[0158] In step S22, the AF detection unit 135 calculates an AF evaluation value based on the imaging signal supplied from the detection frame gate 132. The calculated AF evaluation value is supplied to the mask detection unit .

[0159] In step S23, the mask detection unit 134 determines whether or not a change equal to or greater than the threshold has occurred in the AF evaluation value. If it is determined in step S23 that a change equal to or greater than the threshold has not occurred, the process returns to step S21, and the detection frame is set with a different position or width, and the above processing is repeated.

[0160] On the other hand, if it is determined in step S23 that the AF evaluation value has changed by more than the threshold, in step S24, the mask detection unit 134 identifies the position where the AF evaluation value has changed by more than the threshold as the position of the edge of the mask area.

[0161] The process after the edge position of the mask area is identified is the same as the process after step S15 in Fig. 15. That is, in step S25, the mask detection unit 134 identifies the scope diameter and the center position of the effective area based on the edge width of the effective area.

[0162] In step S26, the mask detection unit 134 outputs information on the scope diameter, the center position of the effective area, and the position of the mask area as the detection result of the mask area to the lens controller 136. After that, the process returns to step S3 in Fig. 14, and the subsequent processes are performed.

[0163] Through the above processing, the CCU 12 can set the area to be calculated for the AF evaluation value at an appropriate position within the effective area and with an appropriate size according to the type of the scope 52 attached to the endoscope 19, which is a rigid endoscope.

[0164] In addition, the CCU 12 can set parameters such as the AF speed and the wobbling amplitude amount to appropriate values according to the specifications of the scope 52, thereby improving the AF performance. By setting the AF speed and the wobbling amplitude amount according to the specifications of the scope 52, it becomes possible to avoid AF fluctuations, avoid out-of-focus prevention, and avoid image quality degradation such as the wobbling operation becoming visible.

[0165] For example, when a scope with a large diameter is attached and AF is performed using an F value and a depth of focus larger than appropriate values, AF fluctuations occur due to the AF speed being too fast. In addition, problems such as the minute vibrations of the wobbling operation being visible on the output image occur.

[0166] Conversely, when a scope with a small diameter is attached and AF is performed using an F value and a depth of focus smaller than appropriate values, problems such as it taking a long time to focus due to the AF speed being too slow occur. In addition, problems such as so-called out-of-focus prevention occur, where the amplitude amount of the wobbling operation is insufficient and the AF stops while out of focus because the focusing direction cannot be determined.

[0167] By performing AF using appropriate values according to the specifications of the scope, it becomes possible to avoid these problems caused by the inappropriate specifications of the scope.

[0168] That is, the endoscope system 1 can provide the surgeon with an endoscope image suitable for the operation.

[0169] <<Other examples>> <Example applied to AE> It is also possible to perform automatic exposure processing for AE based on the detection result of the mask area. AE is performed by automatically adjusting parameters such as shutter speed and ISO sensitivity without the operator's operation.

[0170] FIG. 17 is a block diagram showing an example of the configuration of the CCU 12 that performs AE.

[0171] Among the components shown in Fig. 17, the same components as those described with reference to Fig. 5 are denoted by the same reference numerals. Duplicate descriptions will be omitted as appropriate. The configuration of the CCU 12 shown in Fig. 17 differs from the configuration shown in Fig. 5 in that an exposure detection unit 141 is provided.

[0172] The detection frame gate 132 outputs, to an exposure detection section 141, imaging signals of pixels within a detection frame for exposure evaluation, out of the imaging signals supplied from the camera signal processing section 131.

[0173] The exposure detection unit 141 calculates an AE evaluation value, which is an evaluation value for exposure evaluation, based on the imaging signals of the pixels within the detection frame, and outputs the AE evaluation value to the lens controller 136 .

[0174] The lens controller 136 outputs a control signal for adjusting the shutter speed and ISO sensitivity to the image sensor driver 112 based on the AE evaluation value calculated by the exposure detection unit 141, and performs automatic exposure processing.

[0175] FIG. 18 is a diagram showing an example of setting an evaluation value calculation target region for the AE evaluation value.

[0176] The setting of the evaluation value calculation target region shown in FIG. 18 corresponds to the setting of the evaluation value calculation target region described with reference to FIG.

[0177] 18, the lens controller 136 sets the evaluation value calculation target area, which is the target for calculating the AE evaluation value, inside the effective area so as not to overlap with the mask area. For example, the evaluation value calculation target area is set by enlarging or reducing the size of the default area and shifting the default center position based on the detection result of the mask area.

[0178] A horizontally long rectangular area A11 shown on the right side of Fig. 18 is an evaluation value calculation target area set within an effective area having a diameter shorter than the vertical length of the captured image. In the example of Fig. 18, for example, the evaluation value calculation target area is set by reducing the default size shown on the left side of Fig. 18.

[0179] Such information regarding the evaluation value calculation target region is supplied from the lens controller 136 to the detection frame gate 132, and a detection frame for calculating the AE evaluation value is set within the evaluation value calculation target region.

[0180] In the exposure detection unit 141, an AE evaluation value is calculated based on the imaging signals of pixels within a detection frame set in an evaluation value calculation target area. In addition, an automatic exposure process is performed by the lens controller 136 based on the calculated AE evaluation value.

[0181] This enables the CCU 12 to adjust the exposure of the surgical area captured in the effective area to an appropriate exposure.

[0182] For example, if the detection frame for exposure evaluation is set in a masked area, the AE will control the exposure to increase, which may result in overexposure. By using only the detection frame within the effective area to evaluate the exposure, the CCU12 can adjust the brightness of the surgical site to the appropriate exposure.

[0183] FIG. 19 is a diagram showing another setting example of the evaluation value calculation target region for the AE evaluation value.

[0184] The setting of the evaluation value calculation target region shown in FIG. 19 corresponds to the setting of the evaluation value calculation target region described with reference to FIG.

[0185] Each square shown on the captured image on the left side of Fig. 19 represents a detection frame. In the example of Fig. 19, a plurality of detection frames are set in a tiled pattern around the center of the captured image, 10 vertically and 10 horizontally. In this example, the position of the detection frame is fixed regardless of the range of the effective area. The detection frame gate 132 supplies the exposure detection unit 141 with an image signal of the pixels of each detection frame.

[0186] The lens controller 136 selects, from among the detection frames thus fixedly set, a detection frame that does not overlap the mask area and falls entirely within the effective area as an evaluation value calculation target area for calculating an AE evaluation value. The detection frames shown in color on the right side of Fig. 19 are detection frames selected as the evaluation value calculation target area. On the other hand, a detection frame that overlaps the mask area is treated as an invalid (weight 0) detection frame.

[0187] The lens controller 136 outputs information about the detection frame selected as the evaluation value calculation target region to the exposure detection unit 141, and causes the exposure detection unit 141 to calculate an AE evaluation value for the detection frame selected as the evaluation value calculation target region.

[0188] Such information on the evaluation value calculation target area is supplied from the lens controller 136 to the exposure detection unit 141, and the AE evaluation value is calculated based on the pixel signals of the pixels in the detection frame selected as the evaluation value calculation target area. Furthermore, the lens controller 136 performs automatic exposure processing based on the calculated AE evaluation value.

[0189] This also enables the CCU 12 to adjust the exposure of the surgical site captured in the effective area to an appropriate exposure.

[0190] The automatic exposure process may be performed in combination with the above-described automatic focusing process. That is, the control process performed by the lens controller 136 is a process including at least one of the automatic exposure process and the automatic focusing process.

[0191] <Example applied to AWB> Based on the detection result of the mask area, it is also possible to cause the camera signal processing unit 131 to perform white balance processing for performing AWB (auto white balance) of the endoscope system 1.

[0192] In this case, the detection result of the mask area is supplied from the mask detection unit 134 to the camera signal processing unit 131, and the white balance processing is performed based on the imaging signals of the pixels within the effective area.

[0193] The white balance processing is a process of correcting the color of the captured image so that the surgical site appears in an appropriate color tone. For example, it is a process of estimating the imaging environment based on the imaging signals (color signals) of the pixels within the effective area and correcting the color generated from the imaging signals.

[0194] Alternatively, a white balance process may be performed in which the color temperature of the light source is estimated from the imaging signals of the pixels within the effective area and color correction is performed in accordance with the color temperature of the light source stored in advance.

[0195] Alternatively, when it is determined that the redness within the effective area is weak, a white balance process may be performed to enhance the redness so that the blood vessels are emphasized.

[0196] In this way, by performing the white balance process using the imaging signals of the pixels within the effective area, it becomes possible to obtain a captured image in which the surgical site appears in an appropriate color tone.

[0197] <Other examples> Although the case where AF, AE, and AWB using the detection result of the mask area are performed in the endoscope system 1 has been described, it is also applicable to the case where AF, AE, and AWB are performed in a microscope system.

[0198] Although the detection of the mask area is performed based on the detection value obtained by the detection unit 133 or the AF evaluation value obtained by the AF detection unit 135, it may be performed using both the detection value and the AF evaluation value.

[0199] The above-mentioned series of processes can be executed by hardware or software. When the series of processes is executed by software, the program constituting the software is installed from a program recording medium into a computer incorporated in dedicated hardware, or into a general-purpose personal computer, etc.

[0200] FIG. 20 is a block diagram showing an example of the hardware configuration of a computer that executes the above-mentioned series of processes by a program.

[0201] A CPU (Central Processing Unit) 1001, a ROM (Read Only Memory) 1002, and a RAM (Random Access Memory) 1003 are interconnected by a bus 1004.

[0202] An input / output interface 1005 is further connected to the bus 1004. An input unit 1006 including a keyboard, a mouse, etc., and an output unit 1007 including a display, a speaker, etc. are connected to the input / output interface 1005. In addition, a storage unit 1008 including a hard disk or non-volatile memory, a communication unit 1009 including a network interface, etc., and a drive 1010 that drives a removable medium 1011 are connected to the input / output interface 1005.

[0203] In the computer configured as above, the CPU 1001 loads a program stored in the storage unit 1008 into the RAM 1003 via the input / output interface 1005 and the bus 1004 and executes the program, thereby performing the above-mentioned series of processes.

[0204] The programs executed by the CPU 1001 are installed in the storage unit 1008, for example, by being recorded on a removable medium 1011, or provided via a wired or wireless transmission medium such as a local area network, the Internet, or digital broadcasting.

[0205] In addition, the program executed by the computer may be a program in which processing is performed chronologically in the order described in this specification, or it may be a program in which processing is performed in parallel or at the required timing, such as when called.

[0206] In this specification, a system refers to a collection of multiple components (devices, modules (parts), etc.), regardless of whether all the components are in the same housing. Therefore, multiple devices housed in separate housings and connected via a network, and a single device in which multiple modules are housed in a single housing, are both systems.

[0207] It should be noted that the effects described in this specification are merely examples and are not limiting, and other effects may also be obtained.

[0208] The embodiments of the present technology are not limited to the above-described embodiments, and various modifications are possible without departing from the gist of the present technology.

[0209] For example, the present technology can be configured as cloud computing, in which a single function is shared and processed collaboratively by multiple devices via a network.

[0210] Furthermore, each step described in the above flow chart can be executed by one device, or can be shared and executed by a plurality of devices.

[0211] Furthermore, when a single step includes multiple processes, the multiple processes included in the single step can be executed by a single device, or can be shared and executed by multiple devices.

[0212] <Examples of configuration combinations> The present technology can also be configured as follows.

[0213] (1) A light source device that irradiates light onto a surgical field; an imaging device that images the surgical field through a detachable scope; connected to the imaging device and the light source device, a detection unit that detects an effective area of ​​the scope in an image captured by the imaging device; a control unit that executes a control process including at least one of an automatic focusing process, an automatic exposure process, and an automatic white balance process based on the evaluation value within the effective area; An information processing device comprising: 2. An endoscope system comprising: (2) The effective area is an area where there is no vignetting due to the scope. The endoscope system described in (1) above. (3) A first detection unit that detects a luminance signal of each area set in the captured image, The detection unit detects the effective area based on a detection value of each area of ​​the captured image. The endoscope system according to (1) or (2). (4) The detection unit detects an edge of the effective area based on the detection value of each area set in a tiled shape in the captured image, and identifies a diameter of the scope and a center position of the effective area. The endoscope system described in (3) above. (5) a second detection unit that detects within a detection frame set in the captured image and calculates an evaluation value for the automatic focusing process; the detection unit detects the effective area based on an evaluation value for the automatic focusing process; The control unit executes the automatic focusing process as the control process based on the evaluation value for the automatic focusing process. The endoscope system described in (1) above. (6) The second detection unit changes at least one of a position and a size of the detection frame to calculate an evaluation value for the automatic focusing process used for detecting the effective area. The endoscope system described in (5) above. (7) The detection unit detects an edge of the effective area based on an evaluation value for the automatic focusing process, and identifies a diameter of the scope and a center position of the effective area. The endoscope system described in (6) above. (8) The control unit sets an evaluation value calculation target area, which is a target for calculating the evaluation value, within the effective area represented by the diameter of the scope and the center position of the effective area. An endoscope system according to any one of (1) to (7). (9) The control unit executes the control process based on the evaluation value calculated in an evaluation value calculation target area within an effective area represented by a diameter of the scope and a center position of the effective area, the evaluation value calculation target area being set in a tiled manner in the captured image and being the target of calculation of the evaluation value. An endoscope system according to any one of (1) to (8). (10) The control unit sets a parameter that defines the automatic focusing process based on at least one of an F-number and a focal depth of the scope that are estimated based on a detection result by the detection unit. An endoscope system according to any one of (1) to (9). (11) The control unit refers to a table including at least one of the F-number or the focal depth of the scope stored in advance based on the detection result by the detection unit, and sets parameters that define the automatic focusing process based on the reference result. An endoscope system according to any one of (1) to (9). (12) The light source device irradiates light onto the surgical field area, An imaging device images the surgical field through a detachable scope; an information processing device connected to the imaging device and the light source device, Detecting an effective area of ​​the scope from an image captured by the imaging device; Based on the evaluation value within the effective area, a control process including at least one of an automatic focusing process, an automatic exposure process, and an automatic white balance process is executed. A method for controlling an endoscope system. (13) A light source device that irradiates light onto a surgical field and an imaging device that images the surgical field via a detachable scope are connected to the light source device, a detection unit that detects an effective area of ​​the scope in an image captured by the imaging device; a control unit that executes a control process including at least one of an automatic focusing process, an automatic exposure process, and an automatic white balance process based on the evaluation value within the effective area; An information processing device for an endoscope system comprising: (14) A computer, which is an information processing device of an endoscope system, is connected to a light source device that irradiates a surgical field with light and an imaging device that images the surgical field via a detachable scope. Detecting an effective area of ​​the scope from an image captured by the imaging device; Based on the evaluation value within the effective area, a control process including at least one of an automatic focusing process, an automatic exposure process, and an automatic white balance process is executed. A program for executing a process. [Explanation of symbols]

[0214] 1 Endoscope system, 12 CCU, 19 Endoscope, 51 Camera head, 52 Scope, 111 Image sensor, 112 Image sensor driver, 113 Lens driver, 114 Zoom lens drive unit, 115 Focus lens drive unit, 131 Camera signal processing unit, 132 Detection frame gate, 133 Detection unit, 134 Mask detection unit, 135 AF detection unit, 136 Lens controller, 141 Exposure detection unit

Claims

1. a detection unit that detects within a detection frame set in an image captured by the endoscope and calculates a first evaluation value; a detection unit that detects an effective area of ​​the captured image, which is an area free of vignetting caused by a scope of the endoscope, based on the first evaluation value; a control unit that executes at least one of an automatic focusing process, an automatic exposure process, and an automatic white balance process based on a second evaluation value that targets an evaluation value calculation target area set within the effective area; An information processing device comprising:

2. The control unit executes the automatic exposure process by adjusting a shutter speed or an imaging sensitivity based on the second evaluation value for exposure evaluation targeting the evaluation value calculation target area. The information processing device according to claim 1 .

3. The control unit estimates an imaging environment based on an imaging signal of a pixel in the effective area, and performs the automatic white balance processing by correcting a color generated from the imaging signal. The information processing device according to claim 1 .

4. The detection unit performs detection within the detection frame that is set by changing a position, and calculates the first evaluation value used for detecting the effective area. The information processing device according to claim 1 .

5. The detection unit performs detection within the detection frame that is set by changing the size as well as the position, and calculates the first evaluation value used for detecting the effective area. The information processing device according to claim 4.

6. The detection unit detects an edge of the effective area based on the first evaluation value, and identifies a diameter of the scope and a center position of the effective area. The information processing device according to claim 1 .

7. The control unit sets the evaluation value calculation target region within the effective region represented by the diameter of the scope and the center position of the effective region. The information processing device according to claim 6.

8. The control unit executes at least one of the automatic focusing process, the automatic exposure process, and the automatic white balance process based on the second evaluation value calculated in the evaluation value calculation target area within the effective area represented by the diameter of the scope and the center position of the effective area, out of the evaluation value calculation target areas set in a tiled pattern on the captured image. The information processing device according to claim 6.

9. The control unit sets a parameter that defines at least one of the automatic focusing process, the automatic exposure process, and the automatic white balance process, based on at least one of an F-number and a focal depth of the scope estimated based on a detection result by the detection unit.

9. An information processing device according to claim 1, 6, 7 or 8.

10. The control unit refers to a table including at least one of an F-number or a focal depth of the scope stored in advance based on a detection result by the detection unit, and sets parameters defining at least one of the automatic focusing process, the automatic exposure process, and the automatic white balance process based on the reference result.

9. An information processing device according to claim 1, 6, 7 or 8.

11. A method for operating an information processing device, comprising: a detection unit that detects within a detection frame set in an image captured by the endoscope to calculate a first evaluation value; a detection unit detects an effective area of ​​the captured image, which is an area free of vignetting caused by a scope of the endoscope, based on the first evaluation value; The control unit executes at least one of an automatic focusing process, an automatic exposure process, and an automatic white balance process based on a second evaluation value that is set within the effective area and that is targeted for an evaluation value calculation target area. How it works.

12. On the computer, calculating a first evaluation value by detecting within a detection frame set in an image captured by the endoscope; detecting an effective area of ​​the captured image, which is an area free of vignetting caused by a scope of the endoscope, based on the first evaluation value; At least one of an automatic focusing process, an automatic exposure process, and an automatic white balance process is executed based on a second evaluation value that is set within the effective area and is targeted at an evaluation value calculation target area. A program that executes a process.

13. A light source device that irradiates light onto a surgical field; an endoscope for imaging the surgical field; An information processing device connected to the endoscope and the light source device, a detection unit that detects within a detection frame set in an image captured by the endoscope and calculates a first evaluation value; a detection unit that detects an effective area of ​​the captured image, which is an area free of vignetting caused by a scope of the endoscope, based on the first evaluation value; a control unit that executes at least one of an automatic focusing process, an automatic exposure process, and an automatic white balance process based on a second evaluation value that targets an evaluation value calculation target area set within the effective area; Equipped The information processing device; 2. An endoscope system comprising:

Citation Information

Patent Citations

  • Endoscope apparatus

    JP1993130973A

  • Character recognizing method and device

    JP2002230482A

  • Autofocus system

    JP2005250127A

  • Endoscope dimming system

    JP2010207493A

  • Imager and white balance correction method

    JP2011044788A