Information processing device, operation method, program, and endoscope system

Abstract

To capture an endoscopic image suitable for surgery.SOLUTION: An information processing device calculates a first evaluation value by performing detection of a detection frame set for a captured endoscopic image, detects an effective region of the captured image, which is a region free of vignetting by a scope of the endoscope, on the basis of the first evaluation value, and performs at least one of automatic focusing processing, automatic exposure processing, and automatic white balance processing on the basis of a second evaluation value with an evaluation value calculation target region set in the effective region as a target. This technology can be applied to an endoscope system.SELECTED DRAWING: Figure 12

Description

[Technical Field]

[0001] The present technology relates to an information processing device, an operating method, a program, and an endoscope system, and more particularly to an information processing device, an operating method, a program, and an endoscope system that are capable of capturing endoscopic images suitable for surgery. [Background technology]

[0002] In recent years, minimally invasive endoscopic surgery has been attracting attention. To obtain endoscopic images suitable for surgery, it has been proposed to equip endoscopes with AF (Auto Focus), AE (Automatic Exposure), and AWB (Auto White Balance) functions.

[0003] For example, Patent Document 1 discloses a technique for calculating a focus evaluation value based on an endoscopic image and performing AF for the endoscope. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Japanese Patent Application Laid-Open No. 2013-80108 Summary of the Invention [Problem to be solved by the invention]

[0005] Normally, extremely dark masked areas are created near the edges of an endoscopic image due to optical shadows (vignetting) from the scope. If AF is performed based on such an endoscopic image, the focus may be adjusted based on the masked area.

[0006] Furthermore, when the endoscope is a rigid endoscope, the position and size of the mask area change depending on the type of scope attached, so the area to be used for calculating the evaluation value cannot be fixed in advance.

[0007] The present technology has been made in view of such circumstances, and makes it possible to capture endoscopic images suitable for surgery. [Means for solving the problem]

[0008] An information processing device according to one aspect of the present technology includes a detection unit that performs detection within a detection frame set in an image captured by an endoscope and calculates a first evaluation value, a detection unit that detects an effective area of ​​the captured image, which is an area free of vignetting caused by a scope possessed by the endoscope, based on the first evaluation value, and a control unit that executes at least one of automatic focusing processing, automatic exposure processing, and automatic white balance processing based on a second evaluation value targeted at an evaluation value calculation target area set within the effective area.

[0009] According to another aspect of the present technology, there is provided an endoscopic system including a light source device that irradiates a surgical field with light, an endoscope that captures an image of the surgical field, and an information processing device connected to the image capture device and the light source device. The information processing device includes: a detection unit that performs detection within a detection frame set in an image captured by the endoscope and calculates a first evaluation value; a detection unit that detects an effective area of ​​the captured image, which is an area free from vignetting caused by a scope of the endoscope, based on the first evaluation value; and a control unit that performs at least one of automatic focusing processing, automatic exposure processing, and automatic white balance processing based on a second evaluation value targeted at an evaluation value calculation target area set within the effective area.

[0010] In this technology, a first evaluation value is calculated by detecting within a detection frame set on an image captured by an endoscope, an effective area of ​​the captured image, which is an area free from vignetting caused by the scope of the endoscope, is detected based on the first evaluation value, and at least one of automatic focusing processing, automatic exposure processing, and automatic white balance processing is executed based on a second evaluation value targeted at an evaluation value calculation target area set within the effective area. [Effects of the Invention]

[0011] According to the present technology, it is possible to capture endoscopic images suitable for surgery.

[0012] The effects described here are not necessarily limited to those described herein, and may be any of the effects described in this disclosure. [Brief explanation of the drawings]

[0013] [Figure 1] FIG. 1 is a diagram illustrating an example of the configuration of an endoscope system to which the present technology is applied. [Figure 2] FIG. 1 is a perspective view showing the appearance of an endoscope. [Figure 3] FIG. 10 is a diagram illustrating an example of a captured image. [Figure 4] FIG. 10 is a diagram illustrating an example of a captured image. [Figure 5] FIG. 2 is a block diagram showing an example of the configuration of a CCU and an endoscope. [Figure 6] FIG. 10 is a diagram illustrating an example of setting a detection frame. [Figure 7] FIG. 10 is a diagram illustrating an example of detecting an edge of a mask area. [Figure 8] FIG. 10 is a diagram illustrating an example of detecting an edge of a mask area. [Figure 9] 10A and 10B are diagrams illustrating an example of setting a detection frame when a mask area is detected based on an AF evaluation value. [Figure 10] 10A and 10B are diagrams illustrating other examples of setting the detection frame when detecting a mask area based on an AF evaluation value. [Figure 11] 10 is a diagram showing yet another example of setting the detection frame when detecting a mask area based on an AF evaluation value. FIG. [Figure 12] FIG. 10 is a diagram illustrating an example of setting an evaluation value calculation target region. [Figure 13] FIG. 10 is a diagram illustrating another example of setting the evaluation value calculation target region. [Figure 14] 10 is a flowchart illustrating an AF process of a CCU. [Figure 15] 15 is a flowchart illustrating a mask area detection process performed in step S3 of FIG. 14. [Figure 16]15 is a flowchart illustrating another mask area detection process performed in step S3 of FIG. 14. [Figure 17] FIG. 10 is a block diagram showing an example of the configuration of a CCU that performs AE. [Figure 18] 10A and 10B are diagrams illustrating an example of setting an evaluation value calculation target region for an AE evaluation value. [Figure 19] 10A and 10B are diagrams illustrating other setting examples of the evaluation value calculation target region for the AE evaluation value. [Figure 20] FIG. 1 is a block diagram illustrating an example of the configuration of a computer. DETAILED DESCRIPTION OF THE INVENTION

[0014] Hereinafter, embodiments of the present technology will be described in the following order. 1. Endoscope system configuration 2. First embodiment: Example of detecting a mask area based on detected values 3. Second embodiment: Example of detecting a mask area based on an AF evaluation value 4. Third embodiment: Example 1 of setting evaluation value calculation target area 5. Fourth embodiment: Example 2 of setting evaluation value calculation target area 6. Fifth embodiment: Example of automatic focusing process 7. CCU Operation 8. Other Examples

[0015] <<Configuration of the endoscope system>> FIG. 1 is a diagram showing an example of the configuration of an endoscope system to which the present technology is applied.

[0016] The endoscope system 1 in FIG. 1 includes a display device 11, a CCU (camera control unit) 12, a light source device 13, a treatment tool device 14, an insufflation device 15, a recorder 16, and a cart 18 on which a printer 17 is mounted.

[0017] The endoscope system 1 also includes an endoscope 19, an energy treatment tool 20, forceps 21, trocars 22 to 25, a foot switch 26, and a patient bed 27. The endoscope system 1 is installed, for example, in an operating room, and supports a surgeon performing laparoscopic surgery on an affected area in the abdomen 31 of a patient lying on the patient bed 27.

[0018] The display device 11 is configured by a stationary 2D display, a head-mounted display, etc. The display device 11 displays an image of the operative part (operative field area) supplied from the CCU 12, etc.

[0019] The CCU 12 is connected to various devices including the light source device 13 and the endoscope 19. The CCU 12 receives an image of the surgical site captured by the endoscope 19 and transmitted via a camera cable, and displays the image on the display device 11. The CCU 12 outputs the image captured by the endoscope 19 to the recorder 16 or the printer 17 as necessary. The CCU 12 and the endoscope 19 may be connected via wireless communication.

[0020] Furthermore, the CCU 12 performs an automatic focusing process, which is a process for performing AF of the endoscope 19. That is, in the endoscope system 1, the focus of the endoscope 19 is adjusted automatically under the control of the CCU 12, without being operated by the operator.

[0021] The light source device 13 is connected to the endoscope 19 via a light guide cable. The light source device 13 outputs light of various wavelengths to the endoscope 19 by switching between them.

[0022] The treatment tool device 14, which is a high-frequency output device, is connected via a cable to the energy treatment tool 20 and the foot switch 26. The treatment tool device 14 outputs a high-frequency current to the energy treatment tool 20 in response to an operation signal supplied from the foot switch 26.

[0023] The insufflation device 15 includes an air supply means and an air intake means. The insufflation device 15 supplies air into the inside of the abdomen 31 through a hole in a trocar 24, which is a hole-opening instrument attached to the abdominal wall of the abdomen 31.

[0024] The recorder 16 records the captured images supplied from the CCU 12 .

[0025] The printer 17 prints the captured images supplied from the CCU.

[0026] The endoscope 19 is inserted into the inside of the abdomen 31 through a hole in a trocar 22 attached to the abdominal wall of the abdomen 31. The endoscope 19 irradiates the inside of the abdomen 31 with light emitted from the light source device 13 and captures an image of the inside of the abdomen 31. The endoscope 19 outputs the captured image obtained by capturing the image of the inside of the abdomen 31 to the CCU 12.

[0027] The energy treatment device 20 is composed of an electric scalpel or the like. The energy treatment device 20 is inserted into the inside of the abdomen 31 through a hole in a trocar 23 attached to the abdominal wall of the abdomen 31. The energy treatment device 20 denatures or cuts the inside of the abdomen 31 using electric heat.

[0028] The forceps 21 are inserted into the inside of the abdomen 31 through a hole in a trocar 25 attached to the abdominal wall of the abdomen 31. The forceps 21 grasp the inside of the abdomen 31. The endoscope 19, the energy treatment tool 20, and the forceps 21 are grasped by the surgeon, an assistant, a scopist, a robot, or the like.

[0029] The foot switch 26 receives an operation performed by the foot of an operator, an assistant, etc. The foot switch 26 outputs an operation signal indicating the content of the received operation to the CCU 12 and the treatment tool device 14.

[0030] By using the endoscope system 1, the surgeon can perform operations such as resecting an affected area in the abdomen 31 while viewing the captured image displayed on the display device 11.

[0031] FIG. 2 is a perspective view showing the appearance of the endoscope 19. As shown in FIG.

[0032] As shown in Figure 2, the endoscope 19, which is a rigid endoscope, is composed of a camera head 51 and a scope 52 with a long, thin lens barrel. There are two types of endoscopes: flexible endoscopes, in which the portion inserted into the body is bendable, and rigid endoscopes, in which the portion is not bendable. The endoscope 19 is the latter type. The surgeon holds the camera head 51 and inserts the scope 52 into the patient's body to perform surgery.

[0033] Inside the camera head 51, there are provided an imaging element that performs photoelectric conversion of light from inside the body that is guided by the lens in the scope 52, a drive unit for driving the lens in the scope 52, and the like. The camera head 51 guides light emitted from the light source device 13 through the scope 52 to illuminate the inside of the abdomen 31 and capture images of the operative area. The camera head 51 outputs the captured images obtained by capturing images to the CCU 12 via a camera cable.

[0034] The scope 52 is detachable from the camera head 51. There are multiple types of scopes 52 with different specifications, such as scopes with different diameters (scope diameters) and scopes with different lens F-numbers. The type of scope 52 to be used is selected appropriately depending on the type of surgery, the condition of the surgical site, etc., and is attached to the camera head 51.

[0035] FIG. 3 is a diagram showing an example of a captured image.

[0036] As shown in FIG. 3, a circular effective area is formed substantially in the center of the horizontally long rectangular captured image, and a mask area, which is a dark area, is formed outside the effective area.

[0037] The mask area is an area formed by the optical shadow (vignetting) of the scope 52. Because the shape of the scope 52 is a long, thin cylinder, a pitch-black area appears outside the image circle. The condition of the surgical site is displayed within the effective area, which is an area without vignetting. Note that vignetting by the scope 52 refers to an optical shadow that occurs when the optical path is physically blocked by, for example, the side wall of the scope 52.

[0038] As described above, the scope 52 is switched depending on the type of surgery, etc. The position, size, and shape of the effective area in the captured image change depending on the type of scope 52 attached to the camera head 51.

[0039] 4 is a diagram showing an example of a captured image, in which the state of the operation site within the effective area is not shown.

[0040] The captured image shown in A of Fig. 4 is captured when a scope 52 having a scope diameter shorter than the vertical length of the captured image is attached. The captured image in A of Fig. 4 includes the entire circular effective area, similar to the captured image in Fig. 3.

[0041] On the other hand, the captured image shown in Fig. 4B is captured when a scope 52 is attached whose diameter is longer than the vertical length of the captured image. The effective area in Fig. 4B is a circle with the top and bottom cut off.

[0042] In this way, an effective area having a diameter corresponding to the diameter of the scope is formed in the captured image. From the size of the diameter of the effective area included in the captured image, it becomes possible to identify the size of the diameter of the scope 52 attached to the camera head 51.

[0043] As described above, in the endoscope system 1, the focus of the endoscope 19 is automatically adjusted by AF.

[0044] If the AF evaluation value, which is an evaluation value for performing AF, is calculated for the entire captured image, the AF evaluation values for each region, including the mask region, will be calculated, which is not preferable. When calculating the AF evaluation value including the mask region, for example, an operation such as focusing on the edge (boundary) of the mask region is performed.

[0045] Also, if the region for calculating the AF evaluation value is fixed and set within the effective region, it can only be set in a narrow range near the center of the captured image that becomes the effective region regardless of which scope 52 is attached. As shown in FIG. 4, the range of the effective region changes depending on the attached scope 52.

[0046] In the CCU 12, based on the captured image, the scope diameter of the scope 52 attached to the endoscope 19, the center position of the effective region, etc. are specified, and based on the specified information, the evaluation value calculation target region, which is the region for calculating the AF evaluation value, is set within the effective region. Also, the AF evaluation value is calculated for the evaluation value calculation target region, and automatic focusing processing is performed.

[0047] Since the AF evaluation value is calculated for the evaluation value calculation target region set within the effective region and automatic focusing processing is performed, the CCU 12 can surely focus on the surgical site shown in the effective region.

[0048] In this way, a series of processes of the CCU 12 that sets the evaluation value calculation target region according to the scope 52 attached to the endoscope 19 and performs automatic focusing processing will be described later.

[0049] <<Example of Detecting Mask Region Based on Detection Value in the First Embodiment>> <Configuration Example of CCU and Endoscope> FIG. 5 is a block diagram showing a configuration example of the CCU 12 and the endoscope 19.

[0050] As shown on the left side of FIG. 5, the camera head 51 is made up of an image sensor 111, an image sensor driver 112, a lens driver 113, a zoom lens driver 114, and a focus lens driver 115.

[0051] The image sensor 111 is configured by, for example, a CMOS image sensor or a CCD image sensor. The image sensor 111 converts an optical image formed on an imaging surface into an electric signal by photoelectric conversion, and outputs the electric signal to the CCU 12 as an image signal.

[0052] The imaging element driver 112 is a driver for driving the imaging element 111. The imaging element driver 112 outputs a drive signal to cause the imaging element 111 to perform predetermined operations such as an imaging operation and a reset operation. The drive signal output by the imaging element driver 112 controls, for example, the shutter speed of the imaging element 111.

[0053] The lens driver 113 is configured by a processor such as a CPU (Central Processing Unit), a DSP (Digital Signal Processor), etc. The lens driver 113 controls the operations of a zoom lens driving unit 114 and a focus lens driving unit 115 in accordance with a control signal supplied from the CCU 12.

[0054] The zoom lens driver 114 moves the zoom lens 101 in the scope 52 along the optical axis to adjust the imaging magnification.

[0055] The focus lens driver 115 moves the focus lens 102 in the scope 52 along the optical axis to adjust the focus.

[0056] As shown on the right side of FIG. 5, the CCU 12 includes a camera signal processing unit 131, a detection frame gate 132, a detection unit 133, a mask detection unit 134, an AF detection unit 135, and a lens controller 136.

[0057] The camera signal processing unit 131 performs various signal processing such as white balance processing and gamma correction processing on the imaging signal supplied from the imaging element 111 .

[0058] The camera signal processing unit 131 outputs the imaging signal obtained by performing the signal processing as a video signal to the display device 11. Based on the video signal output from the camera signal processing unit 131, an image of the operative area is displayed on the display device 11. The imaging signal output from the camera signal processing unit 131 is also supplied to the detection frame gate 132.

[0059] The detection frame gate 132 sets a detection frame in a predetermined area on the captured image under the control of the lens controller 136. For example, before detecting a mask area, the detection frame is set in a predetermined area of ​​the captured image, including the mask area. After detecting the mask area, the detection frame is set in the valid area.

[0060] The detection frame gate 132 outputs the image pickup signals of pixels within the detection frame out of the image pickup signals supplied from the camera signal processing unit 131. The image pickup signals output from the detection frame gate 132 are supplied to the detection unit 133 and the AF detection unit 135.

[0061] The detection unit 133 detects the imaging signal supplied from the detection frame gate 132 and outputs the detection value of each detection frame to the mask detection unit 134. The detection unit 133, for example, integrates the luminance values ​​of the pixels in each detection frame, and obtains the integration result as a detection value.

[0062] The mask detection unit 134 detects the edge of the mask area based on the detection value supplied from the detection unit 133. Although the description will be given assuming that the mask detection unit 134 detects the edge of the mask area, detecting the edge of the mask area also corresponds to detecting the edge of the effective area.

[0063] Based on the position of the edge of the detected mask area, the mask detection unit 134 identifies the diameter of the scope 52 attached to the camera head 51. Based on the position of the edge of the mask area, the mask detection unit 134 also identifies the center position of the effective area and the position (range) of the mask area.

[0064] The mask detection unit 134 outputs information regarding the scope diameter, the center position of the effective area, and the position of the mask area as the mask detection result to the lens controller 136. As will be described later, the detection of the mask area by the mask detection unit 134 may also be performed based on the AF evaluation value calculated by the AF detection unit 135.

[0065] The AF detection unit 135 calculates an AF evaluation value based on the imaging signal supplied from the detection frame gate 132. For example, the AF detection unit 135 calculates the AF evaluation value, which represents contrast, by second-order differentiation using the luminance signals of all pixels within the AF detection frame. Generally, when an image is in focus, the difference in luminance signals between adjacent pixels is greater than when the image is out of focus, resulting in greater contrast.

[0066] The AF evaluation value calculated by the AF detection unit 135 is supplied to the lens controller 136 and also to the mask detection unit 134 as appropriate.

[0067] The lens controller 136 outputs a control signal to the lens driver 113 of the camera head 51 to adjust the position of the zoom lens 101 .

[0068] Furthermore, lens controller 136 outputs a control signal to lens driver 113 of camera head 51 to adjust the position of focus lens 102. The process of adjusting the position of focus lens 102 by outputting the control signal becomes an automatic focusing process for performing AF of endoscope 19. The control signal at this time is a signal that includes at least the position of focus lens 102 of lens driver 113 or the amount of movement of the position of focus lens 102 as an AF control parameter.

[0069] The lens controller 136 adjusts the shutter speed and ISO sensitivity of the image sensor 111 by outputting a control signal to the image sensor driver 112 of the camera head 51 as needed. The process of adjusting the shutter speed and ISO sensitivity by outputting the control signal and controlling the exposure without the operator's operation is the automatic exposure process for performing AE. As will be described later, the AE function is realized based on the detection result of the mask area. The control signal at this time is a signal that includes at least the speed value of the shutter speed or the sensitivity value of the ISO sensitivity as an AE control parameter.

[0070] A focus instruction signal, a zoom instruction signal, a manual / autofocus switching signal, etc. are input to the lens controller 136 in response to operations by the surgeon. The zoom instruction signal is a signal that indicates the content of zoom adjustment by the surgeon, and the focus instruction signal is a signal that indicates the content of focus adjustment by the surgeon.

[0071] The manual / autofocus switching signal is a signal for switching between manual mode and autofocus mode for focus adjustment, and the focus instruction signal is input when manual mode for focus adjustment is selected.

[0072] <Mask area detection> FIG. 6 is a diagram showing an example of setting the detection frame.

[0073] The detection frame is set, for example, over the entire area of ​​the captured image. In the example of Fig. 6, 30 detection frames are set in the column direction (vertical direction) and 40 detection frames are set in the row direction (horizontal direction) in a tiled pattern over the entire area of ​​the captured image. The size of the entire detection frame in Fig. 6 is the same as the size of the captured image.

[0074] In the mask detection unit 134, the edges of the mask area are detected based on the detection value of 1200 obtained based on the image pickup signals of the pixels within each detection frame.

[0075] In this way, a detection frame having a resolution sufficient to detect the edge of the mask area is set for the captured image.

[0076] In the example of FIG. 6, the size of the entire detection frame is the same as the size of the captured image, but the detection frame may be set to a range smaller than the size of the captured image. The number of detection frames is also arbitrary.

[0077] FIG. 7 is a diagram showing an example of detecting the edge of a mask area.

[0078] The sequence of "x" characters inside each detection frame represents a specific detection value. A detection frame containing the number "0" indicates that the detection value is 0, i.e., the detection frame contains black pixels.

[0079] When such detection values ​​are obtained in the detection unit 133, the mask detection unit 134 in FIG. 5 obtains the maximum detection value among the detection values ​​in the detection frames of each column and the maximum detection value among the detection values ​​in the detection frames of each row.

[0080] Furthermore, as indicated by the tip of the white arrow #1, the mask detection unit 134 finds the difference between the detection values ​​in every other column in the array of the maximum detection values ​​in each column. Similarly, in the row direction, the mask detection unit 134 finds the difference between the detection values ​​in every other row in the array of the maximum detection values ​​in each row.

[0081] For example, the mask detection unit 134 looks at the differences in the detection values ​​for every other column, starting from the left side of the image, and detects the position of the column where the difference is greater than or equal to the threshold as the position of the left edge of the mask area.Furthermore, the mask detection unit 134 looks at the differences in the detection values ​​for every other column, starting from the right side of the image, and detects the position of the column where the difference is greater than or equal to the threshold as the position of the right edge of the mask area.

[0082] Similarly, in the row direction, the mask detection unit 134 focuses on the differences in the detection values ​​of every other row, starting from the top of the image, and detects the position of the row where a difference equal to or greater than the threshold is found as the position of the upper edge of the mask area.Furthermore, the mask detection unit 134 focuses on the differences in the detection values ​​of every other row, starting from the bottom of the image, and detects the position of the row where a difference equal to or greater than the threshold is found as the position of the lower edge of the mask area.

[0083] The mask detection unit 134 detects all edges surrounding the effective area, as shown in FIG. 8, based on the positions of the top, bottom, left, and right edges and the fact that the effective area has a circular shape.

[0084] 8, the detection frame indicated by diagonal lines is the detection frame at the position detected as the edge of the mask area. The area outside the detection frame indicated by diagonal lines is the mask area, and the area inside (including the position of the detection frame indicated by diagonal lines) is the valid area.

[0085] If the diameter of the effective area is large and the effective area is cut off at the top and bottom, the left and right edges are detected based on the difference between the detection values ​​of every other column, and the mask area is identified. If no edges are found, it is identified as no mask area.

[0086] When the mask area and the effective area are detected, the mask detection unit 134 determines, for example, the average edge width based on the edge width of the effective area as the diameter of the effective area, i.e., the scope diameter. The mask detection unit 134 also determines the position indicated by the average median value of the edge width as the center position of the effective area. Note that the range of average edge widths and the corresponding scope diameters may be stored in advance, and the scope diameter may be determined by referring to a table based on the determined average edge width.

[0087] Information indicating the thus determined scope diameter and the center position of the effective area, together with information regarding the position of the mask area, is supplied to the lens controller 136. The lens controller 136 sets a detection frame used to calculate the AF evaluation value, etc., based on the detection result of the mask area.

[0088] <<Second embodiment: Example of detecting a mask area based on an AF evaluation value>> It is also possible to detect the mask area based on the AF evaluation value obtained by the AF detection section 135, rather than based on the detection value obtained by the detection section 133.

[0089] First setting example FIG. 9 is a diagram showing an example of setting a detection frame when detecting a mask area based on an AF evaluation value.

[0090] For example, the detection frame gate 132 sets thin strip-shaped detection frames at the top, bottom, left, and right edges of the captured image, as shown as detection frames F1 to F4 on the captured image on the left side of Fig. 9. The detection frames F1 to F4 are set according to control by, for example, the lens controller 136.

[0091] Detection frames F1 and F2 indicated by dashed lines are detection frames for detecting the upper and lower edges of the mask area, respectively. Detection frames F3 and F4 indicated by dashed lines are detection frames for detecting the left and right edges of the mask area, respectively.

[0092] Furthermore, the detection frame gate 132 moves the positions of the detection frames F1 to F4 toward the center of the captured image, as indicated by the white arrows.

[0093] The AF evaluation value is calculated sequentially in the AF detection unit 135 based on the imaging signals corresponding to the pixels within the detection frame set at different positions in this way.

[0094] When a change equal to or greater than a threshold occurs in the AF evaluation value calculated by the AF detection unit 135, the mask detection unit 134 identifies the position where the change is detected as the position of the edge of the mask area. For example, as indicated by the tip of arrow #2, when detection frames F1 to F4 are set at the positions of the edges of the mask area, a change in the AF evaluation value equal to or greater than the threshold is detected.

[0095] Based on the distance L1 between detection frames F1 and F2 and the distance L2 between detection frames F3 and F4, the mask detection unit 134 identifies the diameter of the scope attached to the camera head 51, the center position of the effective area, and the position of the mask area. For example, the average of the distances L1 and L2 is identified as the scope diameter, and the position indicated by the average of the median values ​​of the edge widths is identified as the center position of the effective area.

[0096] Second setting example FIG. 10 is a diagram showing another example of setting the detection frame when detecting a mask area based on the AF evaluation value.

[0097] The detection frame setting method shown in Fig. 10 differs from the setting method in Fig. 9 in that the detection frames F1 to F4 are set so as to gradually increase in width rather than changing the positions of the detection frames F1 to F4. The widths of the detection frames F1 to F4 are gradually increased by moving the inner edges of the narrow strip-shaped detection frames F1 to F4 set on the top, bottom, left, and right edges of the captured image toward the center of the captured image as indicated by the white arrows.

[0098] The AF evaluation value is calculated in sequence in the AF detection unit 135 based on the imaging signals corresponding to the pixels within the detection frame set with varying widths.

[0099] When a change equal to or greater than a threshold occurs in the AF evaluation value calculated by the AF detection unit 135, the mask detection unit 134 identifies the position of the inner edge of the detection frame when the change is detected as the position of the edge of the mask area. For example, as indicated by the tip of arrow #3, when the inner edges of detection frames F1 to F4 are set to the positions of the edges of the mask areas, a change in the AF evaluation value equal to or greater than the threshold is detected.

[0100] The mask detection unit 134 determines the diameter of the scope attached to the camera head 51, the center position of the effective area, and the position of the mask area based on the distance L1 between the detection frame F1 and the detection frame F2, and the distance L2 between the detection frame F3 and the detection frame F4.

[0101] Third setting example FIG. 11 is a diagram showing yet another example of setting the detection frame when detecting a mask area based on the AF evaluation value.

[0102] The detection frame setting method shown in FIG. 11 differs from the setting method shown in FIG. 9 etc. in that, in the initial state, instead of a narrow band-shaped detection frame, a plurality of small detection frames each having a substantially square shape are set.

[0103] That is, the detection frame gate 132 sets approximately square detection frames F1-1 to F1-3 on the upper edge of the captured image, and sets approximately square detection frames F2-1 to F2-3 on the lower edge, as shown on the left side of Fig. 11. The detection frame gate 132 sets approximately square detection frames F3-1 to F3-3 on the left edge of the captured image, and sets approximately square detection frames F4-1 to F4-3 on the right edge.

[0104] Furthermore, the detection frame gate 132 moves the position of each detection frame toward the center of the captured image, or widens the width of the detection frame toward the center of the captured image.

[0105] The AF evaluation value is calculated in sequence in the AF detection unit 135 based on the imaging signals corresponding to the pixels within the detection frame that is set with the position and width changed.

[0106] When a change equal to or greater than a threshold occurs in the AF evaluation value calculated by the AF detection unit 135, the mask detection unit 134 identifies the position where the change is detected as the position of the edge of the mask area.

[0107] For example, as indicated by the tip of arrow #4, when the approximately square detection frames F1-1 to F1-3 and detection frames F2-1 to F2-3 are set at the edge positions of the mask area, a change in the AF evaluation value equal to or greater than the threshold is detected.

[0108] Furthermore, when the inner edges of the detection frames F3-1 to F3-3 and the detection frames F4-1 to F4-3 reach the edge positions of the mask areas, a change in the AF evaluation value equal to or greater than the threshold is detected.

[0109] The mask detection unit 134 identifies the diameter of the scope attached to the camera head 51, the center position of the effective area, and the position of the mask area based on the distance between the opposing detection frames.

[0110] 11, the detection frames F1-1 to F1-3 and F2-1 to F2-3 set at the top and bottom edges of the captured image are set at different positions without changing their shapes, but they may be set at different widths.Furthermore, the detection frames F3-1 to F3-3 and F4-1 to F4-3 set at the left and right edges of the captured image are set at different widths, but they may be set at different positions without changing their shapes.

[0111] The method of changing the shape of the detection frame and the method of changing the width may be used in combination. By changing the position and width of the small detection frame, it becomes possible to detect the detailed shape of the edge of the mask area.

[0112] If the diameter of the effective area is large and the top and bottom of the effective area are cut off, the mask area is detected based only on the AF evaluation value calculated based on the detection frames set on the left and right. If no edge is found, it is determined that there is no mask area.

[0113] Information indicating the scope diameter and the center position of the effective area determined based on the AF evaluation value, together with information regarding the position of the mask area, is supplied to the lens controller 136. The lens controller 136 sets a detection frame used to calculate the AF evaluation value, etc., based on the detection result of the mask area.

[0114] <<Third embodiment: Example 1 of setting evaluation value calculation target area>> FIG. 12 is a diagram showing an example of setting an evaluation value calculation target region.

[0115] 12, the lens controller 136 sets the evaluation value calculation target area, which is the area for calculating the AF evaluation value, inside the effective area so as not to overlap with the mask area. For example, the evaluation value calculation target area is set by enlarging or reducing the size of the default area and shifting the default center position based on the detection result of the mask area.

[0116] A horizontally elongated rectangular area A1 shown in A of Fig. 12 is an evaluation value calculation target area set within an effective area having a diameter shorter than the vertical length of the captured image. An area A2 shown in B of Fig. 12 is an evaluation value calculation target area set within an effective area having a diameter longer than the vertical length of the captured image.

[0117] Each evaluation value calculation target region is set so that it fits entirely within the valid area. In Figures 12A and 12B, two horizontally long rectangles representing the evaluation value calculation target regions are shown within the valid area, which indicates that the size of the evaluation value calculation target region can be set arbitrarily as long as it fits within the valid area.

[0118] In the example of FIG. 12, the shape of the evaluation value calculation target region is a horizontally long rectangle, but it may be another shape such as a square or a circle.

[0119] Information about such an evaluation value calculation target area is supplied from the lens controller 136 to the detection frame gate 132, and a detection frame for calculating an AF evaluation value is set within the evaluation value calculation target area.

[0120] The AF detection unit 135 calculates an AF evaluation value based on the image pickup signals of pixels within a detection frame set within the evaluation value calculation target area. Furthermore, the lens controller 136 performs automatic focusing processing based on the calculated AF evaluation value.

[0121] This allows the CCU 12 to reliably focus on the surgical site that is captured in the effective area.

[0122] <<Fourth embodiment: Example 2 of setting evaluation value calculation target area>> FIG. 13 is a diagram showing another example of setting the evaluation value calculation target region.

[0123] Each square shown on the captured image in Fig. 13 represents a detection frame. In the example of Fig. 13, a plurality of detection frames are set in a tiled pattern, eight vertically and eight horizontally, around the center of the captured image. In this example, the positions of the detection frames are fixed regardless of the range of the effective area. Image capture signals of the pixels in each detection frame are supplied from the detection frame gate 132 to the AF detection unit 135.

[0124] The lens controller 136 selects, from among the detection frames thus fixedly set, a detection frame that does not overlap the mask area and that falls entirely within the valid area as an evaluation value calculation target area for calculating an AF evaluation value. The detection frames shown in color in Fig. 13 are detection frames selected as the evaluation value calculation target area. On the other hand, a detection frame that overlaps the mask area is treated as an invalid detection frame (weight 0).

[0125] The lens controller 136 outputs information about the detection frame selected as the evaluation value calculation target region to the AF detection unit 135, and causes the AF detection unit 135 to calculate an AF evaluation value for the detection frame selected as the evaluation value calculation target region.

[0126] That is, in the example of Figure 12, the detection frame for calculating the AF evaluation value is set based on the detection result of the mask area, whereas in the example of Figure 13, the detection frame for calculating the AF evaluation value is selected from pre-set detection frames.

[0127] In the example of A in Fig. 13, a part of the preset detection frames that falls within the valid area is set as the evaluation value calculation target area. In addition, in the example of B in Fig. 13, all of the preset detection frames fall within the valid area, so all of the detection frames are set as the evaluation value calculation target area.

[0128] In the example of FIG. 13, all the detection frames that fit within the effective area are set as the evaluation value calculation target area, but a part of the detection frames that fit within the effective area may be set as the evaluation value calculation target area.

[0129] Information regarding such an evaluation value calculation target area is supplied from the lens controller 136 to the AF detection unit 135, and an AF evaluation value is calculated based on the pixel signals of the pixels in the detection frame set as the evaluation value calculation target area. Further, based on the calculated AF evaluation value, an automatic focusing process is performed by the lens controller 136.

[0130] Also by this, the CCU 12 can surely focus on the surgical part imaged in the effective area.

[0131] <<Example of Automatic Focusing Process in the Fifth Embodiment>> Based on the scope diameter, the center position of the effective area, and the position of the mask area specified as described above, the F value of the scope 52 may be estimated, and the depth of focus may be obtained based on the estimated F value.

[0132] The F value and the depth of focus of the scope 52 are used by the lens controller 136, for example, to set parameters that define the content of the automatic focusing process, such as the AF speed, the focusing accuracy, and the wobbling amplitude amount. The lens controller 136 calculates the AF speed, the focusing accuracy, and the wobbling amplitude amount respectively based on the F value and the depth of focus of the scope 52, and performs an automatic focusing process based on the calculation results.

[0133] Thereby, the lens controller 136 can perform the automatic focusing process with higher precision. Note that a table of the F value and the depth of focus corresponding to the scope diameter may be stored in advance, and the F value and the depth of focus may be obtained by referring to the table from the specified scope diameter.

[0134] <<Operation of CCU>> <AF Process> The AF processing of the CCU 12 will be described with reference to the flowchart of FIG.

[0135] The processing in FIG. 14 is performed, for example, when the endoscope 19 captures an image of the operation site and an image signal is supplied from the image sensor 111.

[0136] In step S1, the camera signal processing unit 131 performs various signal processing such as white balance processing on the imaging signal supplied from the imaging element 111.

[0137] In step S2, the detection frame gate 132 outputs the image pickup signals of the pixels in each detection frame.

[0138] For example, when the detection of the mask area is performed as described with reference to Figures 7 and 8, the image signals of the pixels within each detection frame set over the entire captured image are output from the detection frame gate 132 and supplied to the detection unit 133.

[0139] Furthermore, when the detection of the mask area is performed as described with reference to Figures 9, 10, and 11, the image signals of the pixels within each detection frame set at a predetermined position in the captured image are output from the detection frame gate 132 and supplied to the AF detection unit 135.

[0140] In step S3, a mask area detection process is performed. Information regarding the scope diameter, the center position of the effective area, and the position of the mask area identified by the mask area detection process is supplied from the mask detection unit 134 to the lens controller 136. Details of the mask area detection process will be described later with reference to the flowcharts in FIGS. 15 and 16.

[0141] In step S4, the lens controller 136 sets an evaluation value calculation target area based on the detection result of the mask area.

[0142] When the evaluation value calculation target area is set as described with reference to Figure 12, the lens controller 136 outputs information regarding the evaluation value calculation target area to the detection frame gate 132, causing a detection frame to be set within the evaluation value calculation target area.

[0143] Furthermore, when the evaluation value calculation target area is set as described with reference to FIG. 13, the lens controller 136 outputs information regarding the evaluation value calculation target area to the AF detection unit 135, and causes the AF evaluation value to be calculated for the detection frame selected as the evaluation value calculation target area.

[0144] In step S5, the AF detection unit 135 calculates an AF evaluation value based on the imaging signal supplied from the detection frame gate 132, using the detection frame of the evaluation value calculation target region as a target.

[0145] In step S6, lens controller 136 performs automatic focusing processing based on the AF evaluation value calculated by AF detection unit 135. A control signal is output from lens controller 136 to lens driver 113 of camera head 51, and AF is achieved by adjusting the position of focus lens 102.

[0146] The above processing is repeated while the imaging signal is supplied from the imaging element 111.

[0147] <Mask area detection process> Next, the mask area detection process performed in step S3 of FIG. 14 will be described with reference to the flowchart of FIG.

[0148] 15 is a process in which the detection of a mask area is performed using detection values ​​obtained by the detection unit 133. The detection frame gate 132 supplies the detection unit 133 with image signals of pixels within each detection frame set for the entire captured image as described with reference to FIGS.

[0149] In step S11, the detection unit 133 detects the imaging signal supplied from the detection frame gate 132 and outputs the detection value of each detection frame.

[0150] In step S12, the mask detection unit 134 detects the maximum detection value among the detection values ​​in the detection frame for each column, and also detects the maximum detection value among the detection values ​​in the detection frame for each row.

[0151] In step S13, the mask detection unit 134 calculates the difference between the detection values ​​for every other column in the array of the maximum detection values ​​for each column. The mask detection unit 134 also calculates the difference between the detection values ​​for every other row in the array of the maximum detection values ​​for each row.

[0152] In step S14, the mask detection unit 134 sequentially focuses on the differences in the detection values ​​for every other column, and detects the positions where the difference is equal to or greater than the threshold as the positions of the left and right edges of the mask area.The mask detection unit 134 also sequentially focuses on the differences in the detection values ​​for every other row, and detects the positions where the difference is equal to or greater than the threshold as the positions of the top and bottom edges of the mask area.

[0153] In step S15, the mask detection unit 134 identifies the scope diameter and the center position of the effective area based on the edge width of the effective area.

[0154] In step S16, the mask detection unit 134 outputs information about the scope diameter, the center position of the effective area, and the position of the mask area as the detection result of the mask area to the lens controller 136. After that, the process returns to step S3 in Fig. 14, and the subsequent processes are performed.

[0155] Next, another mask area detection process performed in step S3 of FIG. 14 will be described with reference to the flowchart of FIG.

[0156] In the process of FIG. 16, the detection of the mask area is performed using the AF evaluation value obtained by the AF detection unit 135.

[0157] In step S21, the lens controller 136 outputs information regarding the position of the detection frame to the detection frame gate 132, causing the detection frame to be set. For example, the lens controller 136 supplies the detection frame gate 132 with information for setting the detection frame at each position of the captured image, as described with reference to Figures 9, 10, and 11. When the detection frame is set, the detection frame gate 132 supplies the AF detection unit 135 with image signals corresponding to pixels within the detection frame.

[0158] In step S22, the AF detection unit 135 calculates an AF evaluation value based on the imaging signal supplied from the detection frame gate 132. The calculated AF evaluation value is supplied to the mask detection unit .

[0159] In step S23, the mask detection unit 134 determines whether a change equal to or greater than the threshold value has occurred in the AF evaluation value. If it is determined in step S23 that a change equal to or greater than the threshold value has not occurred, the process returns to step S21, and the detection frame is set with its position or width changed, and the above processing is repeated.

[0160] On the other hand, if it is determined in step S23 that the AF evaluation value has changed by more than the threshold value, in step S24, the mask detection unit 134 identifies the position where the AF evaluation value has changed by more than the threshold value as the position of the edge of the mask area.

[0161] The processing after the edge position of the mask area is identified is the same as the processing from step S15 onwards in Fig. 15. That is, in step S25, the mask detection unit 134 identifies the scope diameter and the center position of the effective area based on the edge width of the effective area.

[0162] In step S26, the mask detection unit 134 outputs information about the scope diameter, the center position of the effective area, and the position of the mask area as the detection result of the mask area to the lens controller 136. After that, the process returns to step S3 in Fig. 14, and the subsequent processes are performed.

[0163] Through the above processing, the CCU 12 can set, according to the type of the scope 52 attached to the endoscope 19 which is a rigid endoscope, the area to be the calculation target of the AF evaluation value at an appropriate position within the effective area and with an appropriate size.

[0164] Also, the CCU 12 can set parameters such as the AF speed and the wobbling amplitude amount to appropriate values according to the specifications of the scope 52, and thereby can improve the AF performance. By setting the AF speed and the wobbling amplitude amount according to the specifications of the scope 52, it becomes possible to avoid the AF fluctuation, avoid the defocus prevention, and avoid the image quality deterioration such as the wobbling operation being visible.

[0165] For example, when a scope with a large diameter is attached and AF is performed using an F value and a depth of focus larger than appropriate values, AF fluctuations occur due to the AF speed being too fast. Also, problems such as the minute vibration of the wobbling operation being visible on the output image occur.

[0166] Conversely, when a scope with a small diameter is attached and AF is performed using an F value and a depth of focus smaller than appropriate values, problems such as it taking a long time to focus due to the AF speed being too slow occur. Also, problems such as the so-called defocus prevention where the amplitude amount of the wobbling operation is insufficient and the AF stops in a defocused state occur because the focusing direction cannot be obtained.

[0167] By performing AF using appropriate values according to the specifications of the scope, it becomes possible to avoid these problems caused by the inappropriate specifications of the scope.

[0168] That is, the endoscope system 1 can provide the surgeon with an endoscope image suitable for the surgery.

[0169] <<Other examples>> <Example applied to AE> It is also possible to perform automatic exposure processing for AE based on the detection results of the mask area. AE is performed by automatically adjusting parameters such as shutter speed and ISO sensitivity without the operator's operation.

[0170] FIG. 17 is a block diagram showing an example of the configuration of the CCU 12 that performs AE.

[0171] Of the components shown in Fig. 17, the same components as those described with reference to Fig. 5 are assigned the same reference numerals. Duplicate descriptions will be omitted as appropriate. The configuration of the CCU 12 shown in Fig. 17 differs from the configuration shown in Fig. 5 in that an exposure detection unit 141 is provided.

[0172] The detection frame gate 132 outputs, to the exposure detection unit 141, the image pickup signals of pixels within the detection frame for exposure evaluation, among the image pickup signals supplied from the camera signal processing unit 131.

[0173] The exposure detection unit 141 calculates an AE evaluation value, which is an evaluation value for exposure evaluation, based on the imaging signals of the pixels within the detection frame, and outputs the AE evaluation value to the lens controller 136 .

[0174] The lens controller 136 outputs a control signal for adjusting the shutter speed and ISO sensitivity to the image sensor driver 112 based on the AE evaluation value calculated by the exposure detection unit 141, and performs automatic exposure processing.

[0175] FIG. 18 is a diagram showing an example of setting an evaluation value calculation target region for the AE evaluation value.

[0176] The setting of the evaluation value calculation target region shown in FIG. 18 corresponds to the setting of the evaluation value calculation target region described with reference to FIG.

[0177] 18, the lens controller 136 sets the evaluation value calculation target area, which is the target for calculating the AE evaluation value, inside the effective area so as not to overlap with the mask area. For example, the evaluation value calculation target area is set by enlarging or reducing the size of the default area and shifting the default center position based on the detection result of the mask area.

[0178] A horizontally elongated rectangular area A11 shown on the right side of Fig. 18 is an evaluation value calculation target area set within an effective area having a diameter shorter than the vertical length of the captured image. In the example of Fig. 18, the evaluation value calculation target area is set by reducing the default size shown on the left side of Fig. 18, for example.

[0179] Information about such an evaluation value calculation target region is supplied from the lens controller 136 to the detection frame gate 132, and a detection frame for calculating an AE evaluation value is set within the evaluation value calculation target region.

[0180] The exposure detection unit 141 calculates an AE evaluation value based on the image pickup signals of pixels within a detection frame set within the evaluation value calculation target area. Furthermore, the lens controller 136 performs automatic exposure processing based on the calculated AE evaluation value.

[0181] This allows the CCU 12 to adjust the exposure of the surgical area captured in the effective area to an appropriate exposure.

[0182] For example, if the detection frame for exposure evaluation is set in a masked area, the AE controls the exposure to increase, which can result in overexposure. By using only the detection frame within the effective area to evaluate the exposure, the CCU12 can adjust the brightness of the surgical site to the appropriate exposure.

[0183] FIG. 19 is a diagram showing another example of setting the evaluation value calculation target region for the AE evaluation value.

[0184] The setting of the evaluation value calculation target region shown in FIG. 19 corresponds to the setting of the evaluation value calculation target region described with reference to FIG.

[0185] Each square shown on the captured image on the left side of Fig. 19 represents a detection frame. In the example of Fig. 19, a plurality of detection frames are set in a tiled pattern, 10 vertically and 10 horizontally, around the center of the captured image. In this example, the positions of the detection frames are fixed regardless of the range of the effective area. The detection frame gate 132 supplies the exposure detection unit 141 with an image signal of the pixels in each detection frame.

[0186] The lens controller 136 selects, from among the detection frames thus fixedly set, a detection frame that does not overlap the mask area and falls entirely within the valid area as an evaluation value calculation target area for calculating an AE evaluation value. The detection frames shown in color on the right side of Fig. 19 are detection frames selected as the evaluation value calculation target area. On the other hand, a detection frame that overlaps the mask area is treated as an invalid detection frame (weight 0).

[0187] The lens controller 136 outputs information about the detection frame selected as the evaluation value calculation target region to the exposure detection unit 141, and causes the exposure detection unit 141 to calculate an AE evaluation value for the detection frame selected as the evaluation value calculation target region.

[0188] Information relating to such an evaluation value calculation target area is supplied from the lens controller 136 to the exposure detection unit 141, and an AE evaluation value is calculated based on pixel signals of pixels in the detection frame selected as the evaluation value calculation target area. Furthermore, the lens controller 136 performs automatic exposure processing based on the calculated AE evaluation value.

[0189] This also enables the CCU 12 to adjust the exposure of the surgical area captured in the effective area to an appropriate exposure.

[0190] The automatic exposure process may be performed in combination with the automatic focusing process described above. That is, the control process performed by the lens controller 136 is a process including at least one of the automatic exposure process and the automatic focusing process.

[0191] <Example applied to AWB> It is also possible to cause the camera signal processing unit 131 to perform white balance processing for performing AWB (auto white balance) of the endoscope system 1 based on the detection result of the mask area.

[0192] In this case, the detection result of the mask area is supplied from the mask detection unit 134 to the camera signal processing unit 131, and the white balance processing is performed based on the imaging signals of the pixels in the effective area.

[0193] The white balance processing is a process of correcting the color of the captured image so that the surgical site appears with an appropriate color tone. For example, it is a process of estimating the imaging environment based on the imaging signals (color signals) of the pixels in the effective area and correcting the color generated from the imaging signals.

[0194] Alternatively, a white balance process may be performed in which the color temperature of the light source is estimated from the imaging signals of the pixels in the effective area and color correction is performed to match the color temperature of the light source stored in advance.

[0195] Alternatively, when it is determined that the redness in the effective area is weak, a white balance process may be performed to enhance the redness so that the blood vessels are emphasized.

[0196] Thus, by performing the white balance process using the imaging signals of the pixels in the effective area, it becomes possible to obtain a captured image in which the surgical site appears with an appropriate color tone.

[0197] <Other examples> Although the case where AF, AE, and AWB using the detection result of the mask area are performed in the endoscope system 1 has been described, it is also applicable to the case where AF, AE, and AWB are performed in the microscope system.

[0198] Although the detection of the mask area is performed based on the detection value obtained by the detection unit 133 or the AF evaluation value obtained by the AF detection unit 135, it may also be performed using both the detection value and the AF evaluation value.

[0199] The above-described series of processes can be executed by hardware or software. When the series of processes is executed by software, the program constituting the software is installed from a program recording medium into a computer incorporated in dedicated hardware or a general-purpose personal computer.

[0200] FIG. 20 is a block diagram showing an example of the hardware configuration of a computer that executes the above-described series of processes using a program.

[0201] A CPU (Central Processing Unit) 1001 , a ROM (Read Only Memory) 1002 , and a RAM (Random Access Memory) 1003 are interconnected by a bus 1004 .

[0202] An input / output interface 1005 is also connected to the bus 1004. An input unit 1006 including a keyboard, a mouse, etc., and an output unit 1007 including a display, a speaker, etc. are connected to the input / output interface 1005. Also connected to the input / output interface 1005 are a storage unit 1008 including a hard disk, a nonvolatile memory, etc., a communication unit 1009 including a network interface, etc., and a drive 1010 that drives removable media 1011.

[0203] In a computer configured as above, the CPU 1001 performs the above-described series of processes by, for example, loading a program stored in the storage unit 1008 into the RAM 1003 via the input / output interface 1005 and the bus 1004 and executing the program.

[0204] The program executed by the CPU 1001 is installed in the storage unit 1008 by being recorded on a removable medium 1011, or provided via a wired or wireless transmission medium such as a local area network, the Internet, or digital broadcasting.

[0205] The program executed by the computer may be a program that processes in chronological order according to the order described in this specification, or may be a program that processes in parallel or at the required timing, such as when called.

[0206] In this specification, a system refers to a collection of multiple components (devices, modules (components), etc.), regardless of whether all the components are contained in the same housing. Therefore, multiple devices housed in separate housings and connected via a network, and a single device housed in a single housing with multiple modules, are both systems.

[0207] The effects described in this specification are merely examples and are not limiting, and other effects may also be present.

[0208] The embodiments of the present technology are not limited to the above-described embodiments, and various modifications are possible without departing from the spirit of the present technology.

[0209] For example, this technology can be configured as cloud computing, in which a single function is shared and processed collaboratively by multiple devices via a network.

[0210] Furthermore, each step described in the above flowchart can be executed by one device, or can be shared and executed by multiple devices.

[0211] Furthermore, when one step includes multiple processes, the multiple processes included in that one step can be executed by one device or can be shared and executed by multiple devices.

[0212] <Configuration combination example> The present technology can also be configured as follows.

[0213] (1) a light source device that irradiates light onto a surgical field; an imaging device that images the surgical field through a detachable scope; connected to the imaging device and the light source device, a detection unit that detects an effective area of ​​the scope in an image captured by the imaging device; a control unit that executes control processing including at least one of automatic focusing processing, automatic exposure processing, and automatic white balance processing based on the evaluation value within the effective area; an information processing device comprising: An endoscopy system comprising: (2) The effective area is an area where there is no vignetting due to the scope. The endoscope system described in (1) above. (3) a first detection unit that detects a luminance signal of each area set in the captured image; The detection unit detects the effective area based on a detection value of each area of ​​the captured image. The endoscope system according to (1) or (2) above. (4) The detection unit detects the edge of the effective area based on the detection value of each area set in a tiled pattern on the captured image, and identifies the diameter of the scope and the center position of the effective area. The endoscope system described in (3) above. (5) a second detection unit that performs detection within a detection frame set in the captured image and calculates an evaluation value for the automatic focusing process; the detection unit detects the effective area based on the evaluation value for the automatic focusing process; The control unit executes the automatic focusing process as the control process based on the evaluation value for the automatic focusing process. The endoscope system described in (1) above. (6) The second detection unit changes at least one of the position and the size of the detection frame to calculate the evaluation value for the automatic focusing process used to detect the effective area. The endoscope system according to (5) above. (7) The detection unit detects the edge of the effective area based on the evaluation value for the automatic focusing process, and identifies the diameter of the scope and the center position of the effective area. The endoscope system according to (6) above. (8) The control unit sets an evaluation value calculation target region, which is a target for calculating the evaluation value, within the effective region represented by the diameter of the scope and the center position of the effective region. The endoscope system according to any one of (1) to (7). (9) The control unit executes the control process based on the evaluation value calculated in the evaluation value calculation target area within the effective area, which is represented by the diameter of the scope and the center position of the effective area, among evaluation value calculation target areas that are set in a tiled pattern on the captured image and are targets for calculating the evaluation value. The endoscope system according to any one of (1) to (8). (10) The control unit sets parameters that define the automatic focusing process based on at least one of an F-number and a focal depth of the scope estimated based on the detection result by the detection unit. The endoscope system according to any one of (1) to (9). (11) The control unit refers to a table that includes at least one of the F-number or focal depth of the scope stored in advance based on the detection result by the detection unit, and sets parameters that define the automatic focusing process based on the reference result. The endoscope system according to any one of (1) to (9). (12) The light source device irradiates the surgical field with light, an imaging device that captures an image of the surgical field through a detachable scope; an information processing device connected to the imaging device and the light source device, Detecting an effective area of ​​the scope from the captured image captured by the imaging device; Based on the evaluation value within the effective area, a control process including at least one of an automatic focusing process, an automatic exposure process, and an automatic white balance process is executed. A method for controlling an endoscope system. (13) The surgical instrument is connected to a light source device that irradiates a surgical field with light and an imaging device that images the surgical field via a detachable scope. a detection unit that detects an effective area of ​​the scope in an image captured by the imaging device; a control unit that executes control processing including at least one of automatic focusing processing, automatic exposure processing, and automatic white balance processing based on the evaluation value within the effective area; An information processing device for an endoscope system comprising: (14) a computer as an information processing device of an endoscope system connected to a light source device that irradiates a surgical field with light and an imaging device that images the surgical field via a detachable scope; Detecting an effective area of ​​the scope from the captured image captured by the imaging device; Based on the evaluation value within the effective area, a control process including at least one of an automatic focusing process, an automatic exposure process, and an automatic white balance process is executed. A program for executing a process. [Explanation of symbols]

[0214] 1 Endoscope system, 12 CCU, 19 Endoscope, 51 Camera head, 52 Scope, 111 Image sensor, 112 Image sensor driver, 113 Lens driver, 114 Zoom lens drive unit, 115 Focus lens drive unit, 131 Camera signal processing unit, 132 Detection frame gate, 133 Detection unit, 134 Mask detection unit, 135 AF detection unit, 136 Lens controller, 141 Exposure detection unit

Claims

1. a detection unit that performs detection within a detection frame set in an image captured by the endoscope and calculates a first evaluation value; a detection unit that detects an effective area of ​​the captured image, which is an area free from vignetting caused by a scope of the endoscope, based on the first evaluation value; a control unit that executes automatic focusing processing based on a second evaluation value targeted at an evaluation value calculation target area set within the effective area; An information processing device comprising:

2. The control unit further performs automatic exposure processing by adjusting a shutter speed or an imaging sensitivity based on the second evaluation value for exposure evaluation targeted at the evaluation value calculation target area. The information processing device according to claim 1 .

3. The control unit estimates an imaging environment based on imaging signals of pixels within the effective area, and further performs automatic white balance processing by correcting colors generated from the imaging signals. The information processing device according to claim 1 .

4. The detection unit performs detection within the detection frame set by changing the position, and calculates the first evaluation value used to detect the effective area. The information processing device according to claim 1 .

5. The detection unit performs detection within the detection frame that is set by changing the size along with the position, and calculates the first evaluation value used to detect the effective area. The information processing device according to claim 4 .

6. The detection unit detects the edge of the effective area based on the first evaluation value, and identifies the diameter of the scope and the center position of the effective area. The information processing device according to claim 1 .

7. The control unit sets the evaluation value calculation target region within the effective region represented by the diameter of the scope and the center position of the effective region. The information processing device according to claim 6 .

8. The control unit executes the automatic focusing process based on the second evaluation value calculated in the evaluation value calculation target area within the effective area, which is represented by the diameter of the scope and the center position of the effective area, among the evaluation value calculation target areas set in a tiled pattern on the captured image. The information processing device according to claim 6 .

9. The control unit sets parameters that define the automatic focusing process based on at least one of an F-number and a focal depth of the scope estimated based on the detection result by the detection unit.

9. The information processing device according to claim 1, 6, 7, or 8.

10. The control unit sets parameters that define the automatic focusing process based on a result of reference to a table that includes at least one of an F-number or a focal depth of the scope stored in advance, based on a result of detection by the detection unit.

9. The information processing device according to claim 1, 6, 7, or 8.

11. A method for operating an information processing device, comprising: a detection unit that calculates a first evaluation value by detecting within a detection frame set in an image captured by the endoscope; a detection unit detecting an effective area of ​​the captured image, which is an area free from vignetting caused by a scope of the endoscope, based on the first evaluation value; The control unit executes an automatic focusing process based on a second evaluation value for an evaluation value calculation target area set within the effective area. How it works.

12. On the computer, calculating a first evaluation value by detecting within a detection frame set on an image captured by the endoscope; detecting an effective area of ​​the captured image, which is an area free from vignetting caused by a scope of the endoscope, based on the first evaluation value; An automatic focusing process is performed based on a second evaluation value that targets an evaluation value calculation target area set within the effective area. A program that executes a process.

13. a light source device that irradiates light onto a surgical field; an endoscope that captures an image of the surgical field; an information processing device connected to the endoscope and the light source device, a detection unit that performs detection within a detection frame set in the captured image of the endoscope and calculates a first evaluation value; a detection unit that detects an effective area of ​​the captured image, which is an area free from vignetting caused by a scope of the endoscope, based on the first evaluation value; a control unit that executes automatic focusing processing based on a second evaluation value targeted at an evaluation value calculation target area set within the effective area; Equipped with the information processing device; An endoscopy system comprising:

Images