Auto Exposure of Region of Interest for Endoscope Cameras

Abstract

1. An endoscopic camera system having a camera for capturing and outputting images, a camera controller coupled to the camera, and a user input device coupled to the camera or the camera controller, the user input device operable to select a region of interest in the image, the region of interest being a sub-portion of the image, and the camera controller calculating a measured luminance value for the region of interest and adjusting exposure in response to a comparison of the measured luminance value with a target luminance value.

Description

[Technical field]

[0001] The present disclosure relates to devices used in endoscopic surgery, and more particularly to a system and method for automatic exposure of a region of interest in an endoscopic camera image. [Background technology]

[0002] The wider the field of view (FOV) of an endoscopic optical system, the more visual information is presented to the surgeon. It is difficult to uniformly illuminate the view seen by a wide-field endoscope because the light output tends to fall off towards the periphery of the field of view. Conventional auto-exposure algorithms use the central area of ​​the view as a center point to calculate the brightness. However, in wide-field applications, the user may choose to digitally zoom in or center in an area towards the periphery of the view to get a better view of the area of ​​interest. In that case, given the light and optical characteristics at the edge of the view / scope, the user may experience poor image quality (darker than normal) and poor responsiveness due to the change in light at the periphery that would not adjust the overall view exposure. In addition, the exposure level of the view in the area of ​​interest may be different than in the wide-field case overall. The lack of proper illumination and exposure response can result in images that confuse the surgeon, cause eye fatigue, and lead to misidentification of tissues and anatomical structures.

[0003] A need exists for an improved autoexposure system that remedies the deficiencies of the prior art. Summary of the Invention

[0004] The present disclosure relates to an endoscopic camera system having an auto-exposure system that remedies deficiencies of the prior art by controlling the auto-exposure based on a region of interest selected by a user. In one implementation, the endoscopic camera system includes a camera that captures and outputs an image, a camera controller coupled to the camera, and a user input device coupled to the camera or the camera controller. The user input device is operable to select a region of interest of the image, the region of interest being a sub-portion of the image. The camera controller calculates a measured luminance value of the region of interest and adjusts the exposure in response to a comparison of the measured luminance value to a target luminance value. In one implementation, the luminance value is a weighted sum of at least one of an average green intensity, an average red intensity, and an average blue intensity of the region of interest. The luminance value may be a weighted sum of the average green intensity of the region of interest. Adjusting the exposure may include adjusting at least one of an exposure time, a light source intensity, a gain, a sensitivity, and a variable aperture.

[0005] Optionally, the camera has a longitudinal axis and captures images at a non-zero angle relative to the longitudinal axis. The camera may capture images at a capture angle of 45 degrees relative to the longitudinal axis. In one implementation, the user input device is operable to select a region of interest having an apparent capture angle that differs from the actual capture angle. The camera may capture and output images having a field of view greater than 90 degrees. In one implementation, the camera captures and outputs images having a field of view of about 140 degrees. The user input device may be operable to select a region of interest having an apparent field of view that is smaller than the image field of view.

[0006] In one implementation, the image has an image center and an orientation indicator, the region of interest has a center of the region of interest positioned a fixed distance from the image center, and the camera controller changes the center of the region of interest based on a change in the position of the orientation indicator. Adjusting the exposure may further include adjusting a gain of at least some of the pixels in the region of interest. Optionally, the gain adjustment is not uniform across all of the pixels in the region of interest. The pixel gain may be adjusted using a gradient depending on the position of the pixel from the center of the image.

[0007] According to one implementation, a method for adjusting exposure of an imaging system includes receiving an image from a camera, receiving a region of interest of the image from a user, where the region of interest is a sub-portion of the image, calculating a measured luminance value for the region of interest, and adjusting exposure in response to comparing the measured luminance value to a target luminance value. Calculating the measured luminance value may further include calculating a weighted sum of an average green intensity, an average red intensity, and an average blue intensity for the region of interest. In one implementation, calculating the measured luminance value further includes calculating a weighted sum of an average green intensity for the region of interest. Adjusting the exposure may further include adjusting at least one of an exposure time, a light source intensity, a gain, a sensitivity, and a variable aperture.

[0008] The image may be captured at an actual capture angle greater than zero relative to the longitudinal axis, and the received region of interest may have an apparent capture angle that differs from the actual capture angle. In one implementation, the image has a field of view greater than 90 degrees, and the received region of interest has an apparent field of view that is smaller than the image field of view.

[0009] In one implementation, the image further includes an image center and an orientation indicator, and the region of interest has a center of the region of interest located a fixed distance from the image center, and the method further includes calculating a new center of the region of interest based on the change in position of the orientation indicator. Adjusting the exposure may further include adjusting a gain of at least some of the pixels in the region of interest. Optionally, the gain adjustment is not uniform across all of the pixels in the region of interest. Optionally, the pixel gain is adjusted using a gradient depending on the position of the pixel from the center of the image.

[0010] These and other features are described below. [Brief description of the drawings]

[0011] The features, aspects, and advantages of the present invention will become better understood with regard to the following description, appended claims, and accompanying drawings. [Figure 1] FIG. 1 is a schematic diagram of an endoscopic camera system according to one implementation. [Diagram 2] FIG. 2 is a schematic diagram of an endoscopic camera system according to an additional implementation. [Diagram 3] FIG. 3 is a schematic diagram of a distal portion of a camera according to one implementation. [Figure 4] FIG. 4 shows an example of a region of interest. [Diagram 5] FIG. 5 is a flow chart illustrating a method for auto-exposure according to one implementation. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0012] In the following description of the preferred implementations, reference is made to the accompanying drawings, which show, by way of illustration, specific implementations in which the present invention may be practiced. Wherever possible, the same reference numbers will be used throughout the drawings to refer to the same or similar parts. It will be understood that other implementations may be utilized and structural and functional changes may be made without departing from the scope of the present disclosure.

[0013] 1-3, an endoscopic camera system 10 according to one implementation includes a camera 12. The camera 12 has a shaft axis 14 that is connectable to a handpiece 16. The handpiece 16 may have an input device 18, such as a button, switch, or dial. The handpiece 16 is connectable to a camera controller 20 ("CCU" or "camera controller"). The handpiece 16 and the camera controller 20 may be connected via wires to facilitate data transfer between the camera and the camera controller. The camera 12 and the camera controller 20 may also be connected wirelessly, such as via IEEE 802.11b or IEEE 802.11n or ultra-wideband (UWB), to facilitate data transfer. The camera controller 20 may be connectable to at least one input device 22, such as a mouse, keyboard, touchpad, or touchscreen monitor. In addition, the camera controller 20 may be connectable to a display 24 and a storage device 26, such as for storing images.

[0014] An image sensor 28 may be positioned within shaft 14 and proximal to a distal tip 30 of shaft 12. Image sensor 28 may be, for example, a charge-coupled device (CCD) or a complementary metal-oxide semiconductor (CMOS). An optical lens, such as a wide-angle lens 32, directs light to image sensor 28.

[0015] The position of the image sensor 28 and optical lens may provide a field of view generally along the longitudinal axis 34 of the shaft 14 (approximately a zero degree capture angle relative to the longitudinal axis) such that the image field is directly in front of the distal tip of the shaft. In some implementations, the optical lens may provide an image at a non-zero capture angle relative to the longitudinal axis of the shaft 14. For example, the capture angle may be approximately 30 degrees or approximately 70 degrees relative to the longitudinal axis. As shown in FIG. 3, in one implementation, the optical lens may provide an image along an image axis 48 having an approximately 45 degree capture angle relative to the longitudinal axis 34 of the shaft 14. Additionally, the camera 12 may be coupled to a light source 36. The light source 32 may be internal to the camera 12.

[0016] The light source 36 includes a lamp. The lamp may be, for example, a laser or a semiconductor light source such as an LED for illuminating the field of view. The light source 36 is configured to appropriately illuminate the field of view of the video camera. Furthermore, the generated light as well as the camera sensitivity may extend beyond the visible spectrum. The illumination may be intended to directly excite fluorescence in the target or in a fluorescent substance such as indocyanine green, which is then detected by the camera. For example, the light source 36 may generate illumination in the near infrared (NIR) range and the camera may detect fluorescence at longer IR wavelengths. The illumination and camera sensitivity may extend continuously from UV to NIR or may consist of separate narrow bands.

[0017] 2, the camera controller 20 is preferably a programmable unit that encompasses sufficient processing power to accommodate a wide range of control, user interface, and image acquisition / processing functions. The camera controller 20 has a processor 38 that executes program applications that provide a variety of capabilities. For example, image capture and display capabilities allow for both image capture and display of a live feed of images through a display 24 coupled to the camera controller 20. Captured images may be stored, such as in an internal storage device 40 or an external storage device 26, or may be transmitted to other devices.

[0018] The timing of the video camera must be very accurate and consistent. A processor field programmable gate array (FPGA) 42 may be used to control and process the output from the image sensor 28. Although other controllers may be used, using one or more FPGAs to process the video images allows the system to achieve the precise timing required to generate a standard video output signal. User interface logic and possible external network connections may be performed by software running on the processor 38.

[0019] In one implementation, analog RGB data is sent from image sensor 28 to camera controller 20. The analog RGB data passes through an analog-to-digital converter 44 to processor FPGA 42 where the video is processed. The processed video is then conveyed to a video output, which may include a formatter FPGA 46 where the video is formatted into various display formats. Formatter FPGA 46 may also overlay information onto the video, such as patient information and / or physician information. The formatted video may be converted back to an analog signal for display. The formatted video is sent to display 24 and / or storage device 26. Alternatively, an analog-to-digital converter may be located within the camera head, and digital RGB data is sent from camera head 12 to camera controller 20. In addition, image sensor 28 itself may include an analog-to-digital converter.

[0020] Camera controller 20 may issue commands to camera 12 to adjust its operating characteristics, and camera 12 may send confirmation to camera controller 20 that the camera received the commands. Processor FPGA 42 and / or processor 38 may communicate with a shutter driver in either camera controller 20 or camera 12 to control the exposure duration of image sensor 28. In addition, processor FPGA 42 and / or processor 38 communicate with light source 32 to control the drive current to a lamp of light source 32.

[0021] 3 and 4, the wide angle lens 32 enables a wide angle image 50. The image may have a field of view greater than about 90 degrees, and more preferably greater than about 140 degrees. Using an input device, such as the camera input device 18 or the camera controller input device 22, a user may select a region of interest 52 within the wide angle image 50. The user may select the region of interest 52 as desired, for example, to magnify a portion of the wide angle image 50 or to simulate an apparent capture angle different from the actual capture angle. For example, but not limited to, the user may select an apparent capture angle of 30 degrees, 45 degrees, or 70 degrees.

[0022] For example, as shown in FIG. 3, the camera may be configured to capture wide angle images 50 along an image axis 48 having a capture angle of approximately 45 degrees relative to the longitudinal axis 34. However, a user may select a region of interest 52 to simulate an endoscopic camera capturing images at a capture angle of approximately 30 degrees relative to the longitudinal axis. The region of interest has a center 54 and an area 56. Once the region of interest is identified, the center 54 and the area 56 are used for automatic exposure compensation, as described below. In some cases, the region of interest is circular, and the area 56 is calculated based on a radius of the region of interest. However, the region of interest may have a different shape.

[0023] As shown in Figure 4, as the shaft 14 is rotated from a first position to a second position, the image center 53 does not change, but the region of interest 52 rotates with the shaft. Once the region of interest 52 is selected, the position of the region of interest relative to the image center 53 is known. The orientation of the scope may be obtained from an orientation indicator 58, such as the shape of a stop mask. As can be seen in Figure 4, the orientation indicator 58 rotates with the scope and may be used to track and update the position of the region of interest 52. The exposure of the region of interest is automatically updated as the scope is rotated.

[0024] 5, the camera controller 20 receives an image from the camera 12 (step 60). The camera controller 20 further receives a region of interest identification from the user, the region of interest being a sub-portion of the image (step 62). Once the controller receives the region of interest identification from the user, including the center and area of ​​the region of interest, the camera controller 20 calculates a measured luminance value for the region of interest (step 64). The measured luminance value may be obtained by calculating a weighted sum of at least one of an average green intensity, an average red intensity, and an average blue intensity for the region of interest. In one preferred implementation, the measured luminance value is obtained by calculating a weighted sum of the average green intensity for the region of interest.

[0025] Once the measured brightness value has been calculated, it is compared to the target brightness value (step 66). In one implementation, the target brightness value is adjustable by a user, such as by using camera input device 18 or camera controller input device 22.

[0026] In response to the comparison, the camera controller adjusts the exposure to bring the measured luminance value closer to the target luminance value (step 68). The camera controller may adjust the exposure by adjusting one or more of several variables, depending on the configuration of the camera and how much the measured luminance value differs from the target luminance value. For example, the camera controller 20 may adjust the exposure time of the image sensor 28. If the measured luminance value is lower than the target luminance value, the camera controller 20 may increase the exposure time to increase the measured luminance value. If the measured luminance value is higher than the target luminance value, the camera controller 20 may decrease the exposure time.

[0027] In addition, camera controller 20 may adjust the intensity of light source 36. If the measured brightness value is lower than the target brightness value, camera controller 20 may increase the intensity of the light source 36 to increase the measured brightness value. If the measured brightness value is higher than the target brightness value, camera controller 20 may decrease the intensity of the light source.

[0028] In addition, camera controller 20 may adjust the digital gain applied to the captured image. If the measured brightness value is lower than the target brightness value, camera controller 20 may increase the digital gain applied to the captured image to increase the measured brightness value. If the measured brightness value is higher than the target brightness value, camera controller 20 may decrease the digital gain.

[0029] In addition, if the camera has a variable aperture that controls the amount of light that reaches the image sensor 28, the camera controller may control the variable aperture to change the amount of light that reaches the image sensor. If the measured brightness value is lower than the target brightness value, the camera controller 20 may increase the size of the aperture, allowing more light to reach the image sensor 28 and increasing the measured brightness value. If the measured brightness value is higher than the target brightness value, the camera controller 20 may decrease the size of the aperture, allowing less light to reach the image sensor. In addition, the camera controller 20 may adjust the sensitivity of the image sensor 28.

[0030] The camera controller 20 may need to adjust multiple parameters. For example, if the camera controller 20 has already increased the exposure time to the maximum possible exposure time and the measured brightness value is still lower than the target brightness value, the camera controller may adjust another parameter, such as digital gain, to further increase the measured brightness value.

[0031] In one implementation, known camera spatial illumination characteristics are further taken into account when adjusting exposure, where the exposure compensation is non-uniform across the region of interest. For example, if the camera 12 is known to have reduced light towards the periphery of the wide-angle image, a gradient may be calculated and applied to modify the auto-exposure settings. If the selected region of interest includes pixels near the periphery of the wide-angle image, pixels in the region of interest closer to the periphery may be provided with increased digital gain. In one implementation, the gradient is applied depending on how far each pixel in the region of interest is from the center of the wide-angle image.

[0032] A surgical imaging system and method that completely and effectively overcomes the disadvantages associated with the prior art has been disclosed in the above description and drawings. However, it will be apparent that variations and modifications of the implementations of the present disclosure may be made without departing from the principles of the invention. The presentation of implementations herein is provided for purposes of illustration only, and not limitation, with the true scope and spirit of the invention being indicated by the following claims. The following is an invention described at the beginning of the present application. <Configuration 1> a camera for capturing and outputting images; a camera controller coupled to the camera; a user input device coupled to the camera or the camera controller, the user input device operable to select a region of interest in the image, the region of interest being a sub-portion of the image; The camera controller, calculating a measured luminance value of said region of interest; and adjusting exposure in response to comparing the measured brightness value with a target brightness value. <Configuration 2> 2. The system of claim 1, wherein the luminance value is a weighted sum of at least one of an average green intensity, an average red intensity, and an average blue intensity of the region of interest. <Configuration 3> 2. The system of claim 1, wherein the brightness value is a weighted sum of average green intensities of the region of interest. <Configuration 4> 2. The system of claim 1, wherein adjusting the exposure further comprises adjusting at least one of an exposure time, a light source intensity, a gain, a sensitivity, and a variable aperture. <Component 5> 2. The system of configuration 1, wherein the camera has a longitudinal axis and captures the image at a non-zero angle relative to the longitudinal axis. <Component 6> 6. The system of configuration 5, wherein the camera captures the image at a capture angle of 45 degrees relative to the longitudinal axis. <Component 7> 6. The system of configuration 5, wherein the user input device is operable to select a region of interest having an apparent capture angle that differs from an actual capture angle. <Component 8> 2. The system of configuration 1, wherein the camera captures and outputs images having a field of view greater than 90 degrees. <Component 9> 9. The system of claim 8, wherein the user input device is operable to select a region of interest having an apparent field of view smaller than the image field of view. <Component 10> the image further comprises an image center and an orientation indicator; the region of interest has a center of interest located a fixed distance from the image center; 2. The system of claim 1, wherein the camera controller changes a center of the region of interest based on a change in position of the orientation indicator. <Component 11> 2. The system of claim 1, wherein adjusting the exposure further comprises adjusting a gain of at least some pixels within the region of interest. <Component 12> 12. The system of configuration 11, wherein the gain adjustment is not uniform across all of the pixels in the region of interest. <Component 13> 13. The system of claim 12, wherein pixel gain is adjusted using a gradient depending on the position of the pixel from a center of the image. <Component 14> 1. A method for adjusting exposure in an imaging system, comprising: Receiving an image from a camera; receiving a region of interest of the image from a user, the region of interest being a sub-portion of the image; calculating a measured luminance value of the region of interest; and adjusting exposure in response to comparing the measured luminance value to a target luminance value. <Component 15> 15. The method of claim 14, wherein calculating the measured luminance value further comprises calculating a weighted sum of an average green intensity, an average red intensity, and an average blue intensity of the region of interest. <Component 16> 15. The method of claim 14, wherein calculating the measured luminance value further comprises calculating a weighted sum of average green intensities of the region of interest. <Component 17> 15. The method of claim 14, wherein adjusting the exposure further comprises adjusting at least one of an exposure time, a light source intensity, a gain, a sensitivity, and a variable aperture. <Component 18> 15. The method of claim 14, wherein the image is captured at an actual capture angle greater than zero relative to a longitudinal axis, and the received region of interest has an apparent capture angle that differs from the actual capture angle. <Component 19> 15. The method of claim 14, wherein the image has a field of view greater than 90 degrees and the received region of interest has an apparent field of view smaller than the image field of view. <Component 20> the image further comprises an image center and an orientation indicator; the region of interest has a center of interest located a fixed distance from the image center; 15. The method of claim 14, further comprising calculating a center of a new region of interest based on the change in position of the orientation indicator. <Component 21> 15. The method of claim 14, wherein adjusting the exposure further comprises adjusting a gain of at least some pixels within the region of interest. <Component 22> 21. The method of claim 20, wherein the gain adjustment is not uniform across all of the pixels in the region of interest. <Component 23> 22. The method of claim 21, wherein pixel gain is adjusted using a gradient depending on the position of the pixel from the center of the image.

Claims

1. a camera for capturing and outputting images; a camera controller coupled to the camera; a user input device coupled to the camera or the camera controller, the user input device operable to select a region of interest in the image, the region of interest being a sub-portion of the image; The camera controller, calculating a measured luminance value of said region of interest; and adjusting exposure in response to a comparison of the measured luminance value and a target luminance value, the image further comprises an image center and an orientation indicator; the region of interest has a center of interest located a fixed distance from the image center; An endoscopic camera system, wherein the camera controller changes a center of the region of interest based on a change in position of the orientation indicator.

2. The system of claim 1 , wherein the measured luminance value is a weighted sum of at least one of an average green intensity, an average red intensity, and an average blue intensity of the region of interest.

3. The system of claim 1 , wherein the measured luminance value is a weighted sum of average green intensities of the region of interest.

4. The system of claim 1 , wherein adjusting the exposure further comprises adjusting at least one of an exposure time, a light source intensity, a gain, a sensitivity, and a variable aperture.

5. The system of claim 1 , wherein the camera has a longitudinal axis and captures the images at a non-zero angle relative to the longitudinal axis.

6. The system of claim 5 , wherein the camera captures the images at a capture angle of 45 degrees relative to the longitudinal axis.

7. The system of claim 5 , wherein the user input device is operable to select a region of interest having an apparent capture angle that differs from an actual capture angle.

8. The system of claim 1 , wherein the camera captures and outputs an image field of view having a field of view greater than 90 degrees.

9. The system of claim 8 , wherein the user input device is operable to select a region of interest having an apparent field of view smaller than the image field of view.

10. The system of claim 1 , wherein adjusting the exposure further comprises adjusting a gain of at least some pixels within the region of interest.

11. The system of claim 10 , wherein adjusting the gain comprises adjusting a gain non-uniformly across all of the pixels in the region of interest.

12. The system of claim 11 , wherein pixel gain is adjusted using a gradient depending on the position of the pixel from the center of the image.

13. a camera for capturing and outputting images; a camera controller coupled to the camera; a user input device coupled to the camera or the camera controller, the user input device operable to select a region of interest in the image, the region of interest being a sub-portion of the image, the method comprising: receiving the image from a camera; receiving a region of interest of the image from the user input device, the region of interest being a sub-portion of the image; calculating a measured luminance value of the region of interest; adjusting exposure in response to comparing the measured luminance value to a target luminance value; the image further comprises an image center and an orientation indicator; the region of interest has a center of interest located a fixed distance from the image center; The method of claim 1, wherein the camera controller changes a center of the region of interest based on a change in position of the orientation indicator.

14. The method of claim 13 , wherein calculating the measured luminance value further comprises calculating a weighted sum of an average green intensity, an average red intensity, and an average blue intensity for the region of interest.

15. The method of claim 13 , wherein calculating the measured luminance value further comprises calculating a weighted sum of average green intensities for the region of interest.

16. The method of claim 13 , wherein adjusting the exposure further comprises adjusting at least one of an exposure time, a light source intensity, a gain, a sensitivity, and a variable aperture.

17. The method of claim 13 , wherein the images are captured at an actual capture angle greater than zero relative to a longitudinal axis, and the received region of interest has an apparent capture angle that differs from the actual capture angle.

18. The method of claim 13, wherein receiving the image includes receiving an image field of view having a field of view greater than 90 degrees, and the received region of interest has an apparent field of view smaller than the image field of view.

19. The method of claim 13 , wherein adjusting the exposure further comprises adjusting a gain of at least some pixels within the region of interest.

20. The method of claim 19 , wherein adjusting the gain comprises adjusting the gain non-uniformly across all of the pixels in the region of interest.

21. The method of claim 20, wherein pixel gain is adjusted using a gradient depending on the position of the pixel from the center of the image.

Images