Dynamic region of interest (ROI) based auto exposure control

Abstract

The present disclosure provides methods, apparatuses, and systems for exposure control of digital images. The present implementations more specifically relate to dynamic region of interest (ROI) based automatic exposure control. In some implementations, an imaging system can capture a first image in a series of images using a first exposure setting. The imaging system can detect one or more regions of interest (ROIs) in the first image. The imaging system can determine a first brightness value associated with a first ROI in the first image. The imaging system can determine a second exposure setting based on the first brightness value. The imaging system can capture a second image in the series of images immediately following the first image using the second exposure setting.

Description

[0001] Cross-reference to related applications This application claims the benefit of U.S. Provisional Application No. 63 / 744604 (titled “DYNAMIC REGION-OF-INTEREST(ROI)-BASED AUTO-EXPOSURE CONTROL”, filed January 13, 2025), which is incorporated herein by reference in its entirety. Technical Field

[0002] This implementation generally involves exposure control for digital images, and specifically involves automatic exposure control based on dynamic regions of interest (ROI). Background Technology

[0003] Computer vision is the field of artificial intelligence (AI) that mimics the human visual system to derive inferences about the environment from images or videos of it. Example computer vision techniques include object detection, object classification, object recognition, and object tracking, among others. Object recognition encompasses a variety of techniques for identifying specific objects detected in an image (e.g., facial recognition for identifying a specific person).

[0004] In some instances, image brightness can affect the accuracy of object recognition. For example, an image that is too bright or too dark may blur details on an object that could be useful for object recognition. The brightness of an image captured by an imaging device can be controlled or adjusted via one or more exposure settings, such as exposure time or sensor gain. Some imaging devices have automatic exposure control (also known as “auto-exposure” control) to dynamically adjust the exposure settings based on feedback or hysteresis from previously captured images by the imaging device. However, many existing auto-exposure techniques take the brightness of the captured image as a whole when determining how to adjust the exposure settings used to capture subsequent images. This often results in images that are too bright or too dark in certain areas, such as regions of interest (ROIs) containing object details that could be useful for object recognition or other computer vision applications. Summary of the Invention

[0005] This summary is provided to introduce, in a simplified form, the selection of concepts further described in the detailed description below. This summary is not intended to identify key or essential features of the claimed subject matter, nor is it intended to limit the scope of the claimed subject matter.

[0006] An innovative aspect of the subject matter of this disclosure can be implemented in the following method: capturing a first image in a series of images using a first exposure setting; detecting one or more regions of interest (ROIs) in the first image; determining a first brightness value associated with the first ROI in the first image; determining a second exposure setting based on the first brightness value; and capturing a second image in the series of images immediately following the first image using the second exposure setting.

[0007] Another innovative aspect of the subject matter of this disclosure can be implemented in a computing system including an imaging sensor, one or more processors, and a memory coupled to the one or more processors. The memory stores instructions that, when executed by the one or more processors, cause the computing system to: capture a first image from a series of images using a first exposure setting; detect one or more regions of interest (ROIs) in the first image; determine a first brightness value associated with the first ROI in the first image; determine a second exposure setting based on the first brightness value; and capture a second image from the series of images immediately following the first image using the second exposure setting. Attached Figure Description

[0008] This implementation is illustrated by way of example and is not intended to be limited to the figures in the accompanying drawings.

[0009] Figure 1 A block diagram of an example imaging system is shown, based on some implementation methods.

[0010] Figure 2 A decision flow diagram is shown, illustrating an example process for dynamic automatic exposure control based on some implementation methods.

[0011] Figure 3 Another block diagram of an example imaging system based on some implementations is shown.

[0012] Figure 4 An illustrative flowchart depicting example operations for dynamic automatic exposure control is shown, based on some implementation methods. Detailed Implementation

[0013] In the following description, numerous specific details (such as examples of specific components, circuits, and processes) are set forth to provide a thorough understanding of this disclosure. As used herein, the term “coupled” means a direct connection to or through one or more intermediate components or circuits. The terms “electronic system” and “electronic device” can be used interchangeably to refer to any system capable of electronically processing information. Furthermore, in the following description, and for purposes of explanation, specific terminology is set forth to provide a thorough understanding of aspects of this disclosure. However, it will be apparent to those skilled in the art that these specific details are not required to practice the exemplary embodiments. In other instances, well-known circuits and devices are shown in block diagram form to avoid obscuring this disclosure. Some portions of the subsequent detailed description are presented in accordance with other notations of procedures, logic blocks, processes, and operations on data bits within computer memory.

[0014] These descriptions and representations are the means by which those skilled in the art of data processing most effectively communicate the substance of their work to others skilled in the art. In this disclosure, procedures, logic blocks, processes, etc., are considered as a self-consistent sequence of steps or instructions that lead to a desired result. These steps are those that require physical manipulation of physical quantities. Typically, although not always, these quantities take the form of electrical or magnetic signals that can be stored, transferred, combined, compared, and otherwise manipulated in a computer system. However, it should be remembered that all terms in these and similar terms are to be associated with appropriate physical quantities and are merely convenient notations applied to those quantities.

[0015] Unless otherwise specifically stated, as will be apparent from the following discussion, throughout this application, the use of terms such as “access,” “receive,” “send,” “use,” “select,” “determine,” “normalize,” “multiply,” “average,” “monitor,” “compare,” “apply,” “update,” “measure,” “derive,” etc., relates to the actions and processes of a computer system or similar electronic computing device that manipulates data represented as physical (electronic) quantities in the registers and memories of the computer system and transforms it into other data similarly represented as physical quantities in the computer system’s memory or registers or other such information storage, transmission, or display devices.

[0016] In the accompanying drawings, a single block may be described as performing one or more functions; however, in practice, the one or more functions performed by that block may be performed in a single component or across multiple components, and / or may be performed using hardware, software, or a combination of hardware and software. To clearly illustrate this interchangeability of hardware and software, various illustrative components, blocks, modules, circuits, and steps are generally described below according to their functionality. Whether such functionality is implemented as hardware or software depends on the specific application and design constraints imposed on the overall system. Those skilled in the art may implement the described functionality in different ways for each specific application, but such implementation decisions should not be construed as causing a departure from the scope of this disclosure. Furthermore, example input devices may include components different from those shown, including known components such as processors, memory, etc.

[0017] The techniques described herein can be implemented in hardware, software, firmware, or any combination thereof, unless specifically described as being implemented in a particular manner. Any feature described as a module or component may also be implemented together as an integrated logic device or separately as discrete but interoperable logic devices. If implemented in software, the techniques may be implemented at least in part by a non-transitory processor-readable storage medium comprising instructions that, when executed, perform one or more of the methods described above. The non-transitory processor-readable data storage medium may form part of a computer program product, which may include encapsulation material.

[0018] Non-transitory processor-readable storage media may include random access memory (RAM) such as synchronous dynamic random access memory (SDRAM), read-only memory (ROM), non-volatile random access memory (NVRAM), electrically erasable programmable read-only memory (EEPROM), flash memory, and other known storage media. Alternatively or additionally, the technology may be implemented, at least in part, by a processor-readable communication medium that carries or transmits code in the form of instructions or data structures and is accessible, read, and / or executed by a computer or other processor.

[0019] The various illustrative logic blocks, modules, circuits, and instructions described in conjunction with the embodiments disclosed herein can be executed by one or more processors (or processing systems). As used herein, the term "processor" can mean any general-purpose processor, special-purpose processor, conventional processor, controller, microcontroller, and / or state machine capable of executing scripts or instructions of one or more software programs stored in memory.

[0020] As described above, computer vision techniques can include object recognition, which can be used to identify specific objects in an image. Object recognition techniques can identify objects based on visual details on objects in an image. The brightness (also known as luminance) of an image can affect the accuracy of object recognition. Images that are too bright or too dark may blur details on objects in the image. The brightness of an image captured by an imaging device can be controlled or adjusted via one or more exposure settings. Imaging devices can implement automatic exposure control to dynamically adjust exposure settings based on feedback or hysteresis from images previously captured by the imaging device. However, many existing automatic exposure controls consider the brightness of the captured image as a whole when determining how to adjust the exposure settings used to capture subsequent images. This disclosure recognizes that such methods may produce images that are too bright or too dark in certain areas, such as regions of interest (ROIs) containing object details that may be useful for object recognition or other computer vision applications. Therefore, in some aspects, automatic exposure control can dynamically adjust exposure settings based on whether the image includes at least one ROI.

[0021] Various aspects of this disclosure generally relate to automatic exposure control, and more specifically to automatic exposure control based on dynamic ROIs. In some aspects, an imaging system may be configured to use a first exposure setting to capture a first image in a series of images; detect one or more regions of interest (ROIs) in the first image; determine a first brightness value associated with the first ROI in the first image; determine a second exposure setting based on the first brightness value; and use the second exposure setting to capture a second image in the series of images immediately following the first image.

[0022] Specific implementations of the subject matter described in this disclosure can be implemented to achieve one or more of the following potential advantages. By determining the exposure settings based on the brightness of the ROI in the image, aspects of this disclosure can capture images with enhanced brightness within the ROI. By enhancing the brightness within a specific ROI (such as by increasing or decreasing the brightness to a target or desired level), aspects of this disclosure can improve the accuracy and reliability of object recognition and / or other computer vision techniques.

[0023] Figure 1 A block diagram of an example imaging system 100 according to some implementations is shown. In some aspects, the imaging system 100 may be configured to detect one or more regions of interest (ROIs) and to adjust one or more exposure settings based on the detected ROIs.

[0024] Imaging system 100 includes an image capture component 110, an image analysis component 120, and an exposure control component 130. The image capture component 110 can be any imaging sensor or device (such as a camera) configured to capture a pattern of light in its field of view (FOV) 112 and convert the light pattern into a digital image (e.g., image 114). For example, digital image 114 may include an array of pixels (or pixel values) representing the pattern of light in the FOV 112 of image capture component 110. In some implementations, image capture component 110 may continuously (or periodically) capture a series of images representing digital video. Figure 1 In the example, the object of interest 102 located within FOV 112 is depicted as a person. As a result, image 114 may include object of interest 102. If FOV 112 includes multiple objects of interest, the resulting captured image may accordingly include multiple objects of interest.

[0025] Image capture assembly 110 uses one or more exposure settings to capture image 114. As used herein, exposure settings are settings or configurations of image capture assembly 110 that affect the amount of light reaching the imaging sensor in image capture assembly 110 and / or the amplification of signals generated by the imaging sensor, thereby affecting the brightness of the resulting image. Examples of exposure settings used herein include exposure time (which may also be referred to as “shutter speed”), aperture, and gain (which may also be referred to as “sensor gain”). In some implementations, the gain may be analog gain, digital gain, or color gain. Depending on various conditions in FOV 112 (e.g., lighting conditions, object position) and exposure settings, different areas of image 114 may have varying degrees of brightness.

[0026] In some aspects, image analysis component 120 may detect one or more ROIs 116 corresponding to a corresponding object of interest in image 114 based on object detection model 122. Object detection model 122 may be trained or otherwise configured to detect objects of interest in an image or video. For example, object detection model 122 may apply one or more transformations to pixels in image 114 to create one or more features that can be used for object detection. More specifically, object detection model 122 may compare features extracted from image 114 with a known set of features that uniquely identify objects of a particular class (such as humans) to determine the presence or location of any object of interest in image 114. In some implementations, object detection model 122 may be a neural network model. In some other implementations, object detection model 122 may be a statistical model. In some implementations, object detection model 112 may determine a bounding box indicating the corresponding ROI for each detected object of interest in the image. Image analysis component 120 may output ROI information 116. ROI information 116 is data indicating whether an ROI was detected in image 114 or whether an ROI was missing.

[0027] In some implementations, ROI information 116 may include an annotation image comprising one or more bounding boxes corresponding to the ROIs detected in image 114. In some implementations, ROI information 116 may include coordinates (e.g., xy coordinates in the coordinate space of the image capture component 110 and / or the digital image 114) defining two or more angles of each ROI bounding box. It should be understood that while in some implementations an ROI may be represented by a rectangular bounding box, in other implementations an ROI may be represented by a boundary region of any shape. That is, an ROI does not necessarily have to be rectangular. For example, in some implementations, the boundary region indicating the corresponding ROI may have a circular, elliptical, square, or even irregular shape (e.g., a shape whose boundary outlines the contour of an object of interest in the image). Furthermore, in some implementations, an ROI may be a segment of an object of interest in the image.

[0028] In some implementations, the image 114 captured by the image capture component 110 and / or the ROI detected by the image analysis component 120 can be provided as input to another system or component for further processing or analysis. For example, image 114 and ROI 116 can be provided to an object recognition component to identify a specific object of interest 102 within ROI 116 (e.g., identifying a specific person). Visual details in image 114 (and particularly visual details associated with object of interest 102 in image 114) can be useful information for object recognition and other computer vision applications applied to image 114. This disclosure recognizes that if image 114 is too bright or too dark within ROI 116, the visual details associated with object of interest 102 may be blurred (e.g., washed out by brightness, too dark to be detected). As a result, potentially useful information for object recognition and / or other computer vision applications may be lost or unavailable.

[0029] Exposure control component 130 is configured to control one or more exposure settings 118 of image capture component 110 based at least in part on ROI 116. More specifically, exposure control component 130 can adjust exposure settings 118 such that the brightness of ROI 116 in a subsequent image captured by image capture device 110 is within a threshold or target range and / or converges toward a target. Exposure control component 130 includes brightness determination component 132 and exposure determination component 134. Exposure control component 130 can receive ROI 116 from image analysis component 120 and can also receive image 114 from image capture component 110. Brightness determination component 132 can be configured to determine the brightness of image 114. As used herein, brightness (or luminance) for an image refers to the perceived intensity of light in the image (e.g., the perceived intensity of light radiated or reflected by a scene or by objects in the image). The brightness of an image can be expressed as a numerical value. Exposure determination component 134 can be configured to calculate one or more exposure settings 118 to be implemented by image capture device 110.

[0030] In some implementations, the brightness determination component 132 may determine the actual brightness 136 of the image 114 based on the ROI information 116 received from the image analysis component 120 and the image 114 received from the image capture component 110. If the ROI information 116 indicates that at least one ROI is detected in the image 114, the brightness determination component 132 may calculate the brightness value of the ROI in the image 114.

[0031] In some implementations, luminance 136 is expressed as a luminance (luma) value. Therefore, the luminance determination component 132 can determine the actual luminance value of image 114 as a whole or the actual luminance value of the ROI within image 114. In some implementations, the luminance determination component 132 first calculates the corresponding luminance value Y for one or more pixels in the image. Assuming image 114 is in RGB format, the pixel values ​​used for calculation... Y pixel The equation for the luminance value is given below as Equation 1: (Equation 1) in R pixel , G pixel and B pixel These are the R, G, and B values ​​of a pixel, respectively. In some implementations, Equation 1... R pixel , G pixel and B pixel The coefficients can vary depending on the standard followed by image 114. For example, the coefficients in Equation 1 above apply to images following the ITU BT.709 standard. For images following the CCIR 601 standard, the coefficients can be different, as shown in Equation 2 below: (Equation 2) If the captured image 114 has not yet been converted to RGB format (e.g., image 114 has not yet been de-mosaiced), then Y pixel They can be calculated differently. For example, for Bayer color filter image data, it is used to calculate... Y pixel The equation is given below as Equation 3: (Equation 3) in G pixel It is the G value of the pixel.

[0032] Brightness determination component 132 can calculate the brightness value according to Equation 4: (Equation 4) If image 114 includes a Region of Interest (ROI) (e.g., ROI information 116 indicates a single detected ROI), then the brightness determination component 132 uses only the pixels within the ROI. Y pixel Value to calculate LumaIn other words, the calculated brightness value is the brightness value of the ROI. If image 114 includes two or more ROIs (e.g., ROI information 116 indicates multiple detected ROIs), then the brightness determination component 132 uses only the pixels from the ROI with the largest value among the two or more ROIs. Y pixel Value to calculate Luma Alternatively, in some implementations, the brightness determination component 132 can use the pixels from all ROIs in two or more ROIs. Y pixel Value to calculate Luma .

[0033] If image 114 does not include a ROI (e.g., ROI information 116 indicates an undetected ROI), then brightness determination component 132 can use data from the entire image. Y pixel Value (i.e., the value of all pixels from the image's pixels) Y pixel To calculate (value) Luma In other words, the calculated Luma This is the brightness value of the image as a whole. Alternatively, in some implementations, the brightness determination component 132 can use a value from a portion of the image (e.g., a portion of the image far from the light source). Y pixel The brightness is calculated using values. For example, if the image includes a light source at the top of the image, the brightness determining component 132 can use values ​​from the bottom of the image (e.g., the bottom two-thirds). Y pixel Value to calculate Luma .

[0034] Exposure determination component 134 is configured to determine one or more exposure settings 118 to be used by image capture device 110 to capture a subsequent image. Exposure control component 130 provides the determined exposure settings(s) 118 to image capture component 110, which can then use the determined exposure settings to capture a subsequent image. This subsequent image can be the basis for a subsequent determination of the exposure settings, similar to image 114 described above.

[0035] In some implementations, the exposure determination component 134 may calculate the exposure setting 118 based on the brightness 136 of the image 114 determined by the brightness determination component 132, the current exposure setting used by the image capture component 110 to capture the image 114, and a predetermined set of values ​​and / or constants. In some implementations, the exposure determination component 134 may calculate the exposure setting 118 according to an algorithm that converges the actual brightness 136 of the image captured by the image capture component 110 (e.g., the luminance value of the ROI) toward a target brightness (e.g., a target luminance value). As mentioned above, the exposure setting may include exposure time, aperture, and / or gain. Example algorithms and techniques for calculating the exposure setting 118 are described below.

[0036] The exposure determination component 134 determines the exposure time (also known as the "next exposure time") of the next image according to the algorithm shown in Table 1 below. next .

[0037] Table 1 If image 114 includes a region of interest (ROI), then the brightness values ​​shown in Table 1 are... Luma It is the ROI (or the largest ROI among multiple ROIs in image 114). Luma If image 114 does not contain any ROI, then the brightness values ​​shown in Table 1 are... Luma Image 114 as a whole Luma .

[0038] As shown in Table 1, the exposure determination component 134 first... Luma With L MIN and L MAX The defined ranges are compared. If L MIN <= Luma <=L MAX Then the exposure determination component 134 will set the current exposure time E current The exposure time (used to capture image 114) is set to the next exposure time E used to capture subsequent images. next In other words, the next exposure time is the same as the current exposure time. In some implementations, L MIN and L MAX Define a range of brightness values ​​that are empirically determined to be suitable for image processing and analysis (e.g., object recognition).

[0039] if Luma In L MIN and L MAX Outside the defined range, the exposure determination component 134 will... Luma Compare with the highest and lowest possible brightness values ​​of 255 and 0, respectively. If Luma If E is within those values ​​and not equal to those values, then NEXT Calculated as E NEXT =E CURRENT *(L) TARGET / Luma ), where L TARGET Define the target brightness value. Similar to L. MIN and L MAX L TARGET These are target brightness values ​​that have been empirically determined to be suitable for image processing and analysis (e.g., object recognition). In some implementations, they are used to determine E. NEXT The algorithm is designed to adjust exposure settings to bring the luminance values ​​of the ROIs included in the captured image (FOV 112) toward the target luminance value L. TARGET convergence.

[0040] if Luma If E equals the highest possible brightness value of 255, then E NEXT Calculated as E NEXT =E CURRENT / factor .if Luma If E equals the lowest possible brightness value of 0, then E NEXT Calculated as E NEXT =E CURRENT * factor ,in factor It is a predetermined scalar or constant that can be used to adjust E. NEXT In order to make Luma Converges towards the target brightness value. Alternatively, in some implementations, if image 114 is saturated or overexposed (e.g., image 114's brightness value is too high), the brightness value will be lowered. Luma Equal to or exceeding a predetermined threshold, wherein the predetermined threshold is higher than L MAX But it may be less than the highest possible brightness value), then E NEXT It can be calculated as E NEXT =E CURRENT / factor Similarly, in some implementations, if image 114 is underexposed (e.g., image 114's exposure is insufficient), Luma Equal to or below a predetermined threshold, wherein the predetermined threshold is lower than L MIN But it may be greater than the lowest possible brightness value), then E NEXT It can be calculated as E NEXT =E CURRENT * factor .

[0041] As pointed out above, L MIN L MAX L TARGET and factorThese are predefined parameters. In some implementations, one or more of these parameters are determined and set empirically. Table 2 below provides an example set of values ​​for these predefined parameters: Table 2 Exposure determination component 134 can also be based on the gain G of the current image. CURRENT To calculate the gain G of the next image NEXT For example, by repeating the above algorithm and using E respectively NEXT and E CURRENT Replace G NEXT and G CURRENT Exposure control component 130 will E NEXT and / or G NEXT Exposure setting 118 is provided to image capture component 110. Image capture component 110 uses E... NEXT and G NEXT To capture a new image, which can be processed by the image analysis component 120 and used to determine the exposure setting 118 to be used by the exposure control component 130 to capture the next image in a series of images, as described above.

[0042] Figure 2 A decision flowchart for an example process 200 for dynamic automatic exposure control, according to some implementation methods, is shown. Process 200 illustrates the decision flow by which the imaging system 100 determines the exposure settings for capturing the next image.

[0043] Process 200 begins at step 202, where image capture component 110 captures an image (e.g., image 114). In step 204, brightness determination component 132 calculates the brightness of the captured image as a whole. In step 206, image analysis component 120 may perform object detection on the captured image to detect Regions of Interest (ROIs) in the image, thereby generating ROI information (e.g., ROI information 116). Therefore, in process 200, brightness determination component 132 may calculate the brightness of the image as a whole (e.g., using the brightness of pixels from the entire image) before or simultaneously with object detection to detect ROIs in the image. Y pixel The brightness value is determined regardless of whether the image includes a ROI. Alternatively, the brightness determination component 132 may determine the brightness of the image as a whole after step 206, as described below.

[0044] In step 208, exposure control component 130 determines whether at least one ROI is detected in the image (e.g., based on ROI information 116 received from image analysis component 120). If exposure control component 130 determines that at least one ROI is detected in the image (208 – yes), process 200 proceeds to step 210, where exposure control component 130 (e.g., brightness determination component 132) calculates a brightness value (e.g., ROI luminance value) for the ROI in the image, which can be provided to exposure determination component 134 as brightness 136. Process 200 then proceeds to step 212.

[0045] If the exposure control component 130 determines that no ROI is detected in the image (208 – No), process 200 proceeds to step 212. In some implementations, the brightness determination component 132 may calculate the brightness value of the image as a whole between steps 208 and 212. This brightness value of the image as a whole may be provided to the exposure determination component 134 as brightness 136.

[0046] In step 212, the exposure determination component 134 calculates the exposure settings for subsequent image capture based on the image brightness 136. As described above, the brightness 136 provided to the exposure determination component 134 can be the brightness of the ROI in the image or the brightness of the image as a whole.

[0047] In some implementations, at optional step 214, imaging system 100 may skip an image. That is, imaging system 100 may skip capturing an immediately following subsequent image and instead capture another subsequent image using the calculated exposure settings. Regardless of whether process 200 includes step 214, process 200 then returns to step 202 with the calculated exposure settings, where image capture component 110 uses the calculated exposure settings to capture a new image. In some implementations, skipping an image in step 214 may allow the exposure settings to stabilize.

[0048] In some implementations, in step 212, the exposure determination component 134 calculates the exposure settings according to the algorithm described above with reference to Table 1. In other implementations, the exposure determination component 134 calculates the exposure settings according to another algorithm. In particular, this algorithm can use brightness values ​​from at least two captured images (the current image and a previous image (e.g., the image captured immediately before or even earlier)) to calculate the exposure settings for capturing the next image, such that the brightness values ​​converge toward the target. The algorithm calculates the exposure settings according to the following equations 5-9: E 1f = ((L T – L1) / (L T + L1)) + 1 (Equation 5) E 2f = ((L T – L2) / (L T + L2)) + 1 (Equation 6) E 1T = E 1f * E1 (Equation 7) E 2T = E 2f * E2 (Equation 8) E3 = (E 1T +E 2T ) / 2 (Equation 9) In Equation 5-9, E1 and E2 are the exposure settings used to capture the current image and the previous image, respectively. E2 is the exposure setting calculated for capturing subsequent images. T This is the target brightness value, similar to L in Table 1 above. TARGET L1 and L2 are the brightness values ​​of the current image and the previous image, respectively. As described above with reference to Equations 1-4, each of L1 and L2 can be the brightness value of the ROI or the brightness value of the image as a whole (depending on whether the image includes the ROI). 1f and E 2f These are the exposure setting adjustment factors for E1 and E2, respectively. Therefore, in Equation 5-9, the exposure determination component 134 calculates the adjusted exposure setting E for the current image based on the brightness value of the current image and the target brightness value. 1T And, based on the brightness values ​​of the previous image and the target brightness value, calculate the adjusted exposure setting E for the previous image. 2T Exposure determines component 134, then E is calculated. 1T and E 2T The average value is used to obtain the exposure setting E3 for capturing subsequent images. Therefore, Equation 5-9 calculates E3 according to the sliding window method, where the values ​​(e.g., brightness value, exposure setting) of the current image and the previous image (e.g., the image immediately in front) are used.

[0049] In some implementations, equation 5-9 can be simplified to equation 10 below: (Equation 10).

[0050] In some implementations, for the initial two captured images in process 200, E1 and E2 can be set to predetermined values ​​(e.g., maximum and minimum exposure time or gain, respectively). For example, if E3 is the gain for subsequent image captures, and E1 and E2 are the current gain and the previous gain, respectively, then predetermined gains can be set for the initial two images. For example, E1 can be set to the maximum gain. max And E2 can be set to minimum gain. min In some implementations, gain max =79, and gain min =1.

[0051] Furthermore, in some implementations, L T It may vary depending on the image resolution. For example, for a VGA resolution image, L... T It can be pre-set to have a value of 110, while for HD resolution images, L T It can be pre-ordered to have a value of 40.

[0052] Furthermore, in some implementations, if one exposure setting (e.g., exposure time) reaches its maximum or minimum value while another exposure setting (e.g., gain) remains constant, the same equation can be used to calculate the value of the other exposure setting while keeping the exposure setting that has reached its maximum or minimum value constant. For example, if the exposure time reaches its maximum value via process 200 while the gain remains constant, the exposure control component 130 can keep the exposure time constant (e.g., at its maximum value) and continue calculating the gain value via process 200.

[0053] Figure 3 Another block diagram of an example imaging system 300 according to some implementations is shown. More specifically, the imaging system 300 can be configured to detect one or more regions of interest (ROIs) in an image. Furthermore, the imaging system 300 can be configured to determine an exposure setting based on the detected ROIs and use that exposure setting to capture subsequent images. In some implementations, the imaging system 300 may be... Figure 1 An example of an imaging system 100. Imaging system 300 includes device interface 310, processing system 320, and memory 330.

[0054] Device interface 310 is configured to interface with one or more components of the image capture device (such as...) Figure 1 The device interface 310 communicates with the image capture component 110. In some implementations, the device interface 310 may include an image sensor interface (I / F) 312 configured to receive images via the image capture device. In some implementations, the image sensor interface 312 may use exposure settings to capture images.

[0055] Memory 330 may include data storage 331 configured to store one or more models for object detection, and data storage 332 configured to store one or more received images and output data of object detection of the images (including, for example, ROI information). Memory 330 may also include non-transitory computer-readable media (including one or more non-volatile memory elements, such as EPROM, EEPROM, flash memory, or hard disk drives, and other examples), which may store at least the following software (SW) modules: • Object detection SW module 334, used to detect one or more objects based on the first image and determine each corresponding ROI in the first image; • Brightness determination SW module 336, used to determine the brightness of the image; and • Exposure determination SW module 738, used to determine exposure settings based on brightness.

[0056] Each software module includes instructions that, when executed by the processing system 320, cause the imaging system 300 to perform a corresponding function.

[0057] The processing system 320 may include any suitable one or more processors capable of executing scripts or instructions of one or more software programs stored in the imaging system 300 (such as in memory 330). For example, the processing system 320 may execute an object detection SW module 334 to determine one or more ROIs in a first image, execute a brightness determination SW module 336 to determine the brightness of an image, and execute an exposure determination SW module 338 to determine an exposure setting based on the brightness.

[0058] Figure 4 An illustrative flowchart depicting an example operation 400 for object detection according to some implementations is shown. In some implementations, example operation 400 may be performed by, for example... Figure 1 The imaging system 100 is used to perform the imaging.

[0059] The imaging system can use a first exposure setting to capture a first image in a series of images (402). The imaging system can detect one or more regions of interest (ROIs) in the first image (404). The imaging system can determine a first brightness value associated with the first ROI in the first image (406). The imaging system can determine a second exposure setting based on the first brightness value (408). The imaging system can use the second exposure setting to capture a second image immediately following the first image in the series of images (410).

[0060] In some aspects, the imaging system can calculate the average brightness value of the first ROI.

[0061] In some aspects, the imaging system can compare a first brightness value with a predetermined range of brightness values.

[0062] In some aspects, the imaging system can set the first exposure setting as the second exposure setting based on the determination that the first brightness value is within a predetermined range of brightness values.

[0063] In some aspects, the imaging system can determine the second exposure setting based on the first brightness value and the target brightness value.

[0064] In some aspects, the imaging system can further determine a second exposure setting based on a second brightness value associated with a third image and a third exposure setting in the series of images, the third image being captured using the third exposure setting before the first image.

[0065] In some aspects, the imaging system may calculate a first exposure adjustment factor based on a first luminance value and a target luminance value; calculate a first target exposure setting based on a first exposure setting and the first exposure adjustment factor; calculate a second exposure adjustment factor based on a second luminance value and a target luminance value; calculate a second target exposure setting based on a third exposure setting and the second exposure adjustment factor; and calculate a second exposure setting based on the first target exposure setting and the second target exposure setting.

[0066] In some aspects, the imaging system can calculate the sum of the first target exposure settings and the second target exposure settings.

[0067] In some aspects, the imaging system can detect multiple regions of interest (ROIs) in a first image, and the first ROI has the largest area among the multiple ROIs.

[0068] In some aspects, the imaging system may determine a second brightness value associated with the first image; determine a third exposure setting based on the second brightness value; and use the third exposure setting to capture a third image in the series of images that immediately follows the first image.

[0069] Those skilled in the art will recognize that information and signals can be represented using any of a wide variety of different techniques and arts. For example, data, instructions, commands, information, signals, bits, symbols, and chips, which can be referenced throughout the description above, can be represented by voltage, current, electromagnetic waves, magnetic fields or particles, optical fields or particles, or any combination thereof.

[0070] Furthermore, those skilled in the art will recognize that the various illustrative logic blocks, modules, circuits, and algorithmic steps described in conjunction with the aspects disclosed herein can be implemented as electronic hardware, computer software, or a combination of both. To clearly illustrate this interchangeability between hardware and software, various illustrative components, blocks, modules, circuits, and steps have been generally described above in terms of their functionality. Whether such functionality is implemented as hardware or software depends on the specific application and the design constraints imposed on the overall system. Those skilled in the art may implement the described functionality in different ways for each specific application, but such implementation decisions should not be construed as causing a departure from the scope of this disclosure.

[0071] The methods, sequences, or algorithms described in conjunction with the aspects disclosed herein may be implemented directly in hardware, as a software module executed by a processor, or a combination of both. The software module may reside in RAM memory, flash memory, ROM memory, EPROM memory, EEPROM memory, registers, hard disks, removable disks, CD-ROMs, or any other form of storage medium known in the art. An exemplary storage medium is coupled to a processor, enabling the processor to read information from and write information to the storage medium. Alternatively, the storage medium may be integrated with the processor.

[0072] In the foregoing description, embodiments have been described with reference to specific examples thereof. However, it will be apparent that various modifications and changes may be made therein without departing from the broader scope of this disclosure as set forth in the appended claims. Accordingly, the description and drawings are to be viewed in an illustrative rather than a limiting sense.

Claims

1. A method comprising: Use the first exposure setting to capture the first image in a series of images; Detect one or more regions of interest (ROIs) in the first image; Determine a first brightness value associated with a first ROI in the first image; The second exposure setting is determined based on the first brightness value; as well as Capture the second image in the series that immediately follows the first image using the second exposure setting.

2. The method as described in claim 1, wherein, Determining the first brightness value includes calculating the average brightness value of the first ROI.

3. The method as described in claim 1, wherein, Determining the second exposure setting includes comparing the first brightness value with a predetermined range of brightness values.

4. The method of claim 3, wherein, Determining the second exposure setting also includes: Based on the determination that the first brightness value is within the predetermined range of brightness values, the first exposure setting is set to the second exposure setting.

5. The method of claim 1, wherein, Determining the second exposure setting includes determining the second exposure setting based on the first brightness value and the target brightness value.

6. The method of claim 5, wherein, Determining the second exposure setting includes further determining the second exposure setting based on a second brightness value associated with a third image in the series of images and a third exposure setting, the third image being captured before the first image using the third exposure setting.

7. The method of claim 6, wherein, Determining the second exposure settings includes: The first exposure adjustment factor is calculated based on the first brightness value and the target brightness value; The first target exposure setting is calculated based on the first exposure setting and the first exposure adjustment factor; The second exposure adjustment factor is calculated based on the second brightness value and the target brightness value; The second target exposure setting is calculated based on the third exposure setting and the second exposure adjustment factor; and The second exposure setting is calculated based on the first target exposure setting and the second target exposure setting.

8. The method of claim 7, wherein, Calculating the second exposure setting includes calculating the sum of the first target exposure setting and the second target exposure setting.

9. The method of claim 1, wherein, Detecting one or more regions of interest (ROIs) in the first image includes detecting multiple ROIs in the first image, wherein the first ROI has the largest area among the multiple ROIs.

10. The method of claim 1, further comprising: Determine a second brightness value associated with the first image; The third exposure setting is determined based on the second brightness value; as well as Use the third exposure setting to capture the third image in the series that immediately follows the first image.

11. A computing system, comprising: Imaging sensor; One or more processors; as well as A memory coupled to the one or more processors, the memory storing instructions that, when executed by the one or more processors, cause the computing system to: Use the first exposure setting to capture the first image in a series of images; Detect one or more regions of interest (ROIs) in the first image; Determine a first brightness value associated with a first ROI in the first image; The second exposure setting is determined based on the first brightness value; as well as Capture the second image in the series that immediately follows the first image using the second exposure setting.

12. The computing system of claim 11, wherein, When executed by the one or more processors, the instruction causes the computing system to calculate the average brightness value of the first ROI.

13. The computing system of claim 11, wherein, When executed by the one or more processors, the instruction causes the computing system to compare the first brightness value with a predetermined range of brightness values.

14. The computing system of claim 13, wherein, When the instruction is executed by the one or more processors, the computing device sets the first exposure setting to the second exposure setting based on the determination that the first brightness value is within the predetermined range of the brightness value.

15. The computing system of claim 11, wherein, When executed by the one or more processors, the instruction causes the computing device to determine the second exposure setting based on the first brightness value and the target brightness value.

16. The computing system of claim 15, wherein, When executed by the one or more processors, the instructions cause the computing device to further determine the second exposure setting based on a second brightness value associated with a third image and a third exposure setting in the series of images, the third image being captured before the first image using the third exposure setting.

17. The computing system of claim 16, wherein, When the instructions are executed by the one or more processors, the computing device: The first exposure adjustment factor is calculated based on the first brightness value and the target brightness value; The first target exposure setting is calculated based on the first exposure setting and the first exposure adjustment factor; The second exposure adjustment factor is calculated based on the second brightness value and the target brightness value; The second target exposure setting is calculated based on the third exposure setting and the second exposure adjustment factor; as well as The second exposure setting is calculated based on the first target exposure setting and the second target exposure setting.

18. The computing system of claim 17, wherein, When executed by the one or more processors, the instruction causes the computing device to calculate the sum of the first target exposure setting and the second target exposure setting.

19. The computing system of claim 11, wherein, When executed by the one or more processors, the instruction causes the computing device to detect a plurality of regions of interest (ROIs) in the first image, and the first ROI has the largest area among the plurality of ROIs.

20. The computing system of claim 11, wherein, When the instructions are executed by the one or more processors, the computing device: Determine a second brightness value associated with the first image; The third exposure setting is determined based on the second brightness value; and Use the third exposure setting to capture the third image in the series that immediately follows the first image.

Images