Region of Interest (ROI)-based dynamic automatic exposure control

Abstract

The present invention provides a method, apparatus, and system for controlling the exposure of digital images. [Solution] More specifically, this embodiment relates to region of interest (ROI)-based dynamic automatic exposure control. In some embodiments, the imaging system includes capturing a first image in a series of images using a first exposure setting, detecting one or more regions of interest (ROIs) in the first image, determining a first brightness value for the first ROI in the first image, determining a second exposure setting based on the first brightness value, and capturing a second image following the first image in a series of images using a second exposure setting.

Description

Technical Field

[0001] This application claims the benefit of U.S. Provisional Application No. 63 / 744,604, filed on January 13, 2025, under the title "Region of Interest (ROI)-Based Dynamic Automatic Exposure Control", which is hereby incorporated by reference in its entirety.

[0002] This embodiment relates generally to exposure control of digital images, and more particularly to region of interest (ROI)-based dynamic automatic exposure control.

Background Art

[0003] Computer vision is a field of artificial intelligence (AI) that mimics the human visual system to make inferences about the environment from images and videos of the environment. Examples of computer vision include techniques such as object detection, object classification, object recognition, and object tracking. Object recognition includes various techniques for identifying specific objects detected in an image (e.g., face recognition for identifying a specific person).

[0004] The brightness of an image can, in some cases, affect the accuracy of object recognition. For example, an image that is too bright or too dark may obscure details of objects that could be useful for object recognition. The brightness of an image captured by an imaging device can be controlled or adjusted by one or more exposure settings (e.g., exposure time, sensor gain). Some imaging devices are equipped with automatic exposure control (also referred to as "auto exposure" control), which dynamically adjusts the exposure settings based on feedback or hysteresis from images previously captured by the imaging device. However, many existing automatic exposure techniques consider the brightness of the entire captured image when determining how to adjust the exposure settings for subsequent image capture. As a result, it frequently occurs that the image is too bright or too dark in a specific region, such as a region of interest (ROI) that contains details of objects that could be useful for object recognition and other computer vision applications.

Summary of the Invention

[0005] This abstract is provided in a concise form to introduce the selection of concepts further described below in modes for carrying out the invention. This abstract is not intended to identify any important or essential features of the claimed subject matter, nor is it intended to limit the scope of the claimed subject matter.

[0006] One of the innovative aspects of the subject matter of this disclosure can be implemented in a method that involves capturing a first image in a series of images using a first exposure setting, detecting one or more regions of interest (ROIs) in the first image, determining a first luminance value for the first ROI in the first image, determining a second exposure setting based on the first luminance value, and capturing a second image following the first image in the series of images using the second exposure setting.

[0007] Another innovative aspect of the subject matter of this disclosure can be implemented in a computing system comprising an image sensor, one or more processors, and a memory coupled to the one or more processors. The memory stores instructions, when executed by the one or more processors, to cause the computing system to capture a first image in a series of images using a first exposure setting, to detect one or more regions of interest (ROIs) in the first image, to determine a first brightness value for the first ROI in the first image, to determine a second exposure setting based on the first brightness value, and to capture a second image following the first image in a series of images using a second exposure setting. [Brief explanation of the drawing]

[0008] This embodiment is described by illustration and is not intended to be limited to the form shown in the accompanying drawings.

[0009] [Figure 1] Figure 1 shows a block diagram of an example imaging system according to several embodiments.

[0010] [Figure 2]Figure 2 shows a decision flowchart illustrating an exemplary process for dynamic automatic exposure control according to several embodiments.

[0011] [Figure 3] Figure 3 shows another block diagram of an example of an imaging system according to several embodiments.

[0012] [Figure 4] Figure 4 shows an exemplary flowchart illustrating an example of operation for dynamic automatic exposure control according to several embodiments. [Modes for carrying out the invention]

[0013] The following description includes many specific details, such as examples of specific components, circuits, and processes, in order to fully understand the disclosure. The term “combined” as used herein means directly connected or connected via one or more intervening components or circuits. The terms “electronic system” and “electronic device” may be used synonymously to refer to any system capable of electronically processing information. Furthermore, certain terminology is provided in the following description for explanatory purposes to fully understand the aspects of the disclosure. However, it will be apparent to those skilled in the art that these specific details are not necessarily required to implement the embodiments. In other instances, well-known circuits and devices are shown in block diagram form to avoid obscuring the disclosure. Some parts of the embodiments for carrying out the following inventions are presented in the form of procedures, logical blocks, processes, and other symbolic representations of operations on data bits in computer memory.

[0014] These descriptions and expressions are means used by those skilled in the field of data processing technology to most effectively communicate the nature of the work to others skilled in the field. In this disclosure, procedures, logical blocks, processes, etc., are considered to be self-contained sequences of steps or instructions that produce a desired result. Steps require the physical manipulation of physical quantities. These quantities typically take the form of electrical or magnetic signals that can be stored, transferred, combined, compared, or otherwise manipulated within a computer system. However, it should be noted that all these and similar terms should be associated with the appropriate physical quantities and are merely convenient labels attached to those quantities.

[0015] As will be apparent from the following discussion, unless otherwise specifically stated, throughout this application, descriptions using terms such as “access,” “receive,” “transmit,” “use,” “select,” “determine,” “normalize,” “multiply,” “average,” “monitor,” “compare,” “apply,” “update,” “measure,” and “derive” refer to the operations and processes by which a computer system or similar electronic computing device manipulates and transforms data represented as physical quantities (electronic quantities) in the registers or memory of the computer system, and transforms it into other data similarly represented as physical quantities in the memory, registers, or other information storage, transmission, and display devices of the computer system.

[0016] In the diagrams, a single block may be described as performing one or more functions. However, in actual operation, the functions performed by that block may be performed by a single component, by multiple components, and by hardware, by software, or by a combination of hardware and software. To clearly demonstrate this compatibility between hardware and software, various exemplary components, blocks, modules, circuits, and steps are described below in relation to their functions. Whether such functions are implemented as hardware or software depends on the design constraints imposed on the individual application and the system as a whole. A person skilled in the art may implement the described functions in various ways for each individual application, but such implementation decisions should not be construed as causing a departure from the scope of this disclosure. Also, examples of input devices include well-known components such as processors and memory, and may include components other than those shown.

[0017] Unless otherwise specifically stated, the technologies described herein may be implemented in hardware, software, firmware, or any combination thereof. Any configuration described as a module or component may be implemented as a single integrated logical device or as separate but collaborative logical devices. When implemented in software, the technology may be implemented at least in part by a non-temporary computer-readable storage medium containing instructions that perform one or more of the methods described above at runtime. The non-temporary computer-readable storage medium may form part of a computer program product, which may include packaging materials.

[0018] Non-transient processor-readable storage media may consist of random access memory (RAM), including synchronous dynamic random access memory (SDRAM), read-only memory (ROM), non-volatile random access memory (NVRAM), electrically erasable programmable read-only memory (EEPROM), flash memory, and other known storage media. In addition to or instead of the above, the technology may be at least partially implemented by a processor-readable communication medium that transmits or transmits code in the form of instructions or data structures, and which can be accessed, read, and / or executed by a computer or other processor.

[0019] Various exemplary logic blocks, modules, circuits, and instructions described in connection with the embodiments disclosed herein may be executed by one or more processors (or processing systems). As used herein, the term “processor” may mean any general-purpose processor, application processor, conventional processor, controller, microcontroller, and / or state machine capable of executing scripts or instructions of one or more software programs stored in memory.

[0020] As described above, computer vision technologies may include object recognition, which can be used to identify specific objects within an image. Object recognition technologies may identify objects based on visual details about the objects in the image. The brightness (sometimes called luminance) of an image can affect the accuracy of object recognition. Images that are too bright or too dark can obscure the details of objects within the image. The brightness of an image captured by an imaging device can be controlled or adjusted by one or more exposure settings. Imaging devices may implement automatic exposure control to dynamically adjust exposure settings based on feedback or hysteresis from images previously captured by the device. However, many existing automatic exposure controls consider the overall brightness of the captured image when determining how to adjust the exposure settings for subsequent images. Aspects of this disclosure recognize that such an approach can result in images being too bright or too dark in specific areas, such as regions of interest (ROIs) that contain details of objects that may be useful for object recognition or other computer vision applications. Therefore, in some aspects, automatic exposure control may dynamically adjust exposure settings based on whether the image contains at least one ROI.

[0021] Various aspects of this disclosure relate to automatic exposure control in general, and more specifically to ROI-based dynamic automatic exposure control. In some aspects, the imaging system is configured to capture a first image in a series of images using a first exposure setting, detect one or more regions of interest (ROIs) in the first image, determine a first brightness value for the first ROI in the first image, determine a second exposure setting based on the first brightness value, and capture a second image following the first image in a series of images using a second exposure setting.

[0022] Each embodiment of the subject matter described herein can be implemented to achieve one or more of the following potential advantages: According to aspects of this disclosure, it is possible to capture images with enhanced brightness within ROIs by determining exposure settings based on the brightness of ROIs in the image. By enhancing the brightness within individual ROIs (for example, by increasing or decreasing the brightness to a target or desired level), according to aspects of this disclosure, the accuracy and reliability of object recognition and / or other computer vision technologies can be improved.

[0023] Figure 1 shows a block diagram illustrating an exemplary imaging system 100 according to several embodiments. In some embodiments, the imaging system 100 may be configured to detect one or more regions of interest (ROIs) and adjust one or more exposure settings based on the detected ROIs.

[0024] The imaging system 100 comprises an image acquisition component 110, an image analysis component 120, and an exposure control component 130. The image acquisition component 110 may be any imaging sensor or device (e.g., a camera) configured to capture a pattern of light within a field of view (FOV) 112 and convert that pattern of light into a digital image (e.g., image 114). For example, the digital image 114 may include an array of pixels (or pixel values) representing the pattern of light within the FOV 112 of the image acquisition component 110. In some embodiments, the image acquisition component 110 may capture a series of images representing a digital video in a continuous (or periodic) manner. In the example in Figure 1, the object of interest 102 located within the FOV 112 is depicted as a person. As a result, image 114 may include the object of interest 102. If multiple objects of interest are included in the FOV 112, the captured image may consequently include multiple objects of interest.

[0025] The image capture component 110 captures an image 114 using one or more exposure settings. As used herein, an exposure setting refers to a setting of the image capture component 110 that affects the amount of light reaching the image sensor within the image capture component 110 and / or the amplification of the signal generated by the image sensor, thereby affecting the brightness of the acquired image. Examples of exposure settings used herein include exposure time (sometimes also called "shutter speed"), aperture, and gain (sometimes also called "sensor gain"). Depending on the embodiment, the gain may be an analog gain, a digital gain, or a color gain. Depending on the various conditions within the FOV 112 (e.g., lighting conditions, object position) and the exposure setting, different areas of the image 114 may have different degrees of brightness.

[0026] Depending on the embodiment, the image analysis component 120 may detect one or more regions of interest 116 corresponding to each object of interest in the image 114, based on the object detection model 122. The object detection model 122 may be trained or otherwise configured to detect objects of interest in images or videos. For example, the object detection model 122 may apply one or more transformations to pixels in the image 114 to generate one or more features that can be used for object detection. More specifically, the object detection model 122 may compare the features extracted from the image 114 with a known set of features that uniquely identify a particular object class (e.g., humans) to determine the presence or location of any object of interest in the image 114. In some embodiments, the object detection model 122 may be a neural network model. In other embodiments, the object detection model 122 may be a statistical model. In some embodiments, the object detection model 112 may determine a bounding box indicating the corresponding ROI for each object of interest detected in the image. The image analysis component 120 may output ROI information 116. ROI information 116 is data indicating the presence or absence of ROIs detected in image 114.

[0027] In some embodiments, the ROI information 116 may include an annotated image containing one or more bounding boxes corresponding to ROIs detected in the image 114. In some embodiments, the ROI information 116 may include the coordinates of two or more vertices defining each ROI bounding box (e.g., xy coordinates in the coordinate space of the digital image 114 and / or the image acquisition component 110). It should be noted that in some embodiments, ROIs may be represented by rectangular bounding boxes, while in other embodiments, ROIs may be represented by bounding regions of arbitrary shape. That is, ROIs do not necessarily have to be rectangular. For example, in some embodiments, the bounding region indicating a corresponding ROI may be circular, elliptical, square, or even have an irregular shape (e.g., the boundary may trace the contour of an object in the image). Furthermore, in some embodiments, an ROI may be a division of an object of interest in the image.

[0028] Depending on the embodiment, the image 114 captured by the image acquisition component 110 and / or the ROI detected by the image analysis component 120 may be provided as input to another system or component for further processing or analysis. For example, the image 114 and ROI 116 may be provided to an object recognition component to identify a specific object of interest 102 within ROI 116 (e.g., to identify a specific person). Visual details within the image 114, particularly those related to the object of interest 102 within the image 114, can be useful information for object recognition or other computer vision applications applied to the image 114. Aspects of this disclosure recognize that if the image 114 is too bright or too dark within ROI 116, visual details concerning the object of interest 102 may become unclear (e.g., overexposed due to brightness or too dark to be detected). As a result, information that could be useful for object recognition or other computer vision applications may be lost or unavailable.

[0029] The exposure control component 130 is configured to control one or more exposure settings 118 of the image acquisition component 110, at least in part, based on the ROI 116. More specifically, the exposure control component 130 may adjust the exposure settings 118 so that the luminance of the ROI 116 in the image subsequently captured by the image acquisition device 110 falls within a threshold or target range and / or converges toward a target value. The exposure control component 130 comprises a luminance determination component 132 and an exposure determination component 134. The exposure control component 130 may receive the ROI 116 from the image analysis component 120 and may also receive an image 114 from the image acquisition component 110. The luminance determination component 132 may be configured to determine the luminance of the image 114. As used herein, image luminance (or brightness) refers to the perceived light intensity in the image (e.g., the perceived intensity of light emitted or reflected through the environment or by objects in the image). Image luminance may be expressed as a numerical value. The exposure determination component 134 may be configured to calculate one or more exposure settings 118 to be performed by the image acquisition device 110.

[0030] In some embodiments, the brightness determination component 132 may determine the actual brightness 136 of the image 114 based on the ROI information 116 received from the image analysis component 120 and the image 114 received from the image acquisition component 110. If the ROI information 116 indicates that at least one ROI has been detected in the image 114, the brightness determination component 132 calculates the brightness value of the ROI in the image 114.

[0031] In some embodiments, brightness 136 is expressed as a brightness value. Therefore, the brightness determination component 132 may determine the actual brightness value of the entire image 114 or of the ROI within the image 114. In some embodiments, the brightness determination component 132 first calculates the brightness value Y of one or more pixels in the image. Assuming the image 114 is in RGB format, the brightness value Y of the pixel pixelThe formula for calculation is shown below as formula (1).

[0032] [Number]

[0033] Here, R pixel , G pixel and B pixel are the R value, G value, and B value of the pixel respectively. Depending on the implementation, the coefficients of R pixel , G pixel , B pixel in formula (1) may vary according to the standard that image 114 conforms to. For example, the coefficients of the above formula (1) are applicable to images conforming to the ITU BT.709 standard. For images conforming to the CCIR 601 standard, the coefficients may be different as shown in formula (2).

[0034] [Number]

[0035] When the captured image 114 has not yet been converted to the RGB format (for example, when image 114 still needs to be demosaicked), Y pixel may be calculated in a different way. For example, for Bayer color filter image data, the formula for calculating Y pixel is shown below as formula (3):

[0036] ] [Number]

[0037] Here, G pixel is the G value of the pixel.

[0038] The luminance determination component 132 may calculate the luminance value according to formula (4).

[0039] [Number]

[0040] If image 114 contains one ROI (for example, if ROI information 116 indicates that a single ROI has been detected), the brightness determination component 132 determines the Y of the pixels within the ROI. pixel Luma is calculated using only the values. That is, the calculated luminance value becomes the luminance value of the ROI. If image 114 contains two or more ROIs (for example, if ROI information 116 indicates that multiple ROIs have been detected), the luminance determination component 132 determines the Y of the pixels in the largest ROI among the two or more ROIs. pixel Luma is calculated using only the values. Alternatively, in some embodiments, the brightness determination component 132 uses the Y of pixels included in all two or more ROIs. pixel You can also calculate Luma using the values.

[0041] If image 114 does not contain an ROI (for example, if ROI information 116 indicates that no ROI was detected), the brightness determination component 132 determines the Y from the entire image. pixel The value (i.e., Y from all pixels of the image) pixel Luma may be calculated using the Y value. In other words, the calculated Luma is the brightness value of the entire image. Alternatively, depending on the embodiment, the brightness determination component 132 may use the Y value of a part of the image (for example, a part of the image away from the light source). pixel Brightness may be calculated using the values. For example, if the light source is in the upper part of the image, the brightness determination component 132 will use the Y value of the lower part of the image (e.g., the bottom two-thirds). pixel You can also calculate Luma using the values.

[0042] The exposure determination component 134 is configured to determine one or more exposure settings 118 that the image acquisition device 110 should use when capturing a subsequent image. The exposure control component 130 provides the determined exposure settings 118 to the image acquisition component 110. This allows the image acquisition component 110 to capture a subsequent image using the determined exposure settings. This subsequent image, like the image 114 described above, can serve as the basis for determining subsequent exposure settings.

[0043] In some embodiments, the exposure determination component 134 may calculate the exposure setting 118 based on the brightness 136 of the image 114 determined by the brightness determination component 132, the current exposure setting used to capture the image 114 by the image capture component 110, and a set of predetermined values ​​and / or constants. In some embodiments, the exposure determination component 134 may calculate the exposure setting 118 according to an algorithm that causes the actual brightness 136 of the image captured by the image capture component 110 (e.g., the brightness value of the ROI) to converge toward a target brightness (e.g., a target brightness value). As mentioned above, the exposure setting may include exposure time, aperture, and / or gain. Examples of algorithms and techniques for calculating the exposure setting 118 are described below.

[0044] The exposure determination component 134 determines the exposure time for the next image (hereinafter also referred to as "next exposure time") according to the algorithm shown in Table 1 below. next To decide. [Table 1]

[0045] If Image 114 contains an ROI, the Luma value shown in Table 1 is the luminance of that ROI (or the largest ROI among multiple ROIs in Image 114). If Image 114 does not contain an ROI, the Luma value shown in Table 1 is the luminance of the entire Image 114.

[0046] As shown in Table 1, the exposure determination component 134 first sets Luma to L MIN and L MAX Compare with the range defined by L. MIN ≤Luma≦L MAX In this case, the exposure determination component 134 determines the current exposure time E current (The exposure time used to take image 114) is used as the next exposure time E for taking the next image. next Set as follows: That is, the next exposure time is the same as the current exposure time. In some embodiments, L MIN and L MAX This defines a range of luminance values ​​that have been empirically determined to be suitable for image processing and analysis (e.g., object recognition).

[0047] Luma is L MIN and L MAX If it is outside the range defined by, the exposure determination component 134 compares Luma to the possible highest and lowest luminance values ​​of 255 and 0, respectively. If Luma is within the range of those values ​​and is not equal to those values, E NEXT That's abnormal NEXT =E CURRENT ×(L TARGET It is calculated as / Luma). Here, L TARGET L defines the target brightness value. MIN Ya L MAX Similarly, L TARGET This is a target luminance value that has been empirically determined to be suitable for image processing and analysis (e.g., object recognition). In some embodiments, E NEXT The algorithm for determining the target brightness value L is determined by the brightness value of the ROI included in the captured image with an FOV of 112. TARGET The goal is to adjust the exposure settings to converge to a specific point.

[0048] If Luma is equal to the maximum brightness value of 255, E NEXT is E NEXT =E CURRENT It is calculated as / factor. If the luminance is equal to the minimum luminance value of 0, E NEXT is E NEXT =E CURRENTIt is calculated as ×factor, where factor is E such that Luma converges to the target brightness value. NEXT A predetermined scalar or constant that can be used to adjust. Alternatively, depending on the embodiment, if image 114 is saturated or overexposed (for example, if Luma of image 114 is L MAX If it is above a certain threshold (which may be higher but lower than the highest possible brightness value), E NEXT However, E NEXT =E CURRENT It may be calculated as / factor. Similarly, depending on the embodiment, if image 114 is underexposed (for example, if the Luma of image 114 is L MIN (If it is below a predetermined threshold, but can be below the lowest possible brightness value, E NEXT is, E NEXT =E CURRENT It is sometimes calculated as ×factor.

[0049] As mentioned above, L MIN , L MAX , L TARGET , and factor are predetermined parameters. In some embodiments, one or more of these parameters are determined and set empirically. Table 2 shows examples of values ​​for these predetermined parameters: [Table 2]

[0050] The exposure determination component 134 repeats the above algorithm, for example, and G NEXT and G CURRENT E NEXT and E CURRENT By substituting this, the current image gain G CURRENT Based on the following image G NEXT The gain can also be calculated. The exposure control component 130 is E NEXT and / or G NEXT The exposure setting 118 is provided to the image capture component 110. The image capture component 110 is E NEXT and GNEXT A new image is captured using this method. As mentioned above, this image may be processed by the image analysis component 120 and used to determine the exposure setting 118 that should be used by the exposure control component 130 to capture the next image in the series.

[0051] Figure 2 shows a decision flowchart of an exemplary process 200 for dynamic automatic exposure control according to several embodiments. Process 200 represents a decision flow in which the imaging system 100 can determine the exposure settings for capturing the next image.

[0052] Process 200 begins with step 202, in which the image acquisition component 110 captures an image (e.g., image 114). In step 204, the brightness determination component 132 calculates the brightness of the entire captured image. In step 206, the image analysis component 120 performs object detection on the captured image to detect ROIs within the image, and ROI information (e.g., ROI information 116) may be generated. Therefore, in process 200, the brightness determination component 132, regardless of whether the image contains ROIs, performs object detection to detect ROIs within the image before or simultaneously with object detection (e.g., Y of pixels of the entire image). pixel The brightness of the entire image may be calculated using the values. Alternatively, as described later, the brightness determination component 132 may determine the brightness of the entire image after step 206.

[0053] In step 208, the exposure control component 130 determines whether at least one ROI is detected in the image (for example, based on ROI information 116 received from the image analysis component 120). If the exposure control component 130 determines that at least one ROI is detected in the image (208-Yes), process 200 proceeds to step 210. In step 210, the exposure control component 130 (for example, the luminance determination component 132) calculates the luminance value of the ROI in the image (for example, the luma value of the ROI). This luminance value may be provided to the exposure determination component 134 as luminance 136. Process 200 then proceeds to step 212.

[0054] If the exposure control component 130 determines that no ROI is detected in the image (208-NO), process 200 proceeds to step 212. In some embodiments, the brightness determination component 132 may calculate the brightness value of the entire image between steps 208 and 212. This brightness value of the entire image may be provided to the exposure determination component 134 as brightness 136.

[0055] In step 212, the exposure determination component 134 calculates the exposure setting for subsequent image capture based on the image luminance 136. As described above, the luminance 136 provided to the exposure determination component 134 may be the luminance of an ROI in the image, or it may be the luminance of the entire image.

[0056] In some embodiments, in an optional step 214, the imaging system 100 may skip an image. That is, the imaging system 100 may skip capturing the image immediately following and instead capture a subsequent image using the calculated exposure setting. Regardless of whether process 200 includes step 214, process 200 returns to step 202 with the calculated exposure setting. In step 202, the image capture component 110 captures a new image using the calculated exposure setting. In some embodiments, skipping an image in step 214 may stabilize the exposure setting.

[0057] In some embodiments, in step 212, the exposure determination component 134 may calculate the exposure setting according to the algorithm described above with reference to Table 1. In other embodiments, the exposure determination component 134 may calculate the exposure setting according to a different algorithm. In particular, this algorithm may calculate the exposure setting for the next image using the luminance values ​​from at least two captured images, i.e., the current image and the previous image (e.g., the most recently captured image or an image taken even earlier), so that the luminance value converges toward a target value. This algorithm calculates the exposure setting according to the following equations (5) to (9):

[0058] E 1f =((L T -L1) / (L T +L1))+1 (5)

[0059] E 2f =((L T -L2) / (L T +L2))+1 (6)

[0060] E 1T =E 1f ×E1 (7)

[0061] E 2T =E 2f ×E2 (8)

[0062] E3=(E1T +E 2T ) / 2 (9)

[0063] In equations (5) to (9), E1 and E2 are the exposure settings used to capture the current image and the previous image, respectively. E2 is the exposure setting calculated to capture the next image. T This is the target brightness value, and is shown in Table 1 as L TARGET The same applies. L1 and L2 are the luminance values ​​of the current image and the previous image, respectively. As described above with reference to equations (1) to (4), L1 and L2 can be either the luminance value of the ROI or the luminance value of the entire image, depending on whether the associated image contains the ROI. 1f and E 2f These are exposure setting adjustment coefficients for E1 and E2, respectively. Therefore, in equations (5) to (9), the exposure determination component 134 determines the adjusted exposure setting E for the current image based on the current image brightness value and the target brightness value. 1T Calculate the brightness value of the previous image and the target brightness value to set the adjusted post-exposure setting E for the previous image. 2T It calculates the following: Then, the exposure determination component 134 is E 1T and E 2T The average of these values ​​is calculated to obtain the exposure setting E3 for taking the subsequent image. Thus, equations (5) to (9) calculate E3 according to a sliding window method that uses the values ​​(e.g., brightness value, exposure setting, etc.) of the current image and the previous image (e.g., the most recent image).

[0064] Depending on the embodiment, formulas (5) to (9) may be simplified to the following formula (10):

[0065]

number

[0066] Depending on the embodiment, for the first two captured images in process 200, E1 and E2 may be set to predetermined values (e.g., maximum and minimum exposure times or gains, respectively). For example, if E3 is the gain in subsequent image capture and E1 and E2 are the current and previous gains, respectively, a predetermined gain may be set for the first two images. For example, E1 may be set to the maximum gain gain max and E2 may be set to the minimum gain gain min . In some embodiments, gain max = 79 and gain min = 1.

[0067] Furthermore, depending on the embodiment, L T may vary depending on the resolution of the image. For example, while L T is preset to a value of 110 for an image with VGA resolution, L T may be preset to a value of 40 for an image with HD resolution.

[0068] Furthermore, depending on the embodiment, when one exposure setting (e.g., exposure time) reaches the maximum or minimum value while another exposure setting (e.g., gain) is kept constant, the same formula may be used to calculate the value of the other exposure setting while keeping the exposure setting that reached the maximum or minimum value constant. For example, if the exposure time reaches the maximum value by process 200 while the gain is kept constant, the exposure control component 130 may calculate the value of the gain by process 200 while keeping the exposure time constant (e.g., at the maximum value).

[0069] Figure 3 shows another block diagram of the imaging system 300 according to several embodiments. More specifically, the imaging system 300 may be configured to detect one or more regions of interest (ROIs) in an image. Furthermore, the imaging system 300 may be configured to determine exposure settings based on the detected ROIs and to capture subsequent images using those exposure settings. In some embodiments, the imaging system 300 may be an example of the imaging system 100 shown in Figure 1. The imaging system 300 includes a device interface 310, a processing system 320, and a memory 330.

[0070] The device interface 310 is configured to communicate with one or more components of an image acquisition device (for example, the image acquisition component 110 in Figure 1). In some embodiments, the device interface 310 may include an image sensor interface (I / F) 312 configured to receive images via the image acquisition device. In some embodiments, the image sensor interface 312 may capture images using exposure settings.

[0071] Memory 330 may include a data store 331 configured to store one or more models for object detection, and a data store 332 configured to store one or more received images and output data for object detection of the images, including, for example, ROI information. Memory 330 may also include a non-temporary computer-readable medium (equipped with one or more non-volatile memory elements, including, for example, EPROM, EEPROM, flash memory, hard drive, etc.) which may store at least the following software (SW) modules: Object detection software module 334 detects one or more objects based on the first image and determines the corresponding ROI within the first image. • Brightness determination software module 336 for determining the brightness of an image, and • Exposure determination software module 738 that determines the exposure setting based on the brightness. Each software module contains instructions that, when executed by the processing system 320, cause the imaging system 300 to perform the corresponding function.

[0072] The processing system 320 may include one or more suitable processors capable of executing scripts or instructions of one or more software programs stored in the imaging system 300 (for example, in the memory 330). For example, the processing system 320 may execute an object detection SW module 334 to determine one or more ROIs in the first image, execute a brightness determination SW module 336 to determine the brightness of the image, or execute an exposure determination SW module 338 to determine the exposure setting based on the brightness.

[0073] Figure 4 shows a flowchart illustrating exemplary operations 400 for object detection according to several embodiments. In some embodiments, these exemplary operations 400 may be performed by an imaging system such as the imaging system 100 in Figure 1.

[0074] The imaging system may capture the first image in a series of images using a first exposure setting (402). The imaging system may detect one or more regions of interest (ROI) in the first image (404). The imaging system may determine a first brightness value for the first ROI in the first image (406). The imaging system may determine a second exposure setting based on the first brightness value (408). The imaging system may capture a second image following the first image in a series of images using the second exposure setting (410).

[0075] Depending on the configuration, the imaging system may calculate the average brightness value of the first ROI.

[0076] Depending on the configuration, the imaging system may compare the first luminance value with a predetermined luminance value range.

[0077] Depending on the configuration, the imaging system may set the first exposure setting as the second exposure setting based on a determination that the first brightness value is within a predetermined brightness value range.

[0078] Depending on the configuration, the imaging system may determine a second exposure setting based on a first brightness value and a target brightness value.

[0079] Depending on the embodiment, the imaging system may determine the second exposure setting based on a third exposure setting and a second brightness value for the third image in the series of images. The third image is captured before the first image using the third exposure setting.

[0080] Depending on the embodiment, the imaging system may calculate a first exposure adjustment coefficient based on a first brightness value and a target brightness value, calculate a first target exposure setting based on a first exposure setting and the first exposure adjustment coefficient, calculate a second exposure adjustment coefficient based on a second brightness value and a target brightness value, calculate a second target exposure setting based on a third exposure setting and the second exposure adjustment coefficient, and calculate a second exposure setting based on the first target exposure setting and the second target exposure setting.

[0081] Depending on the configuration, the imaging system may calculate the sum of the first target exposure setting and the second target exposure setting.

[0082] Depending on the configuration, the imaging system may detect multiple ROIs in the first image, and the first ROI may have the largest area among the multiple ROIs.

[0083] Depending on the configuration, the imaging system may determine a second brightness value for the first image, decide on a third exposure setting based on the second brightness value, and capture the third image following the first image in the series of images using the third exposure setting.

[0084] Those skilled in the art will understand that information and signals can be represented using any of the various different techniques and methods. For example, data, instructions, commands, information, signals, bits, symbols, and chips, which may be mentioned throughout the above description, can be represented by voltage, electric current, electromagnetic waves, magnetic fields or magnetic particles, light fields or optical particles, or any combination thereof.

[0085] Furthermore, those skilled in the art will understand that various exemplary logic blocks, modules, circuits, and algorithmic steps described in relation to the embodiments disclosed herein can be implemented as electronic hardware, computer software, or a combination of both. To clearly demonstrate this hardware-software compatibility, various exemplary components, blocks, modules, circuits, and steps are described above in general terms of their function. Whether such functions are implemented as hardware or software depends on the design constraints imposed on individual applications and the overall system. Those skilled in the art may implement the described functions in various ways for individual applications, but such implementation decisions should not be construed as resulting in a deviation from the scope of the disclosure.

[0086] Methods, sequences, or algorithms described in relation to the embodiments disclosed herein may be implemented directly in hardware, in software modules executed by a processor, or in a combination of both. The software modules may reside in RAM memory, flash memory, ROM memory, EPROM memory, EEPROM memory, registers, hard disks, removable disks, CD-ROMs, or any other form of storage medium known in the art. An exemplary storage medium is connected to the processor so that the processor can read information from and write information to the storage medium. Alternatively, the storage medium may be integrated with the processor.

[0087] The above specification has described embodiments with reference to specific examples. However, it will be apparent that various changes and modifications can be made without departing from the broader scope of the disclosure set forth in the appended claims. Therefore, this specification and the drawings should be interpreted as illustrative rather than restrictive.

Claims

1. The first image in a series of images is taken using the first exposure setting, To detect one or more regions of interest (ROIs) in the first image, To determine the first luminance value relating to the first ROI of the first image, The second exposure setting is determined based on the first brightness value, The second image following the first image in the series of images is taken using the second exposure setting, including, method.

2. Determining the first luminance value includes calculating the average luminance value of the first ROI. The method according to claim 1.

3. Determining the second exposure setting includes comparing the first luminance value with a predetermined range of luminance values. The method according to claim 1.

4. Determining the second exposure setting further includes setting the first exposure setting as the second exposure setting based on the determination that the first luminance value is within a predetermined range of the luminance value. The method according to claim 3.

5. Determining the second exposure setting includes determining the second exposure setting based on the first exposure setting and the target brightness value. The method according to claim 1.

6. Determining the second exposure setting further includes determining the second exposure setting based on the third exposure setting and a second brightness value for the third image in the series of images, The third image was taken before the first image using the third exposure setting. The method according to claim 5.

7. Determining the second exposure setting is The first exposure adjustment coefficient is calculated based on the first luminance value and the target luminance value, The first target exposure setting is calculated based on the first exposure setting and the first exposure adjustment coefficient. The second exposure adjustment coefficient is calculated based on the second luminance value and the target luminance value, The second target exposure setting is calculated based on the third exposure setting and the second exposure adjustment coefficient. The second exposure setting is calculated based on the first target exposure setting and the second target exposure setting. including, The method according to claim 6.

8. Calculating the second exposure setting includes calculating the sum of the first target exposure setting and the second target exposure setting. The method according to claim 7.

9. Detecting one or more regions of interest (ROIs) in the first image includes detecting multiple ROIs in the first image. The first ROI has the largest area among the plurality of ROIs. The method according to claim 1.

10. Determining the second brightness value for the first image, The third exposure setting is determined based on the second brightness value, The third image following the first image in the series of images is taken using a third exposure setting, This also includes, The method according to claim 1.

11. Image sensor and One or more processors, A memory coupled to one or more processors, A computing system comprising, When the memory is executed by the one or more processors, the computing system The first image in the series of images is taken using the first exposure setting. In the first image, one or more regions of interest (ROIs) are detected. Determine the first brightness value related to the first ROI of the first image. Based on the first brightness value, the second exposure setting is determined. It stores a command to take a second image following the first image in the series of images using the second exposure setting. Computing system.

12. When the instruction is executed by one or more processors, the computing system is caused to calculate the average brightness value of the first ROI. The computing system according to claim 11.

13. When the instruction is executed by one or more processors, the computing system is made to compare the first brightness value with a predetermined range of brightness values. The computing system according to claim 11.

14. When the instruction is executed by one or more processors, the computing system causes the first exposure setting to be set as the second exposure setting based on the determination that the first brightness value is within a predetermined range of brightness values. The computing system according to claim 13.

15. When the instruction is executed by one or more processors, it causes the computing system to determine the second exposure setting based on the first exposure setting and the target brightness value. The computing system according to claim 11.

16. When the instruction is executed by the one or more processors, the computing system is caused to determine the second exposure setting based on the third exposure setting and the second brightness value for the third image in the series of images. The third image was taken before the first image using the third exposure setting. The computing system according to claim 15.

17. When the instruction is executed by one or more processors, the computing system A first exposure adjustment coefficient is calculated based on the first luminance value and the target luminance value. Based on the first exposure setting and the first exposure adjustment coefficient, the first target exposure setting is calculated. A second exposure adjustment coefficient is calculated based on the second luminance value and the target luminance value. Based on the third exposure setting and the second exposure adjustment coefficient, the second target exposure setting is calculated. The second exposure setting is calculated based on the first target exposure setting and the second target exposure setting. The computing system according to claim 16.

18. When the instruction is executed by the one or more processors, the computing system is caused to calculate the sum of the first target exposure setting and the second target exposure setting. The computing system according to claim 17.

19. When the instruction is executed by one or more processors, the computing system Multiple ROIs are detected in the first image, The first ROI has the largest area among the plurality of ROIs. The computing system according to claim 11.

20. When the instruction is executed by one or more processors, the computing system Determine the second brightness value for the first image, Based on the second brightness value, the third exposure setting is determined. The third image following the first image in the series of images is taken using a third exposure setting. The computing system according to claim 11.

Images