Brightness adjusting system and method

Abstract

The invention provides a brightness adjusting system and method. The brightness adjusting system comprises a camera, a direction sensor and a processor. The camera is configured to obtain an image based on the exposure setting and the gain. The direction sensor is configured to obtain sensor data. The processor is configured to determine a current direction in which the camera faces based on the sensor data, and adjust an exposure setting or gain of the camera based on the current direction.

Description

Technical Field

[0001] The present disclosure relates to a brightness adjustment system; specifically, the present disclosure relates to a brightness adjustment system and a brightness adjustment method. Background Art

[0002] Cameras have brought convenience to our lives in many ways. For example, users can use cameras to capture memories, share experiences, and record important events. In addition, cameras can be used in various industries, such as in security, monitoring, news, and entertainment. In general, cameras are a convenient and versatile tool that can be used to improve our lives in many ways. Summary of the invention

[0003] The present disclosure relates to a brightness adjustment system and a brightness adjustment method to improve user experience.

[0004] In the present disclosure, a brightness adjustment system is provided. The brightness adjustment system includes a camera, a direction sensor, and a processor. The camera is configured to obtain an image based on an exposure setting and a gain. The direction sensor is configured to obtain sensor data. The processor is configured to determine a current direction facing the camera based on the sensor data, and adjust the exposure setting or gain of the camera based on the current direction.

[0005] In the present disclosure, a brightness adjustment method is provided. The brightness adjustment method comprises the following steps: obtaining sensor data through a direction sensor; determining a current direction facing a camera based on the sensor data; adjusting an exposure setting or a gain of the camera based on the current direction; and obtaining an image based on the exposure setting and the gain.

[0006] Based on the above, according to the brightness adjustment system and the brightness adjustment method, the exposure setting and / or gain of the camera is adjusted before capturing an image based on sensor data, thereby improving user experience.

[0007] In order to make the above contents easier to understand, several embodiments attached with the drawings will be described in detail below. BRIEF DESCRIPTION OF THE DRAWINGS

[0008] The present disclosure includes accompanying drawings to provide a further understanding of the present disclosure, and the accompanying drawings are incorporated in and constitute a part of this specification. The accompanying drawings illustrate exemplary embodiments of the present disclosure and together with the description serve to explain the principles of the present disclosure.

[0009] Figure 1 is a schematic diagram of a brightness adjustment system according to an embodiment of the present disclosure.

[0010] Figure 2 is a schematic diagram of a brightness adjustment scene according to an embodiment of the present disclosure.

[0011] Figure 3A Schematic diagram of the software structure of the brightness adjustment system according to an embodiment of the present disclosure.

[0012] Figure 3B Schematic diagram of the software structure of the brightness adjustment system according to an embodiment of the present disclosure.

[0013] Figure 4 is a schematic flowchart of a brightness adjustment method according to an embodiment of the present disclosure.

[0014] [Explanation of Symbols]

[0015] 100: Brightness adjustment system

[0016] 110: Camera

[0017] 120: Direction sensor

[0018] 130: Processor

[0019] 140: Memory

[0020] 200: Brightness adjustment scene

[0021] 300A, 300B: Software Structure

[0022] 400: Brightness adjustment method

[0023] BA: Bright Area

[0024] D1: First direction

[0025] D2: Second direction

[0026] DA: Dark Area

[0027] ENV: Environment

[0028] ES1: First exposure setting

[0029] ES2: Second exposure setting

[0030] IMG: Image

[0031] MD: Map Data

[0032] S301, S302, S303, S304, S351, S352, S353, S354, S410, S420, S430, S440: Steps

[0033] SD: Sensor data DETAILED DESCRIPTION

[0034] Throughout the specification and the appended claims of the present disclosure, certain terms are used to refer to specific components. It should be understood by those skilled in the art that electronic device manufacturers may refer to the same components by different names. It is not intended herein to distinguish between those components that have the same function but different names. In the following description and claims, words such as "comprise" and "include" are open-ended terms and should be interpreted as "including but not limited to..." The terms "first", "second" and similar terms are only used to name discrete elements or to distinguish between different embodiments or scopes. Therefore, the terms should not be regarded as limiting the upper or lower limits of the number of elements and should not be used to limit the order of arrangement of elements.

[0035] In order to bring an immersive experience to users, technologies related to extended reality (XR), such as augmented reality (AR), virtual reality (VR), and mixed reality (MR), are being developed. AR technology enables users to bring virtual elements into the real world. VR technology enables users to enter a new virtual world to experience a different life. MR technology combines the real world with the virtual world. In addition, in order to bring a complete immersive experience to users, cameras can be used to provide users with visual content corresponding to the real environment.

[0036] When a camera is used as a visual source, the camera is typically configured to capture images frame by frame. After capturing the current frame of an image, the content of the captured image may be checked to determine whether the captured image is a good image. For example, the illumination value of the captured image may be too high, causing a portion of the captured image to be overexposed and may be determined to be a bad image. In order to capture a good image, the exposure setting or gain of the camera may be adjusted so that the next frame of the captured image (i.e., the image to be captured) may be determined to be a good image. However, since the adjustment is performed after capturing at least one frame of the image (e.g., the current frame), the bad image (i.e., the current frame) may have already affected the user experience. Therefore, a technician in the field seeks to provide a solution for preventing the occurrence of bad images.

[0037] Figure 1 is a schematic diagram of a brightness adjustment system according to an embodiment of the present disclosure. Figure 1, the brightness adjustment system 100 may include a camera 110, a direction sensor 120, and a processor 130. The camera 110 may be configured to obtain an image IMG based on an exposure setting and a gain. The direction sensor 120 may be configured to obtain sensor data SD. The processor 130 may be configured to determine a current direction that the camera 110 is facing based on the sensor data SD. In addition, the processor 130 may be configured to adjust the exposure setting or gain of the camera 110 based on the current direction. In this way, the exposure setting of the camera 110 may be adjusted before capturing the image IMG based on the sensor data SD, and / or the gain of the camera may be adjusted before outputting the image IMG, thereby increasing the user experience.

[0038] It should be noted that the exposure setting may be configured to adjust the exposure value (EV) of the camera 110, and the exposure value may include the size of the aperture of the camera 110, the time of the shutter, and the International Organization for Standardization (ISO) value, but the present disclosure is not limited thereto. In one embodiment, when the exposure value is reduced, the light capture capability of the camera 110 may be reduced so that less light enters the camera 110 to reduce the illumination value of the image IMG. In other words, the aperture size may be reduced (i.e., the f-number is increased), the shutter time of the shutter may be shortened, and / or the ISO value may be reduced to reduce the exposure value. Conversely, when the exposure value is increased, the light capture capability of the camera 110 may be increased so that more light may enter the camera 110 to increase the illumination value of the image IMG. In other words, the aperture size may be increased (i.e., the f-number is reduced), the shutter time of the shutter may be lengthened, and / or the ISO value may be increased to increase the exposure value. However, the present disclosure is not limited thereto.

[0039] In addition, in addition to adjusting the hardware settings (i.e., exposure settings) of the camera 110, the software settings may also be adjusted. More specifically, the ISO value is to adjust the sensitivity of the sensor of the camera 110 to light before the image IMG is captured, and the gain is to perform scaling control on the signal from the sensor of the camera 110 after the image IMG is captured. That is, the gain may be used to increase or decrease the brightness (i.e., illumination value) of the image IMG. For example, after the image IMG is captured, when the illumination value of the image IMG is too low, a higher gain may be applied to the image IMG to output a properly exposed image as a new image IMG. Conversely, when the illumination value of the image IMG is too high, a lower gain may be applied to the image IMG to output a properly exposed image as a new image IMG. When the gain is zero, the image IMG is not scaled. In other words, the camera 110 may be configured to obtain the image IMG not only based on the exposure setting but also based on the gain.

[0040] In one embodiment, the camera 110 may include an auto exposure (AE) function. The AE function may be a function for automatically adjusting the exposure setting to ensure that the next frame of the image IMG is properly exposed. The AE function generally works by measuring the brightness distribution (e.g., illuminance value) of the current frame of the image IMG and adjusting the exposure setting. After adjusting the exposure setting by AE, the camera 110 may be ready to capture the next frame of the image IMG. However, the current frame of the image IMG may be overexposed or underexposed. In other words, the user may not be able to obtain the information the user wants to know through the current frame of the image IMG. Therefore, in this embodiment, by using the direction sensor 120 to detect the current direction of the camera 110, the processor 130 may know that the camera 110 may be changing direction, thereby adjusting the exposure setting and / or gain as early as possible, thereby preventing the current frame of the image IMG from being incorrectly exposed.

[0041] In one embodiment, the camera 110 may include, for example, a complementary metal oxide semiconductor (CMOS) camera, a charge coupled device (CCD) camera, a light detection and ranging (LiDAR) device, a radar, an infrared sensor, an ultrasonic sensor, other similar devices, or a combination of these devices. However, the present disclosure is not limited thereto.

[0042] In one embodiment, the direction sensor 120 may include, for example, a gyroscope, an accelerometer, other similar devices, or a combination of these devices. In one embodiment, the direction sensor 120 is a gyroscope, and the sensor data SD includes three angular velocities of three degrees of freedom (DOF). In one embodiment, the direction sensor 120 is an accelerometer, and the sensor data SD includes three linear acceleration values ​​in the three degrees of freedom. In one embodiment, the direction sensor 120 includes a gyroscope and an accelerometer, and the sensor data SD includes changes in six degrees of freedom, for example: three angular velocities and three linear acceleration values ​​corresponding to three vertical axes (e.g., X-axis, Y-axis, Z-axis), respectively. However, the present disclosure is not limited thereto.

[0043] In one embodiment, the processor 130 may include, for example, a microcontroller unit (MCU), a central processing unit (CPU), a microprocessor, a digital signal processor (DSP), a programmable controller, a programmable logic device (PLD), other similar devices, or a combination of these devices. The present disclosure is not limited to this. In addition, in an embodiment, each of the functions of the processor 130 may be implemented as a plurality of program codes. The program code is stored in a memory (e.g., Figure 1 140 in the memory 140) and executed by the processor 130. Alternatively, in an embodiment, each of the functions of the processor 130 may be implemented as one or more circuits. The present disclosure does not limit the use of software or hardware to implement the functions of the processor 130.

[0044] In some embodiments, the brightness adjustment system 100 may further include a memory 140. The memory 140 may be configured to store the image IMG, the sensor data SD, and / or the exposure setting of the camera 110. However, the present disclosure is not limited thereto. In one embodiment, the memory 140 may include, for example, a NAND flash memory core, a NOR flash memory core, a static random access memory (SRAM) core, a dynamic random access memory (DRAM) core, a magnetoresistive random access memory (MRAM) core, a phase change memory (PCM) core, a resistive random access memory (ReRAM) core, a three-dimensional (3D) XPoint memory core, a ferroelectric random-access memory (FeRAM) core, or other types of memory cores suitable for storing data. However, the present disclosure is not limited thereto.

[0045] In one embodiment, the brightness adjustment system 100 may be included in a head-mounted display (HMD), wearable glasses (e.g., AR / VR goggles), a tablet, a smart phone, a projector, a laptop computer, other similar devices suitable for reality-related technologies, or a combination of these devices. In another embodiment, the brightness adjustment system 100 may be included in an accessory device, such as a wristband, a glove, a gauntlet, a sock, a shoe, or other similar accessories. However, the present disclosure is not limited to this.

[0046] In one embodiment, each of the components of the brightness adjustment system 100 may be directly or indirectly coupled or communicated with each other. In one embodiment, each of the camera 110, the direction sensor 120, the processor 130, and the memory 140 may include or be coupled to a network module. The network module may include, for example, a wired network module, a wireless network module, a Bluetooth module, an infrared module, a radio frequency identification (RFID) module, a Zigbee network module, or a near field communication (NFC) network module, but the present disclosure is not limited thereto. That is, each of the components of the brightness adjustment system 100 may be configured to communicate with each other through wired communication or wireless communication.

[0047] Figure 2 is a schematic diagram of a brightness adjustment scene according to an embodiment of the present disclosure. Figure 1 to Figure 2 The brightness adjustment scene 200 illustrates how to adjust the exposure setting and / or gain of the camera 110 of the brightness adjustment system 100. It should be noted that, although only the camera 110 is illustrated in the brightness adjustment scene 200 to represent the brightness adjustment system 100 for ease of explanation, the present disclosure is not limited thereto.

[0048] In one embodiment, the brightness adjustment system 100 may be arranged in the environment ENV. The environment ENV may include a bright area BA and a dark area DA. In one embodiment, the environment ENV may be a room, and the bright area BA may be a lamp on the ceiling of the room. On the other hand, the dark area DA may be a dark carpet on the floor of the room. However, the present disclosure is not limited thereto. For example, in another embodiment, the bright area BA may include, for example, a window, a mirror for reflecting a light source, white paper, a pile of snow, other similar objects, or a combination of these objects. That is, the bright area BA may be an area or object in the environment ENV, and the illuminance value of the bright area BA may be higher than the average illuminance value of the environment ENV. Similarly, the dark area DA may include, for example, a cave, a shadow, black paper, other similar objects, or a combination of these objects. That is, the dark area DA may be an area or object in the environment ENV, and the illuminance value of the dark area DA may be lower than the average illuminance value of the environment ENV. In addition, the number of bright areas BA and / or the number of dark areas DA may be more than one, and the present disclosure does not limit the number of bright areas BA and / or the number of dark areas DA.

[0049] By using the direction sensor 120 to detect the current direction of the camera 110, the processor 130 can know that the camera 110 may be changing direction, so as to adjust the exposure setting and / or gain as early as possible to prevent the current frame of the image IMG from being incorrectly exposed. For example, when the camera 110 changes the current direction toward the first direction D1, the processor 130 can know that the camera 110 may be about to face the bright area BA. It should be noted that, generally, since the lamp is installed on the ceiling, the first direction D1 may be an upward direction. That is, the processor 130 may be configured to: in response to the current direction of the camera 110 being the upward direction, reduce the exposure value and / or gain of the exposure setting of the camera 110. Conversely, when the camera 110 changes the current direction toward the second direction D2, the processor 130 may know that the camera 110 may be about to face the dark area DA. Similarly, since the brightness of the floor is generally darker than the brightness of the ceiling, the second direction D2 may be a downward direction. That is, the processor 130 may be configured to increase the exposure value and / or gain of the exposure setting of the camera 110 in response to the current direction of the camera 110 being the downward direction. In this way, the exposure setting and / or gain of the camera 110 may be adjusted before the image IMG is captured based on the sensor data SD, thereby preventing a bad image from occurring.

[0050] Figure 3A Schematic diagram of the software structure of the brightness adjustment system according to an embodiment of the present disclosure. Figures 1 to 3A, the software structure 300A of the brightness adjustment system 100 illustrates how the brightness adjustment system 100 establishes map data MD of the environment ENV surrounding the user. The map data MD may include illumination values ​​of all objects or areas in the environment ENV. That is, the map data MD may include information on where the bright area BA is located and / or where the dark area DA is located. In one embodiment, the map data MD may include the bright direction of the bright area BA and the dark direction of the dark area DA, but the present disclosure is not limited thereto. In addition, the memory 140 of the brightness adjustment system 100 may be configured to store the map data MD. Therefore, the processor 130 may be configured to adjust the exposure setting and / or gain of the camera 110 based on the current direction and the map data MD.

[0051] like Figure 3A As shown, the camera 110 may be configured to provide a current frame of the image IMG, and the direction sensor 120 may be configured to provide sensor data SD. In step S301, the processor 130 may be configured to obtain a current illuminance value of the current frame of the image IMG by calculating the average value of the brightness distribution of the current frame of the image IMG. In step S302, in one embodiment, when the camera 110 is changing direction, if the current illuminance value of the current frame of the image IMG suddenly increases by a considerable percentage (i.e., becomes relatively high), or suddenly exceeds a predetermined threshold, this may mean that there may be a light source (i.e., a bright area BA) in this direction. However, the present disclosure is not limited to this. In addition, in addition to finding a light source, step S302 can also be used to find a dark area DA. The implementation details of finding the dark area DA can refer to the description of finding the bright area BA to obtain sufficient teachings, suggestions and implementation examples, and the details will not be repeated one by one here.

[0052] In step S303, the processor 130 may be configured to obtain the current direction of the camera 110 based on the sensor data SD from the direction sensor 120. That is, in response to the camera 110 changing the current direction and the current illuminance value of the current frame of the image IMG being greater than the previous illuminance value of the previous frame of the image IMG by a predetermined percentage, the current direction is determined as the bright direction of the bright area BA. Similarly, in response to the camera 110 changing the current direction and the current illuminance value of the current frame of the image IMG being less than the previous illuminance value of the previous frame of the image IMG by a predetermined percentage, the current direction is determined as the dark direction of the dark area DA. In step S304, the processor 130 may be configured to provide the bright direction of the bright area BA (or the dark direction of the dark area DA) to the memory 140 to store the bright direction (or the dark direction) in the map data MD. The processor 130 may repeatedly perform steps S301 to S304 until all illuminance values ​​of the environment ENV are recorded in the map data MD in the memory 140. As such, by utilizing the map data MD, the exposure setting and / or gain of the camera 110 may be adjusted as early as possible, thereby preventing the current frame of the image IMG from being incorrectly exposed.

[0053] Figure 3B Schematic diagram of the software structure of the brightness adjustment system according to an embodiment of the present disclosure. Figures 1 to 3B The software structure 300B of the brightness adjustment system 100 illustrates how the brightness adjustment system 100 utilizes the AE function and the map data MD to adjust the exposure setting and / or gain of the camera 110 .

[0054] like Figure 3B As shown, the camera 110 may be configured to provide a current frame of the image IMG, the direction sensor 120 may be configured to provide sensor data SD, and the memory 140 may be configured to provide map data MD. In step S351, the processor 130 may be configured to obtain a current illuminance value of the current frame of the image IMG by calculating the average value of the brightness distribution of the current frame of the image IMG. In step S352, the processor 130 may be configured to calculate a gain of the next frame of the image IMG based on the AE function to generate the next frame of the image IMG. In addition, the processor 130 may be configured to generate a first exposure setting ES1 based on the gain, and output the first exposure setting ES1. That is, the first exposure setting ES1 may be determined based on the AE function.

[0055] In step S353, the processor 130 may be configured to obtain the current direction of the camera 110 based on the sensor data SD from the direction sensor 120. In addition, the processor 130 may be configured to determine whether the current direction is about to (will) face the light source (bright area BA) based on the map data MD. That is, the processor 130 may be configured to determine the difference (e.g., angle or distance) between the current direction of the camera 110 and the bright direction of the bright area BA. When the difference between the current direction of the camera 110 and the bright direction of the bright area BA is less than a predetermined threshold and the difference is still decreasing, the processor 130 may be configured to determine that the camera 110 is facing or will face the bright area BA (e.g., a lamp).

[0056] In step S354, when it is determined that the camera 110 is facing or will face the bright area BA, the processor 130 may be configured to adjust the exposure setting of the camera 110 to the second exposure setting ES2 to prevent the next frame of the image IMG from being overexposed. That is, in response to the difference being less than a predetermined threshold and the difference being decreasing, the processor 130 may be configured to reduce the exposure value or gain of the exposure setting of the camera 110. For example, the exposure value may be set to the minimum value of the camera 110, but the present disclosure is not limited thereto. Similarly, when it is determined that the camera 110 is facing or will face the dark area DA, in response to the difference being less than a predetermined threshold and the difference being decreasing, the processor 130 may be configured to increase the exposure value or gain of the exposure setting of the camera 110. For example, the exposure value may be set to the maximum value of the camera 110, but the present disclosure is not limited thereto. In addition, the processor 130 may be configured to output the second exposure setting ES2. That is, the second exposure setting ES2 may be determined based on the sensor data SD and the map data MD.

[0057] It should be noted that although both the first exposure setting ES1 and the second exposure setting ES2 have been generated, the camera 110 may be configured to obtain the next frame of the image IMG based only on the second exposure setting ES2. In addition, in one embodiment, when it is determined that the camera 110 is facing or will face the bright area BA (or the dark area DA), the first exposure setting ES1 may not be output from the processor 130 to the camera 110. However, the present disclosure is not limited to this. In this way, the exposure setting of the camera 110 can be adjusted before capturing the image IMG based on the sensor data SD and the map data MD, thereby increasing the user experience.

[0058] Figure 4 is a schematic flow chart of a brightness adjustment method according to an embodiment of the present disclosure. Figures 1 to 4 The brightness adjustment method 400 may include step S410, step S420, step S430 and step S440.

[0059] In step S410, sensor data SD may be obtained by the direction sensor 120. In step S420, the current direction facing the camera 110 may be determined by the processor 130 based on the sensor data SD. In step S430, the exposure setting and / or gain of the camera 110 may be adjusted by the processor based on the current direction of the camera 110. In step S440, the image IMG may be obtained by the camera 110 based on the exposure setting and gain. In addition, the implementation details of the brightness adjustment method 400 may be referred to. Figures 1 to 4 In order to obtain sufficient teachings, suggestions and implementation examples, the details will not be repeated here one by one.

[0060] In summary, according to the brightness adjustment system 100 and the brightness adjustment method 400, by using the direction sensor 120 to detect the current direction of the camera 110, the processor 130 can know that the camera 110 may be changing direction, so as to adjust the exposure setting and / or gain as early as possible to prevent the current frame of the image IMG from being incorrectly exposed. Therefore, the user experience is improved.

[0061] It will be apparent to those skilled in the art that various modifications and variations may be made to the disclosed embodiments without departing from the scope or spirit of the present disclosure. In summary, the present disclosure is intended to cover various modifications and variations as long as they fall within the scope of the appended claims and their equivalents.

Claims

1. A brightness adjustment system, characterized in that: include: a camera configured to acquire an image based on an exposure setting and a gain; an orientation sensor configured to obtain sensor data; as well as The processor is configured to: Based on the sensor data, a current direction the camera is facing is determined, and based on the current direction, the exposure setting or the gain of the camera is adjusted.

2. The brightness adjustment system according to claim 1, characterized in that: The processor is further configured to: in response to the current direction being an upward direction, reduce the exposure value of the exposure setting or reduce the gain.

3. The brightness adjustment system according to claim 1, characterized in that: The processor is further configured to: in response to the current direction being a downward direction, increase the exposure value of the exposure setting or increase the gain.

4. The brightness adjustment system according to claim 1, characterized in that: Also includes: a memory configured to store map data of an environment surrounding a user, The processor is further configured to adjust the exposure setting or the gain of the camera based on the current direction and the map data.

5. The brightness adjustment system according to claim 4, characterized in that: in The map data includes a bright direction of a bright area in the environment, wherein the illuminance value of the bright area is higher than an average illuminance value of the environment, and The processor is further configured to: In response to the camera changing the current direction and a current illuminance value of a current frame of the image being greater than a previous illuminance value of a previous frame of the image by a predetermined percentage, determining the current direction as the bright direction of the bright area and storing the bright direction in the map data.

6. The brightness adjustment system according to claim 4, characterized in that: in The map data includes a bright direction of a bright area in the environment, wherein the illuminance value of the bright area is higher than an average illuminance value of the environment, and The processor is further configured to: In response to a difference between the current direction and the bright direction being less than a predetermined threshold and the difference being decreasing, the exposure value of the exposure setting is decreased or the gain is decreased.

7. The brightness adjustment system according to claim 6, characterized in that: The processor is further configured to: In response to the difference being less than the predetermined threshold and the difference being decreasing, the exposure value or the gain is set to a minimum value.

8. The brightness adjustment system according to claim 4, characterized in that: in The map data includes dark directions of dark areas in the environment, wherein the illuminance value of the dark areas is lower than an average illuminance value of the environment, and The processor is further configured to: In response to the camera changing the current direction and a current illuminance value of a current frame of the image being less than a previous illuminance value of a previous frame of the image by a predetermined percentage, determining the current direction as the dark direction of the dark area and storing the dark direction in the map data.

9. The brightness adjustment system according to claim 4, characterized in that: in The map data includes dark directions of dark areas in the environment, wherein the illuminance value of the dark areas is lower than an average illuminance value of the environment, and The processor is further configured to: In response to a difference between the current direction and the dark direction being less than a predetermined threshold and the difference being decreasing, increasing the exposure value of the exposure setting or increasing the gain.

10. The brightness adjustment system according to claim 9, characterized in that: The processor is further configured to: In response to the difference being less than the predetermined threshold and the difference being decreasing, the exposure value or the gain is set to a maximum value.

11. A brightness adjustment method, characterized in that: include: Obtain sensor data through the direction sensor; Determining a current direction the camera is facing based on the sensor data; adjusting an exposure setting or gain of the camera based on the current direction; as well as An image is obtained based on the exposure setting and the gain.

Images