Ambient light determination method and device, electronic equipment and storage medium

Abstract

The invention discloses an ambient light determination method and device, electronic equipment and a storage medium, and belongs to the technical field of electronic equipment. The ambient light information determination method is applied to the electronic equipment, the electronic equipment comprises cameras at different positions, and the ambient light information determination method comprises the steps that light source information corresponding to each image in at least two images is acquired, and the light source information corresponding to each image at least comprises first light source projection position information; one image is collected by a camera on the electronic equipment; based on the first light source projection position information corresponding to each image, second light source projection position information corresponding to each image is determined, the first light source projection position information is used for indicating the actual projection position of the light source in the image, and the second light source projection position information is used for indicating the projection position of the light source on the camera corresponding to the image; and determining ambient light information based on the first light source projection position information and the second light source projection position information corresponding to the at least two images.

Description

Technical Field

[0001] This application belongs to the technical field of electronic devices, and particularly relates to a method and device for determining ambient light, an electronic device, and a storage medium. Background Art

[0002] Extended Reality (XR) is a general term that encompasses all real-world experiences extended by technology for human perception. Among them, XR can include: Virtual Reality (VR), Augmented Reality (AR), and Mixed Reality (MR), etc.

[0003] The method for determining ambient light information is a method for measuring the intensity, color, and direction of ambient light, and can obtain the ambient light information of XR devices. In the use of VR, the ambient light information adopted is virtual information. Therefore, there is no need to accurately judge the actual ambient light information. However, in the use of MR / AR, accurate judgment of the ambient light source can make the rendered virtual objects more realistic and integrated with the environment background.

[0004] The existing methods for determining ambient light information often determine the ambient light information by using one of the following devices: an Ambient Light Sensor (ALS), an Infrared (IR) sensor, a Red Green Blue (RGB) camera, and a Red Green Blue - Depth (D) camera. However, no matter which device is used, the ambient light information cannot be accurately determined. Summary of the Invention

[0005] The objective of the embodiments of this application is to provide a method and device for determining ambient light, an electronic device, and a storage medium, which can determine ambient light information with relatively high accuracy.

[0006] In a first aspect, the embodiments of this application provide a method for determining ambient light information. The method for determining ambient light information is applied to an electronic device, and the electronic device includes cameras at different positions. The method for determining ambient light information includes: obtaining the light source information corresponding to each of at least two images, where the light source information corresponding to each image includes at least first light source projection position information, and one image is captured by one camera on the electronic device; determining the second light source projection position information corresponding to each image based on the first light source projection position information corresponding to each image, where the first light source projection position information is used to indicate the actual projection position of the light source in the image, and the second light source projection position information is used to indicate the projection position of the light source on the camera corresponding to the image; determining the ambient light information based on the first light source projection position information and the second light source projection position information corresponding to at least two images.

[0007] In a second aspect, an embodiment of the present application provides an ambient light information determination device, which is applied to an electronic device. The electronic device includes cameras at different positions. The device includes an acquisition module and a processing module. The acquisition module is configured to acquire the light source information corresponding to each of at least two images. The light source information corresponding to each image includes at least first light source projection position information. One image is captured by one camera on the electronic device. The processing module is configured to determine the second light source projection position information corresponding to each image based on the first light source projection position information corresponding to each image. The first light source projection position information is used to indicate the actual projection position of the light source in the image, and the second light source projection position information is used to indicate the projection position of the light source on the camera corresponding to the image. The ambient light information is determined based on the first light source projection position information and the second light source projection position information corresponding to at least two images.

[0008] In a third aspect, an embodiment of the present application provides an electronic device, which includes a processor and a memory. The memory stores a program or instruction that can run on the processor. When the program or instruction is executed by the processor, the steps of the method described in the first aspect are implemented.

[0009] In a fourth aspect, an embodiment of the present application provides a readable storage medium, on which a program or instruction is stored. When the program or instruction is executed by a processor, the steps of the method described in the first aspect are implemented.

[0010] In a fifth aspect, an embodiment of the present application provides a chip, which includes a processor and a communication interface. The communication interface is coupled to the processor. The processor is configured to run a program or instruction to implement the method described in the first aspect.

[0011] In a sixth aspect, an embodiment of the present application provides a computer program product, which is stored in a storage medium and is executed by at least one processor to implement the method described in the first aspect.

[0012] In an embodiment of the present application, the electronic device includes cameras at different positions. The electronic device obtains the light source information corresponding to each of at least two images. The light source information corresponding to each image includes at least the first light source projection position information. One image is captured by one camera on the electronic device. Based on the first light source projection position information corresponding to each image, the second light source projection position information corresponding to each image is determined. The first light source projection position information is used to indicate the actual projection position of the light source in the image, and the second light source projection position information is used to indicate the projection position of the light source on the camera corresponding to the image. Based on the first light source projection position information and the second light source projection position information corresponding to at least two images, the ambient light information is determined. In this solution, since the cameras on the electronic device have a wide field of view, the captured images are more abundant and accurate. By analyzing each of the at least two images, the first light source projection position information and the second light source projection position information corresponding to at least two images with higher accuracy can be determined. Also, since there are differences in the actual light source position information corresponding to different images in the electronic device, the electronic device can determine the ambient light information based on the actual light source projection position information (i.e., the first light source projection position information) and the estimated light source projection position information (i.e., the second light source projection position information) corresponding to each image. That is, the ambient light information determined in this solution takes into account the differences in the actual light source position information corresponding to different images and obtains ambient light information with higher accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] Figure 1 is a schematic diagram of a virtual scene provided by the prior art;

[0014] Figure 2 is a schematic diagram of the field of view angles of an RGB camera and a fish-eye camera provided by an embodiment of the present application;

[0015] Figure 3 is a schematic diagram of an electronic device including cameras at different positions provided by an embodiment of the present application;

[0016] Figure 4 is a schematic diagram of the coverage range of a wide-angle camera provided by an embodiment of the present application;

[0017] Figure 5 is one of the schematic flowcharts of the method for determining ambient light information provided by an embodiment of the present application;

[0018] Figure 6 is another schematic flowchart of the method for determining ambient light information provided by an embodiment of the present application;

[0019] Figure 7 is a schematic diagram of the key point positions of the hand projection area provided by an embodiment of the present application;

[0020] Figure 8It is the third schematic flowchart of the ambient light information determination method provided by the embodiments of the present application;

[0021] Figure 9 It is the schematic structural diagram of the ambient light information determination device provided by the embodiments of the present application;

[0022] Figure 10 It is one of the schematic hardware structures of the electronic device provided by the embodiments of the present application;

[0023] Figure 11 It is the second schematic hardware structure of the electronic device provided by the embodiments of the present application. Detailed implementation manners

[0024] Next, the technical solutions in the embodiments of the present application will be clearly described in conjunction with the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are some, rather than all, of the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art belong to the scope of protection of the present application.

[0025] The terms "first", "second", etc. in the description and claims of the present application are used to distinguish similar objects, rather than to describe a specific order or sequence. It should be understood that such data can be interchanged under appropriate circumstances so that the embodiments of the present application can be implemented in an order other than those illustrated or described herein, and the objects distinguished by "first", "second", etc. generally belong to the same category, and the number of objects is not limited. For example, the second object can be one or more. In addition, "and / or" in the description and claims means at least one of the connected objects, and the character " / " generally means an "or" relationship between the associated objects before and after.

[0026] The terms "at least one (item)", "at least one of", etc. in the description and claims of the present application refer to any one, any two, or a combination of two or more of the included objects. For example, at least one (item) of a, b, and c can represent: "a", "b", "c", "a and b", "a and c", "b and c", and "a, b, and c", where a, b, and c can be single or multiple. Similarly, "at least two (items)" means two or more, and its expressed meaning is similar to that of "at least one (item)".

[0027] To better understand the ambient light information determination method provided by the present application, the prior art will be elaborated in detail first:

[0028] In the prior art, as an independent wearable device, the price, weight, and energy consumption of XR are all restrictive factors. Therefore, generally only necessary and as few hardware as possible are attached, and the working time of the sensor is controlled.

[0029] Existing methods for determining ambient light information often implement the determination of ambient light information by using one of the devices such as ALS, IR sensors, RGB cameras, and RGBD cameras. Specifically,

[0030] For ALS: The position and intensity of the ambient light source can be determined by ALS.

[0031] For IR sensors: The intensity and direction of infrared light in the environment can be determined by IR sensors.

[0032] For RGB cameras / RGBD cameras: Image capture can be performed by RGB cameras / RGBD cameras, and based on the captured images, illumination position analysis can be carried out based on computer vision processing methods, such as specular reflection detection, shadow calculation, etc., so as to obtain the light source orientation.

[0033] It should be noted that whether it is ALS, IR sensors, or RGB cameras / RGBD cameras, the sensing angles (also known as the Field of View (FoV)) of these devices have great limitations, usually less than or equal to the first preset angle threshold (such as 120°), resulting in a limited visible range, and further leading to inaccurate ambient light information finally determined.

[0034] In addition, ALS and IR sensors also have the disadvantages of high price, large weight, and high energy consumption. And RGB cameras / RGBD cameras calculate the ambient light information based on computer vision processing methods. Due to the relatively complex calculation process, the calculation process is time-consuming, shortening the battery life of the device.

[0035] In the prior art, XR pursues the deep integration of the virtual space and the real space. Incorrect ambient light information will directly make users feel unrealistic. Exemplarily, such as Figure 1 shown, is a schematic diagram of a virtual scene provided by the prior art. From Figure 1 it can be seen that the shadow area of the first block (the dotted line is the shadow) is to the right, while the shadow area of the second block is to the right front, which will bring a relatively strange psychological feeling to the user and seem very unrealistic. In popular terms, it is "obviously fake at a glance".

[0036] To solve the above technical problems, an embodiment of the present application provides a method for determining ambient light information, which can be applied to scenarios such as MR interaction or AR interaction.

[0037] In an embodiment of the present application, the electronic device includes cameras at different positions. The electronic device obtains the light source information corresponding to each of at least two images, and the light source information corresponding to each image includes at least the first light source projection position information. One image is captured by one camera on the electronic device. Based on the first light source projection position information corresponding to each image, the second light source projection position information corresponding to each image is determined. The first light source projection position information is used to indicate the actual projection position of the light source in the image, and the second light source projection position information is used to indicate the projection position of the light source on the camera corresponding to the image. Based on the first light source projection position information and the second light source projection position information corresponding to at least two images, the ambient light information is determined. In this solution, since the cameras on the electronic device have a wide field of view, the captured images are more rich and accurate. By analyzing each of the at least two images, the first light source projection position information and the second light source projection position information corresponding to the at least two images with higher accuracy can be determined. Also, since there are differences in the actual light source position information corresponding to different images in the electronic device, the electronic device can determine the ambient light information based on the actual light source projection position information (i.e., the first light source projection position information) and the estimated light source projection position information (i.e., the second light source projection position information) corresponding to each image. That is, the ambient light information determined in this solution takes into account the differences in the actual light source position information corresponding to different images, and the ambient light information with higher accuracy is obtained.

[0038] Based on this, taking the scenario where a user uses an MR device / AR device to watch a virtual performance as an example, when the user is using the MR device / AR device to watch a virtual performance, cameras at different positions on the MR device / AR device can capture images of the user's surrounding environment. Then, based on the first light source projection position information corresponding to each of the at least two captured images, the second light source projection position information corresponding to each image is determined. Next, based on the first light source projection position information and the second light source projection position information corresponding to the at least two images, the ambient light information is determined. Finally, based on the ambient light information, the MR device / AR device can make the generated virtual elements (such as virtual actors, virtual stands, virtual audiences, etc.) match the lighting and shadow effects of the real world, enhancing the user's immersion. In addition, under this ambient light information, the user can also effectively improve visual comfort and enhance the user's immersion.

[0039] In addition, since the electronic device can determine the ambient light information in real time, it can automatically adjust the brightness to avoid being too bright or too dark. In a darker environment, the screen brightness of the electronic device can be reduced to reduce power consumption and increase battery life; or the refresh rate of other sensors or the screen can be controlled.

[0040] The cameras at different positions on the electronic device in the embodiment of the present application are described in detail below:

[0041] The camera on the electronic device is a wide-angle camera (such as a fish-eye camera), which has a relatively wide FoV and is usually greater than or equal to the above first preset angle threshold.

[0042] Exemplarily, such as Figure 2 shown, is a schematic diagram of the field of view angles of the RGB camera and the fish-eye camera provided by an embodiment of the present application. As can be seen from Figure 2 it, the FoV of the RGB camera can be 60° or 120°; the FoV of the fish-eye camera can be 120° or 170°.

[0043] Since the cameras on the electronic device usually involve different algorithms such as Simultaneous Localization and Mapping (SLAM) and gesture tracking, in order to increase the coverage range, multiple (such as 4 or 6) cameras are usually set on the electronic device, and these multiple cameras are respectively set at different positions on the electronic device, so that the coverage range can exceed the second preset angle threshold (such as 180°) to completely cover the range shown by the human eye.

[0044] Exemplarily, such as Figure 3 shown, is a schematic diagram of the electronic device including cameras at different positions provided by an embodiment of the present application. As can be seen from Figure 3 it, the electronic device includes six fish-eye cameras at different positions, namely fish-eye camera A, fish-eye camera B, fish-eye camera C, fish-eye camera D, fish-eye camera E, and fish-eye camera F. Among them, fish-eye camera A and fish-eye camera B are used to look to both sides; fish-eye camera C and fish-eye camera D are used to look forward; fish-eye camera E and fish-eye camera F are used to look downward.

[0045] It should be noted that the more the number of wide-angle cameras, the greater the viewing angle difference. Exemplarily, such as Figure 4 shown, is a schematic diagram of the coverage range of the wide-angle camera provided by an embodiment of the present application. Combining Figure 2 and Figure 4 it can be seen that the coverage range of the wide-angle camera is greater than that of the RGB camera.

[0046] It should be noted that the RGB camera can only obtain images within the coverage range space of (60°, 120°) directly in front of the electronic device.

[0047] Currently, electronic devices mainly involve six degrees of freedom (6DoF) including translational and rotational motions. A larger Field of View (FoV) of the wide-angle camera on the electronic device makes tracking more stable. Based on this, the images captured by the wide-angle camera can be directly used in the Simultaneous Localization and Mapping (SLAM) algorithm. In addition, the gesture algorithm needs to cover the entire hand movement space of the user. Therefore, the images captured by the wide-angle camera can also be directly used in the gesture algorithm.

[0048] In addition, different from RGB cameras / RGBD cameras, the images captured by the wide-angle camera are grayscale images. Among them, grayscale images are single-channel images, while RGB cameras are 3-channel and RGBD cameras are 4-channel. Fewer channels mean that in the subsequent image processing process, the required calculations are simpler and the energy consumption can also be lower. Since grayscale images are inherently light-sensitive and the light source itself appears in the form of a large-area light spot, therefore, whether using traditional image processing or Convolutional Neural Network (CNN) processing, the calculations will be simpler, effectively extending the battery life of the device.

[0049] Since the electronic device comes with a wide-angle camera, there will be no additional burdens in terms of device cost, overall machine weight, price, etc.

[0050] It should be noted that the image acquisition of the wide-angle camera is usually encapsulated by a lower layer (such as the Hardware Abstraction Layer (HAL)), and then, through an interface, it is fed back to the calling layer. This usually goes through a series of system layer processes, such as different steps of Image Signal Processing (ISP), Asynchronous Timewarp (ATW), etc. These steps are usually encapsulated by a system abstraction layer to provide an interface in the form of a service for the upper layer to call. That is to say, in the process of the electronic device acquiring images captured by at least two wide-angle cameras, it is necessary to obtain images captured by at least two wide-angle cameras that have completed time synchronization through relevant interfaces of the system layer or some custom development methods, such as images captured by a multi-camera (taking the common 6-camera as an example) fisheye camera.

[0051] The following elaborates in detail on the method for determining ambient light information provided by the embodiments of the present application:

[0052] It should be noted that for the ambient light information determination method provided in the embodiments of the present application, the execution subject may be an ambient light information determination device, an electronic device (such as a wearable device), or a functional module in a wearable device, etc. In some embodiments of the present application, the wearable device executing the ambient light information determination method is taken as an example to illustrate the ambient light information determination method provided in the embodiments of the present application. Among them, the wearable device includes cameras at different positions.

[0053] Figure 5 FIG. shows a schematic flowchart of the ambient light information determination method provided in the embodiments of the present application. As Figure 5 shown, the ambient light information determination method provided in the embodiments of the present application may include the following steps 501 - step 503.

[0054] Step 501: The wearable device obtains the light source information corresponding to each of at least two images. The light source information corresponding to each image includes at least the first light source projection position information. One image is captured by one camera on the wearable device.

[0055] In the embodiments of the present application, the wearable device may include cameras at different positions. It can be understood that the above at least two images are images captured by at least two cameras at different positions in the wearable device, that is, at least two images correspond one-to-one to at least two cameras in the wearable device.

[0056] Among them, the light source information refers to the relevant data of the light source illuminating the scene in the image, and the light source is a point light source.

[0057] In the embodiments of the present application, the first light source projection position information is used to indicate the actual projection position of the light source in the image. It can be understood that the first light source projection position information refers to the actual projection position of the light source that can be clearly identified in the image.

[0058] It should be noted that if the wearable device includes m cameras, where m > 1, the wearable device can obtain the light source information corresponding to m images. Since these cameras are all wide-angle cameras, and the images captured by wide-angle cameras cover a wider range, the light source information of each image is more abundant and accurate, providing data support for accurately determining the ambient light information subsequently.

[0059] Exemplarily, when the number of wide-angle cameras is 6, each wide-angle camera can capture one image. At this time, the wearable device can obtain the light source information corresponding to each of the 6 images.

[0060] In some embodiments, after the above step 501, the ambient light information determination method may further include one of the following implementation manners:

[0061] Implementation Method 1: The wearable device determines the first light source projection position information corresponding to each image based on the brightness information of each image.

[0062] Among them, the brightness information refers to the brightness values of each pixel point in the image. Since the image is a grayscale image, the brightness values of these pixel points are within the grayscale value range of (0, 255).

[0063] The following operations are performed for each image: In the process of the wearable device determining the first light source projection position information corresponding to the image, it can first analyze the brightness distribution of the image to obtain the brightness information of the image. Specifically, it can first analyze the brightness distribution of the image to obtain the brightness histogram corresponding to the image. By analyzing this brightness histogram, the brightness distribution of the image can be intuitively determined; then, the wearable device determines the image areas in the brightness histogram whose brightness exceeds the preset brightness threshold, and determines the center point of the highest brightness image area among these image areas as the first light source projection position information corresponding to the image. Among them, the highest brightness image area usually means the direct projection position of the light source.

[0064] It should be noted that the brightness histogram can show the number of times each brightness value appears in the image.

[0065] Optionally, the determination of the preset brightness threshold is the default of the wearable device system or pre-set by the user.

[0066] Specifically, the wearable device pre-labels the presence or absence of the light source position in the projection sample using a machine learning or deep learning model (such as CNN) to obtain the result and analyze the brightness distribution. The wearable device then analyzes the brightness distribution difference between the positions with light sources and the positions without light sources based on this result analysis of the brightness distribution, so as to obtain the brightness threshold information under different ambient light conditions, and determines this brightness threshold information as the preset brightness threshold.

[0067] Implementation Method 2: The wearable device determines the first light source projection position information corresponding to each image based on the shadow information of each image.

[0068] Among them, the shadow information is the image information of the shadow area in each image where the brightness value is less than the preset brightness threshold, indicating that the shadow information is darker than other areas in the image.

[0069] The following operations are performed for each image: When the wearable device determines the first light source projection position information corresponding to the image, it may first determine a shadow area with a brightness value less than a preset brightness threshold based on the histogram corresponding to the image, and use the image information of the shadow area as shadow information; then the wearable device determines the quantity of the shadow information: if the quantity is 1, the wearable device infers the general shape of the object that generates the shadow area based on the shape of the shadow area, and further determines the first light source projection position information corresponding to the image, or determines the first light source projection position information based on the relative position between the shadow area and the object that generates the shadow area; if the quantity is greater than 1, the wearable device uses the ray tracing method to determine the intersection of these shadow areas, and determines the intersection as the first light source projection position information corresponding to the image.

[0070] Among them, the ray tracing method means that the wearable device simulates light rays starting from the edges or center points of these shadow areas, and traces back along possible light ray paths to obtain the points where the light rays may intersect, and further obtains the intersection of these shadow areas.

[0071] Optionally, when the quantity of the shadow information is greater than 1, the wearable device may use Bayesian estimation or Kalman filtering to perform fusion optimization of the light source direction to obtain the first light source projection position information with higher accuracy.

[0072] Among them, Bayesian estimation can comprehensively consider multiple pieces of shadow information, and gradually approach the real light source direction by continuously updating the belief of the parameters to obtain the first light source projection position information with higher accuracy.

[0073] Kalman filtering can comprehensively consider the shadow information at multiple moments, and gradually approach the real light source direction by continuously performing prediction and update steps to obtain the first light source projection position information with higher accuracy.

[0074] Step 502: The wearable device determines the second light source projection position information corresponding to each image based on the first light source projection position information corresponding to each image.

[0075] In the embodiment of the present application, the second light source projection position information is used to indicate the projection position of the light source on the camera corresponding to the image, that is, the second light source projection position refers to the projection position of the light source on the camera corresponding to the image, which is the estimated projection position of the light source on the camera corresponding to the image, and can be called the estimated light source projection position information corresponding to the image.

[0076] Since at least two cameras are located at different positions of the wearable device, the first light source projection position information corresponding to the images captured by these at least two cameras is also different. That is, the light source positions corresponding to the images captured by these at least two cameras are also different. At this time, the wearable device determines the second light source projection position information corresponding to each image based on the first light source projection position information corresponding to each image, providing data support for subsequent determination of more accurate ambient light information.

[0077] In some embodiments, the light source information corresponding to each image further includes: light source intensity information and light source orientation information; combined Figure 5 , as Figure 6 shown, step 502 above can be specifically implemented through the following step 502a.

[0078] Step 502a: The wearable device determines the second light source projection position information corresponding to each image based on the first light source projection position information, light source intensity information, and light source orientation information corresponding to each image.

[0079] Among them, the light source intensity information describes the brightness or luminous ability of the light source, which refers to the total amount of light emitted by the light source within a certain period of time and can be measured by luminous flux. The light source intensity information not only determines the brightness of the light, but also affects the distance and range of the light propagation in space.

[0080] The light source orientation information refers to the direction of the light emitted by the light source, that is, the direction or angle of the light source relative to the object in the image. The light source orientation information determines how the light irradiates the object, as well as the uniformity and directionality of the light on the object surface.

[0081] For each image, the following operations are performed: After the wearable device determines the first light source projection position information, light source intensity information, and light source orientation information corresponding to the image, it can integrate the first light source projection position information, light source intensity information, and light source orientation information to obtain the second light source projection position information corresponding to the image.

[0082] Optionally, the wearable device determines the second light source projection position information corresponding to each image based on the first light source projection position information, light source intensity information, and light source orientation information corresponding to each image, which may include: The wearable device obtains the internal parameters and external parameters of the camera corresponding to each image; the wearable device determines the second light source projection position information corresponding to each image based on the first light source projection position information, light source intensity information, light source orientation information, the internal parameters and external parameters of the corresponding camera.

[0083] Among them, the internal parameters refer to the imaging parameters generated before the camera leaves the factory. Optionally, the internal parameters may include: focal length f, principal point (cx, cy), distortion parameters (k1, k2, k3, k4...), etc. Using the internal parameters, the camera coordinates can be converted to pixel coordinates.

[0084] The external parameters refer to the position and orientation conversion relationship between different cameras, usually including two parts: a rotation matrix and a translation vector. Using the external parameters, coordinate transformation between different cameras can be performed, or the conversion between the world coordinate system and the camera coordinate system can be completed.

[0085] The following operations are performed for each image: After the wearable device determines the internal parameters and external parameters of the camera corresponding to the image, it can integrate the first light source projection position information, light source intensity information, light source orientation information, the internal parameters and external parameters of the corresponding camera to obtain the second light source projection position information corresponding to the image.

[0086] It should be noted that the internal parameters and external parameters of the above-mentioned camera are usually calibrated before the wearable device leaves the factory and are finally saved in a certain system file. The wearable device needs to use file reading or other methods to obtain the required internal parameters and external parameters of the camera.

[0087] In some embodiments, before the above step 502a, the ambient light information determination method may further include step 504 and step 505:

[0088] Step 504: The wearable device determines the light source orientation information corresponding to each image based on the shadow information of each image.

[0089] Step 505: The wearable device determines the light source intensity information corresponding to each image based on the light spot information of each image.

[0090] Optionally, the light spot information may include: characteristics such as the brightness, size, shape, and distribution of the light spot, and these characteristics can reflect the light source intensity information. Among them, brightness is the key characteristic reflecting the light source intensity.

[0091] After the wearable device determines the shadow information and light spot information of each image, since the direction of the shadow information is usually opposite to the incident direction of the light source, the shadow information of each image can be analyzed to obtain the light source orientation information corresponding to each image, and based on the mapping relationship between the light spot information and the light source intensity information, the light spot information of each image can be analyzed to obtain the light source intensity information corresponding to each image.

[0092] It should be noted that the timing for the wearable device to determine the light source orientation information and light source intensity information corresponding to each image is not limited.

[0093] In some embodiments, the wearable device determines the first light source projection position information corresponding to each image based on the shadow information of each image, which may include one of the following implementation manners:

[0094] Implementation manner 1: When the shadow area of any image among at least two images is the area projected by a second object, the wearable device determines the light source orientation information corresponding to any image based on the shadow area of any image and the second object.

[0095] Among them, the second object can be called a regular object, which refers to an object that can be described and replicated by a certain clear rule or standard in terms of appearance, shape, structure or specifications, such as a cube.

[0096] When the shadow area of any image among at least two images is the area projected by a second object, the wearable device can determine the light source orientation information corresponding to any image based on the relative position between the shadow area of any image and the second object.

[0097] Optionally, in the process of the wearable device determining the light source orientation information corresponding to any image based on the shadow area of any image and the second object, the wearable device can first use the Sobel operator to accurately determine the boundaries of the shadow area of any image and the second object; then, according to the similarity between the boundary of the shadow area of any image and the boundary of the second object, determine the light source orientation information corresponding to any image, and the accuracy of this light source orientation information is also relatively high.

[0098] Specifically, if the boundaries of the shadow area and the second object are highly similar in shape and direction, it indicates that the light source is located in the vertical direction of the boundary of the second object. Otherwise, it indicates that the light source is not located in the vertical direction of the boundary of the second object.

[0099] Among them, the Sobel operator is an edge detection algorithm that can detect edges by using the extreme values of the first-order derivative of the image grayscale. For the sake of simplifying the calculation, a simplified Sobel operator can be used to only calculate the gradients in the horizontal and vertical directions to achieve edge detection.

[0100] Implementation manner 2: When the projection area of any image among at least two images is the area projected by a first object, the wearable device determines the light source orientation information corresponding to any image based on the projection illumination information of the target key point positions in the projection area of any image.

[0101] In the embodiments of the present application, the target key point positions in the projection area of the above-mentioned any image are the key point positions in the projection area of the any image whose confidence information is greater than the preset confidence threshold.

[0102] In an embodiment of the present application, when the projection area of any one of at least two images is the area after the projection of the first object, the wearable device may determine the light source orientation information corresponding to any one of the images based on the average value of the projection light information of the target key point positions in the projection area of any one of the images. Optionally, the average value here may be any one of the arithmetic mean, weighted mean, geometric mean, mean square error, etc. Specifically, it may be determined according to actual usage requirements, and the embodiments of the present application do not limit it.

[0103] Exemplarily, the first object may be a hand, and the hand movement is natural and continuous. The projection area of any one of the above images may be the hand projection area of any one of the images.

[0104] In an embodiment of the present application, the above target key point positions may be understood as the key point positions that are not occluded in the projection area of any one of the above images, that is, the key point positions with confidence information greater than the preset confidence threshold may indicate that the key point positions are not occluded key point positions.

[0105] It should be noted that the occluded key point positions in the projection area may also be referred to as "self-occluded key points" in the projection area. For the hand projection area, in gesture key point detection, when a certain finger is occluded by another finger, the occluded key point may be called a "self-occluded key point", that is, when a finger causes some joints to be occluded by other fingers due to its own posture (such as making a fist or crossing), these points that are not visible in the projection are called self-occluded key points.

[0106] For example, when the thumb is occluded by the index finger, the proximal interphalangeal joint of the thumb may be completely invisible, and the point of the proximal interphalangeal joint of the thumb is a point that is not visible in the projection and is called a self-occluded key point. Or, when the little finger is occluded by the ring finger, its distal interphalangeal joint may be hidden, and the point of the distal interphalangeal joint is a point that is not visible in the projection and is called a self-occluded key point. Or, when making a fist, the distal interphalangeal joint of the middle finger may be occluded by the ring finger, and the point of the distal interphalangeal joint of the middle finger is a point that is not visible in the projection and is called a self-occluded key point. Or, when making an "OK" gesture, the fingertips of the ring finger and the little finger may be occluded by the thumb, and the points of the fingertips of the ring finger and the little finger are points that are not visible in the projection and are called self-occluded key points.

[0107] Similarly, the above unoccluded key point positions refer to the joint points that are clearly visible and not occluded by other objects (such as a certain finger) in the two-dimensional projection, that is, the key points are not occluded at all in the projection. For example, when the palm is stretched out flat, the fingertips and interphalangeal joints are clearly visible.

[0108] In the embodiments of the present application, when the projection area of any image among at least two images is the area after the projection of the first object, since the wearable device performs gesture operations for each frame, the direction of ambient light, that is, the determination of the light source direction information, can be added to the gesture model. The wearable device calculates the projection light information of the key point positions in the projection area of any image output by the gesture model to obtain the light source direction information corresponding to any image.

[0109] Among them, the gesture model generally refers to a computer model for identifying and analyzing the hand projection area, which can receive the images captured by the wearable device as input and output the projection light information of the key point positions in the hand projection area.

[0110] Optionally, the number of key point positions is at least two.

[0111] Exemplarily, as Figure 7 shown, it is a schematic diagram of the key point positions in the hand projection area provided by the embodiments of the present application. When the shadow area of any image is the area after the projection of the hand, the wearable device can output the projection light information of 21 key point positions in the hand projection area through the gesture model, and perform weighted aggregation on the projection light information of these 21 key point positions to obtain the light source direction information corresponding to any image.

[0112] Optionally, the projection light information can be calculated using the first formula.

[0113] Among them, the first formula is: V world =∑V i , V i >L;

[0114] V world represents the projection light information; Vi represents the projection light information of the i-th key point position, which is a vector; V i >L means that the confidence information of the i-th key point position is greater than L, that is, it means that the i-th key point position is not occluded. L represents the preset confidence threshold, and the confidence information of the i-th key point position can also be expressed as C i . Therefore, ∑V i represents the sum of the projection light information of the unoccluded key point positions among all the key point positions (such as 21 key point positions) in the hand projection area.

[0115] In the gesture model, the projection light information V of the key point positions will be output incidentally. At the same time, there will be confidence information Conf for the 21 key point positions in the hand projection area, and Conf is used to identify the occlusion relationship. For the unoccluded key point positions, the projection light information of these key point positions can be averaged to obtain the light source direction information, and the light source direction information can be expressed as: N represents the number of key point positions that are not occluded.

[0116] Optionally, the first projection position information and the light source orientation information corresponding to each image can also be implemented by using highlight detection, threshold segmentation, morphological analysis, brightness gradient analysis, etc. of traditional vision, or by using algorithms based on deep learning.

[0117] Optionally, after the above step 502 and before step 503, the ambient light information determination method may further include: the wearable device filters the abnormal second light source projection position information through the Random Sample Consensus (RANSAC).

[0118] Among them, RANSAC is a robust parameter estimation method, which can effectively avoid the influence of outliers and improve the robustness of the algorithm.

[0119] It should be noted that in the case where a certain second light source projection position information is an outlier, the corresponding first light source projection position information of the second light source projection position information can also be filtered to ensure the consistency of subsequent data processing.

[0120] Step 503: The wearable device determines the ambient light information based on the first light source projection position information and the second light source projection position information corresponding to at least two images.

[0121] Since there are differences in the determined light source positions in at least two cameras for the same light source, the first light source projection position information and the second light source projection position information corresponding to at least two images can be iteratively solved and calculated to obtain more accurate ambient light information. That is to say, the wearable device can finally obtain more accurate ambient light information by combining the first light source projection position information and the second light source projection position information corresponding to different images under the collaborative cooperation of at least two cameras.

[0122] In some embodiments, in combination with Figure 5 , such as Figure 8 shown, the above step 503 can be specifically implemented through the following steps 503a - 503c.

[0123] Step 503a: The wearable device calculates a loss value based on the first light source projection position information and the second light source projection position information corresponding to at least two images.

[0124] In some embodiments, the above step 503a may specifically include: the wearable device determines the projection deviation value corresponding to each image based on the first light source projection position information and the second light source projection position information corresponding to at least two images; the wearable device calculates a loss value based on the projection deviation value corresponding to each image.

[0125] During the process of calculating the loss value by the wearable device, the projection deviation value corresponding to each image can be determined first based on the first light source projection position information and the second light source projection position information corresponding to each image. In this way, for m images, the wearable device can determine m projection deviation values. Then, the wearable device sums the squares of the projection deviation values corresponding to each image to obtain the loss value.

[0126] Exemplarily, the loss value can be calculated using the second formula.

[0127] Among them, the second formula is:

[0128] L represents the loss value, which is the sum of the squares of the projection deviation values corresponding to each image; N represents the total number of cameras; represents the projection position of the light source Light on the i-th camera, that is, the second light source projection position information corresponding to the i-th image collected by the i-th camera; P Camerai represents the first light source projection position corresponding to the i-th image; represents the projection deviation value corresponding to the i-th image.

[0129] Step 503b: The wearable device determines the actual position information of the light source based on the loss value.

[0130] During the process of the wearable device determining the actual position information of the light source based on the loss value, the current position information of the light source can be continuously adjusted. At this time, the images collected by different cameras are changing in real time, and the first light source projection position and the second light source projection position corresponding to each image are also changing in real time. Therefore, in the iterative solution process, the first light source projection position and the second light source projection position corresponding to each image of the same light source in the images collected by different cameras can be repeatedly calculated through the iterative process, so that the sum of the squares of the projection deviation values corresponding to each image is minimized, that is, the minimum loss can be determined. At this time, the wearable device can determine the current position information of the light source corresponding to the minimum loss value as the actual position information of the light source.

[0131] In this way, even if at least two cameras are located at different positions of the wearable device, the difference in the first light source projection position information corresponding to the images collected by these at least two cameras is extremely small, or even there is no difference.

[0132] Step 503c: The wearable device determines the ambient light information based on the actual position information of the light source.

[0133] The wearable device can fuse the light intensity information corresponding to each image based on the actual position information of the light source to obtain the fused light intensity information, and fuse the light orientation information corresponding to each image to obtain the fused light orientation information. The wearable device obtains environmental light information with relatively high accuracy based on the actual position information of the light source, the fused light intensity information, and the fused light orientation information.

[0134] Optionally, after determining the actual position information of the light source, the method for determining the environmental light information may further include: The wearable device converts the actual position information of the light source from the camera coordinate system to the world coordinate system to obtain new actual position information.

[0135] Exemplarily, the new actual position information can be calculated using the third formula.

[0136] Wherein, the third formula is:

[0137] Light world represents the new actual position information; represents the rotation matrix of the main camera; Light Camera represents the actual position information of the light source; t represents the translation vector of the main camera.

[0138] It should be noted that when rendering at the upper layer, the coordinate values in the world coordinate system (usually the SLAM coordinate system) need to be obtained. Therefore, the actual position information of the light source needs to be converted. The entire conversion process is relatively simple and general. For at least two cameras, usually a main camera is set. At this time, based on the actual position information of the light source in the camera coordinate system, the conversion of the world coordinate system can be completed through the external parameters of the main camera. Then, the light source position information, light source intensity information, and light source orientation information in the world coordinate system are fed back to the upper-layer caller through the Application Programming Interface (API) or service.

[0139] In an embodiment of the present application, the method for determining ambient light information is applied to a wearable device. The wearable device includes cameras at different positions. The wearable device acquires the light source information corresponding to each of at least two images. The light source information corresponding to each image includes at least the first light source projection position information. One image is captured by one camera on the wearable device. Based on the first light source projection position information corresponding to each image, the second light source projection position information corresponding to each image is determined. Based on the first light source projection position information and the second light source projection position information corresponding to at least two images, the ambient light information is determined. In this solution, since the cameras on the wearable device have a wide field of view, the captured images are more abundant and accurate. By analyzing each of the at least two images, the first light source projection position information and the second light source projection position information corresponding to the at least two images with relatively high accuracy can be determined. Also, since there are differences in the actual light source position information corresponding to different images in the wearable device, the wearable device can determine the ambient light information based on the actual light source projection position information (i.e., the first light source projection position information) and the estimated light source projection position information (i.e., the second light source projection position information) corresponding to each image. That is, the ambient light information determined in this solution takes into account the differences in the actual light source position information corresponding to different images, and the obtained ambient light information has relatively high accuracy.

[0140] It should be noted that the above-mentioned various method embodiments, or various possible implementation manners in each method embodiment, can be executed independently, or any two or more of them can be combined and executed. Specifically, it can be determined based on actual usage requirements. The embodiments of the present application do not limit this.

[0141] In the embodiment of the present application, the execution subject of the method for determining ambient light information can be an ambient light information determination device. In the embodiment of the present application, taking the ambient light information determination device executing the method for determining ambient light information as an example, the ambient light information determination device provided by the embodiment of the present application is described. Among them, the ambient light information determination device is applied to an electronic device, and the electronic device includes cameras at different positions.

[0142] Figure 9 The structural schematic diagram of the ambient light information determination device provided by the embodiment of the present application is shown. As Figure 9 shown, the ambient light information determination device 900 provided by the embodiment of the present application may include: an acquisition module 901 and a processing module 902.

[0143] The acquisition module 901 is configured to acquire the light source information corresponding to each of at least two images. The light source information corresponding to each image includes at least the first light source projection position information. One image is captured by one camera on the electronic device.

[0144] A processing module 902 is configured to determine the second light source projection position information corresponding to each image based on the first light source projection position information corresponding to each image, where the first light source projection position information is used to indicate the actual projection position of the light source in the image, and the second light source projection position information is used to indicate the projection position of the light source on the camera corresponding to the image; and, determine the ambient light information based on the first light source projection position information and the second light source projection position information corresponding to at least two images.

[0145] In some embodiments, the light source information corresponding to each image further includes: light source intensity information and light source orientation information; the processing module 902 is specifically configured to determine the second light source projection position information corresponding to each image based on the first light source projection position information, the light source intensity information, and the light source orientation information corresponding to each image.

[0146] In some embodiments, the processing module 902 is specifically configured to calculate a loss value based on the first light source projection position information and the second light source projection position information corresponding to at least two images; determine the actual position information of the light source based on the loss value; and determine the ambient light information based on the actual position information of the light source.

[0147] In some embodiments, the processing module 902 is specifically configured to determine the projection deviation value corresponding to each image based on the first light source projection position information and the second light source projection position information corresponding to at least two images; and calculate a loss value based on the projection deviation value corresponding to each image.

[0148] In some embodiments, the processing module 902 is further configured to determine the first light source projection position information corresponding to each image based on the brightness information of each image; or determine the first light source projection position information corresponding to each image based on the shadow information of each image, where the shadow information is the image information of the shadow area in each image where the brightness value is less than a preset brightness threshold.

[0149] In some embodiments, the processing module 902 is further configured to, when the shadow area of any image among at least two images is the area projected by the second object, determine the light source orientation information corresponding to the any image based on the shadow area of the any image and the second object; or, when the projection area of any image among at least two images is the area projected by the first object, determine the light source orientation information corresponding to the any image based on the projection illumination information of the target key point positions in the projection area of the any image, where the target key point positions are the key point positions in the projection area of the any image whose confidence information is greater than a preset confidence threshold.

[0150] In the embodiments of the present application, since the camera on the electronic device has a wide field of view angle, the captured images are richer and more accurate. By analyzing each of at least two images, it is possible to determine the first light source projection position information and the second light source projection position information corresponding to at least two images with relatively high accuracy. Also, since there are differences in the actual light source position information corresponding to different images in the electronic device, the electronic device can determine the ambient light information based on the actual light source projection position information (i.e., the first light source projection position information) and the estimated light source projection position information (i.e., the second light source projection position information) corresponding to each image. That is, the ambient light information determined by this solution takes into account the differences in the actual light source position information corresponding to different images and obtains relatively accurate ambient light information.

[0151] The ambient light information determining device in the embodiments of the present application can be an electronic device or a component in the electronic device, such as an integrated circuit or a chip. The electronic device can be a terminal or other devices other than terminals. Exemplarily, the electronic device can be a mobile phone, a tablet computer, a laptop computer, a handheld computer, an in-vehicle electronic device, a Mobile Internet Device (MID), an Augmented Reality (AR) / Virtual Reality (VR) device, a robot, an electronic device, an Ultra-Mobile Personal Computer (UMPC), a netbook, or a Personal Digital Assistant (PDA), etc. It can also be a server, a Network Attached Storage (NAS), a Personal Computer (PC), a Television (TV), a teller machine, or a self-service machine, etc. The embodiments of the present application do not make specific limitations.

[0152] The ambient light information determining device in the embodiments of the present application can be a device with an operating system. The operating system can be the Android operating system, the iOS operating system, or other possible operating systems. The embodiments of the present application do not make specific limitations.

[0153] The ambient light information determining device provided by the embodiments of the present application can implement each process implemented by the above-mentioned ambient light information determining method embodiments. To avoid repetition, it will not be elaborated here.

[0154] Optionally, as Figure 10As shown in the figure, an embodiment of the present application further provides an electronic device 1000, including a processor 1001 and a memory 1002. A program or instruction that can run on the processor 1001 is stored on the memory 1002. When the program or instruction is executed by the processor 1001, it implements each step of the above-mentioned embodiment of the ambient light information determination method and can achieve the same technical effect. To avoid repetition, it will not be elaborated here.

[0155] It should be noted that the electronic devices in the embodiments of the present application include the above-mentioned mobile electronic devices and non-mobile electronic devices.

[0156] In the embodiments of the present application, the electronic device may be the wearable device in the above embodiments.

[0157] Figure 11 It is a schematic diagram of the hardware structure of an electronic device for implementing an embodiment of the present application.

[0158] The electronic device 1100 includes but is not limited to: a radio frequency unit 1101, a network module 1102, an audio output unit 1103, an input unit 1104, a sensor 1105, a display unit 1106, a user input unit 1107, an interface unit 1108, a memory 1109, and a processor 1110 and other components.

[0159] Those skilled in the art can understand that the electronic device 1100 may further include a power source (such as a battery) for supplying power to each component. The power source can be logically connected to the processor 1110 through a power management system, so as to implement functions such as management of charging, discharging, and power consumption management through the power management system. Figure 11 The structure of the electronic device shown in the figure does not constitute a limitation on the electronic device. The electronic device may include more or fewer components than shown in the figure, or combine certain components, or have different component arrangements, which will not be elaborated here.

[0160] Among them, the processor 1110 is used to obtain the light source information corresponding to each of at least two images. The light source information corresponding to each image includes at least first light source projection position information. An image is captured by a camera on the electronic device; based on the first light source projection position information corresponding to each image, determine the second light source projection position information corresponding to each image. The first light source projection position information is used to indicate the actual projection position of the light source in the image, and the second light source projection position information is used to indicate the projection position of the light source on the camera corresponding to the image; based on the first light source projection position information and the second light source projection position information corresponding to at least two images, determine the ambient light information.

[0161] Optionally, the light source information corresponding to each image further includes: light source intensity information and light source orientation information; the processor 1110 is specifically configured to determine the second light source projection position information corresponding to each image based on the first light source projection position information, the light source intensity information, and the light source orientation information corresponding to each image.

[0162] Optionally, the processor 1110 is specifically configured to calculate a loss value based on the first light source projection position information and the second light source projection position information corresponding to at least two images; determine the actual light source position information based on the loss value; and determine the ambient light information based on the actual light source position information.

[0163] Optionally, the processor 1110 is specifically configured to determine the projection deviation value corresponding to each image based on the first light source projection position information and the second light source projection position information corresponding to at least two images; and calculate a loss value based on the projection deviation value corresponding to each image.

[0164] Optionally, the processor 1110 is further configured to determine the first light source projection position information corresponding to each image based on the brightness information of each image; or determine the first light source projection position information corresponding to each image based on the shadow information of each image, where the shadow information is the image information of the shadow area in each image where the brightness value is less than a preset brightness threshold.

[0165] Optionally, when the shadow area of any image among at least two images is the area projected by the second object, the processor 1110 is further configured to determine the light source orientation information corresponding to the arbitrary image based on the shadow area of the arbitrary image and the second object; or when the projection area of any image among at least two images is the area projected by the first object, determine the light source orientation information corresponding to the arbitrary image based on the projection illumination information of the target key point positions in the projection area of the arbitrary image, where the target key point positions are the key point positions in the projection area of the arbitrary image whose confidence information is greater than a preset confidence threshold.

[0166] In the embodiments of the present application, since the camera on the electronic device has a wide field of view, the collected images are more abundant and accurate. By analyzing each of the at least two images, the first light source projection position information and the second light source projection position information corresponding to the at least two images with higher accuracy can be determined; and since there are differences in the actual light source position information corresponding to different images in the electronic device, the electronic device can determine the ambient light information based on the actual light source projection position information (i.e., the first light source projection position information) and the estimated light source projection position information (i.e., the second light source projection position information) corresponding to each image. That is, the ambient light information determined by this solution takes into account the differences in the actual light source position information corresponding to different images and obtains more accurate ambient light information.

[0167] It should be understood that in the embodiments of the present application, the input unit 1104 may include a Graphics Processing Unit (GPU) 11041 and a microphone 11042. The graphics processor 11041 processes the image data of static pictures or videos obtained by an image capturing device (such as a camera) in a video capture mode or an image capture mode. The display unit 1106 may include a display panel 11061, and the display panel 11061 may be configured in the form of a liquid crystal display, an organic light emitting diode, etc. The user input unit 1107 includes at least one of a touch panel 11071 and other input devices 11072. The touch panel 11071 is also referred to as a touch screen. The touch panel 11071 may include two parts: a touch detection device and a touch controller. The other input devices 11072 may include, but are not limited to, a physical keyboard, function keys (such as volume control keys, switch keys, etc.), a trackball, a mouse, and a joystick, which will not be elaborated here.

[0168] The memory 1109 can be used to store software programs and various data. The memory 1109 may mainly include a first storage area for storing programs or instructions and a second storage area for storing data. Among them, the first storage area may store an operating system, application programs or instructions required for at least one function (such as a sound playback function, an image playback function, etc.). In addition, the memory 1109 may include a volatile memory or a non-volatile memory, or the memory 1109 may include both a volatile memory and a non-volatile memory. Among them, the non-volatile memory may be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory may be a random access memory (RAM), a static random access memory (SRAM), a dynamic random access memory (DRAM), a synchronous dynamic random access memory (SDRAM), a double data rate synchronous dynamic random access memory (DDR SDRAM), an enhanced synchronous dynamic random access memory (ESDRAM), a synch link dynamic random access memory (SLDRAM), and a direct rambus random access memory (DRRAM). The memory 1109 in the embodiments of the present application includes but is not limited to these and any other suitable types of memories.

[0169] The processor 1110 may include one or more processing units; optionally, the processor 1110 integrates an application processor and a modem processor. Among them, the application processor mainly processes operations related to the operating system, user interface, and application programs, etc., and the modem processor mainly processes wireless communication signals, such as a baseband processor. It can be understood that the above modem processor may not be integrated into the processor 1110.

[0170] The embodiments of the present application also provide a readable storage medium, on which a program or instruction is stored. When the program or instruction is executed by a processor, it implements each process of the above embodiments of the ambient light information determination method and can achieve the same technical effects. To avoid repetition, it will not be elaborated here.

[0171] Among them, the processor is the processor in the electronic device described in the above embodiments. The readable storage medium includes computer-readable storage media such as computer read-only memory ROM, random access memory RAM, magnetic disks, or optical discs, etc.

[0172] Another embodiment of the present application provides a chip, which includes a processor and a communication interface. The communication interface is coupled to the processor. The processor is used to run programs or instructions to implement each process of the above embodiment of the ambient light information determination method, and can achieve the same technical effects. To avoid repetition, it will not be elaborated here.

[0173] It should be understood that the chip mentioned in the embodiments of the present application may also be referred to as a system-on-chip, system chip, chip system, or system-on-chip, etc.

[0174] The embodiments of the present application provide a computer program product. The program product is stored in a storage medium and is executed by at least one processor to implement each process of the above embodiment of the ambient light information determination method, and can achieve the same technical effects. To avoid repetition, it will not be elaborated here.

[0175] It should be noted that in this article, the term "including", "comprising" or any other variant thereof is intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements not only includes those elements, but also includes other elements not explicitly listed, or further includes elements inherent to such process, method, article or device. Without further limitation, an element defined by the statement "including a..." does not exclude the existence of additional identical elements in the process, method, article or device including that element. In addition, it should be pointed out that the scope of the methods and devices in the embodiments of the present application is not limited to performing functions in the order shown or discussed, and may also include performing functions in a substantially simultaneous manner or in a reverse order according to the functions involved. For example, the described methods may be performed in an order different from that described, and various steps may be added, omitted, or combined. In addition, the features described with reference to certain examples may be combined in other examples.

[0176] Through the description of the above embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus a necessary general hardware platform. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation. Based on such an understanding, the technical solution of the present application, in essence or the part that contributes to the prior art, can be embodied in the form of a computer software product. The computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions for causing a terminal (which can be a mobile phone, computer, server, or network device, etc.) to execute the methods described in various embodiments of the present application.

[0177] The embodiments of the present application have been described above in conjunction with the accompanying drawings. However, the present application is not limited to the above specific implementation manners. The above specific implementation manners are merely illustrative and not restrictive. Under the inspiration of the present application, those of ordinary skill in the art can also make many forms without departing from the purpose of the present application and the scope protected by the claims, and all of them belong to the protection scope of the present application.

Claims

1. An ambient light information determination method, characterized in that, Applied to an electronic device, the electronic device includes cameras at different positions, and the method includes: Obtain the light source information corresponding to each of at least two images, where the light source information corresponding to each image includes at least first light source projection position information, and one image is captured by one camera on the electronic device; Based on the first light source projection position information corresponding to each image, determine the second light source projection position information corresponding to each image, where the first light source projection position information is used to indicate the actual projection position of the light source in the image, and the second light source projection position information is used to indicate the projection position of the light source on the camera corresponding to the image; Based on the first light source projection position information and the second light source projection position information corresponding to the at least two images, determine the ambient light information.

2. The method according to claim 1, wherein The light source information corresponding to each image further includes: light source intensity information and light source orientation information; The determining the second light source projection position information corresponding to each image based on the first light source projection position information corresponding to each image includes: Based on the first light source projection position information, light source intensity information, and light source orientation information corresponding to each image, determine the second light source projection position information corresponding to each image.

3. The method according to claim 1, characterized in that, The determining the ambient light information based on the first light source projection position information and the second light source projection position information corresponding to the at least two images includes: Calculate a loss value based on the first light source projection position information and the second light source projection position information corresponding to the at least two images; Based on the loss value, determine the actual position information of the light source; Based on the actual position information of the light source, determine the ambient light information.

4. The method according to claim 3, wherein The calculating the loss value based on the first light source projection position information and the second light source projection position information corresponding to the at least two images includes: Based on the first light source projection position information and the second light source projection position information corresponding to the at least two images, determine the projection deviation value corresponding to each image; Calculate the loss value based on the projection deviation value corresponding to each image.

5. The method according to claim 2, wherein The method further includes: When the projection area of any one of the at least two images is the area after the projection of the first object, based on the projection illumination information of the target key point positions in the projection area of the any one image, determine the light source orientation information corresponding to the any one image, where the target key point positions are the key point positions in the projection area of the any one image whose confidence information is greater than the preset confidence threshold.

6. An ambient light information determining device, characterized in that, Applied to an electronic device, the electronic device includes cameras at different positions, and the device includes: an acquisition module and a processing module; The acquisition module is configured to obtain the light source information corresponding to each of at least two images, where the light source information corresponding to each image includes at least first light source projection position information, and one image is captured by one camera on the electronic device; The processing module is configured to determine the second light source projection position information corresponding to each image based on the first light source projection position information corresponding to each image, where the first light source projection position information is used to indicate the actual projection position of the light source in the image, and the second light source projection position information is used to indicate the projection position of the light source on the camera corresponding to the image; and to determine the ambient light information based on the first light source projection position information and the second light source projection position information corresponding to at least two images.

7. The device according to claim 6, characterized in that The light source information corresponding to each image further includes: light source intensity information and light source orientation information; The processing module is specifically configured to determine the second light source projection position information corresponding to each image based on the first light source projection position information, the light source intensity information, and the light source orientation information corresponding to each image.

8. The apparatus according to claim 6, wherein The processing module is specifically configured to calculate a loss value based on the first light source projection position information and the second light source projection position information corresponding to at least two images; Determine the actual light source position information based on the loss value; Determine the ambient light information based on the actual light source position information.

9. The apparatus according to claim 8, wherein The processing module is specifically configured to determine the projection deviation value corresponding to each image based on the first light source projection position information and the second light source projection position information corresponding to at least two images; and calculate the loss value based on the projection deviation value corresponding to each image.

10. The device according to claim 7, characterized in that, The processing module is further configured to, when the projection area of any one of the at least two images is the area projected by the first object, determine the light source orientation information corresponding to the any one image based on the projection illumination information of the target key point positions in the projection area of the any one image, where the target key point positions are the key point positions in the projection area of the any one image with a confidence information greater than a preset confidence threshold.

11. An electronic device, characterized in that, It includes a processor and a memory, where the memory stores a program or instruction that can run on the processor, and when the program or instruction is executed by the processor, the steps of the ambient light determination method according to any one of claims 1-5 are implemented.

12. A readable storage medium, characterized in that, A program or instruction is stored on the readable storage medium, and when the program or instruction is executed by the processor, the steps of the ambient light determination method according to any one of claims 1-5 are implemented.

Images