Flashlight control method, device, equipment, storage medium and chip

Abstract

The present application discloses a flash control method, apparatus, device, storage medium and chip, belonging to the technical field of image processing. The method includes: obtaining depth information of an environmental preview image collected at a first moment, where the environmental preview image is an image obtained by a camera focusing on a first target object in a shooting environment; obtaining a predicted relative position between the first target object and the camera at a second moment; determining a control parameter of the flash at the second moment based on the depth information and the predicted relative position, where the control parameter is used to control the brightness of the flash. The technical solution provided by the embodiments of the present application determines the control parameter of the flash at the shooting moment by combining the depth information of the environmental preview image and the predicted relative position between the target object and the camera at the shooting moment, so as to obtain a reasonable flash brightness at the shooting moment, without being limited to limited brightness levels, and solve the problems of overexposure or insufficient light filling during the shooting process.

Description

Technical Field

[0001] Embodiments of the present application relate to the field of image processing technology, and particularly to a flash control method, device, equipment, storage medium, and chip. Background Art

[0002] With the increasing frequency of use of shooting devices by users, the requirements of users for the shooting effects of shooting devices are also getting higher and higher.

[0003] When a user uses a shooting device to take a photo, if the light in the current shooting environment is insufficient, the user can turn on the flash to supplement the light in the current shooting environment, so that the shooting effect can be improved to a certain extent.

[0004] However, due to the diverse changes in the distance, position, angle, etc. between the shooting device and the object to be photographed, using the same flash brightness will cause problems of over-supplementing light or insufficient light supplement. Summary of the Invention

[0005] Embodiments of the present application provide a flash control method, device, equipment, storage medium, and chip, which can dynamically adjust the flash brightness at the shooting moment based on the depth information of the environmental preview image and the predicted relative position between the target object and the camera at the shooting moment, so as to obtain a reasonable flash brightness, thereby solving the problems of over-supplementing light or insufficient light supplement during the shooting process. The technical solutions are as follows:

[0006] According to one aspect of the embodiments of the present application, a flash control method is provided, and the method includes:

[0007] Obtain the depth information of the environmental preview image collected at the first moment, where the environmental preview image is an image obtained by the camera focusing on the first target object in the shooting environment;

[0008] Obtain the predicted relative position between the first target object and the camera at the second moment;

[0009] Based on the depth information and the predicted relative position, determine the control parameter of the flash at the second moment, where the control parameter is used to control the brightness of the flash.

[0010] According to one aspect of the embodiments of the present application, a flash control device is provided, and the device includes:

[0011] A depth information acquisition module, configured to obtain the depth information of the environmental preview image collected at the first moment, where the environmental preview image is an image obtained by the camera focusing on the first target object in the shooting environment;

[0012] A predicted position acquisition module, configured to acquire a predicted relative position between the first target object and the camera at a second moment;

[0013] A control parameter determination module, configured to determine a control parameter of the flash lamp at the second moment based on the depth information and the predicted relative position, where the control parameter is used to control the brightness of the flash lamp.

[0014] According to one aspect of the embodiments of the present application, there is provided a device, which includes a processor and a memory, where a computer program is stored in the memory, and the computer program is executed by the processor to implement the above-mentioned flash lamp control method.

[0015] According to one aspect of the embodiments of the present application, there is provided a computer-readable storage medium, where a computer program is stored in the storage medium, and the computer program is used to be executed by a processor to implement the above-mentioned flash lamp control method.

[0016] According to one aspect of the embodiments of the present application, there is provided a chip, which includes a programmable logic circuit and / or program instructions, and when the chip runs, it is used to implement the above-mentioned flash lamp control method.

[0017] According to one aspect of the embodiments of the present application, there is provided a computer program product or a computer program, where the computer program product or the computer program includes computer instructions, the computer instructions are stored in a computer-readable storage medium, and a processor reads and executes the computer instructions from the computer-readable storage medium to implement the above-mentioned flash lamp control method.

[0018] The technical solution provided by the embodiments of the present application can bring the following beneficial effects:

[0019] By determining the control parameter of the flash lamp at the shooting moment based on the depth information of the environmental preview image and the predicted relative position between the target object and the camera at the shooting moment, the dynamic adjustment of the flash brightness at the shooting moment is realized based on the depth information of the environmental preview image and the predicted relative position between the target object and the camera at the shooting moment, so as to obtain a reasonable flash brightness, rather than being limited to a limited brightness level (such as the flash brightness remains unchanged), thereby solving the problems of overexposure or insufficient light filling during the shooting process and improving the rationality of light filling.

[0020] In addition, since the predicted relative position between the target object and the camera at the shooting moment is combined to determine the control parameter, the influence of the change in the shooting distance on the shooting effect can be reduced in the case of a change in the shooting distance. At the same time, the obtained control parameter can also determine the lighting timing of the flash lamp, thereby improving the accuracy of light filling. Description of the Drawings

[0021] To more clearly illustrate the technical solutions in the embodiments of the present application, the following will briefly introduce the accompanying drawings required for the description of the embodiments. Obviously, the accompanying drawings in the following description are only some embodiments of the present application. For those of ordinary skill in the art, without creative efforts, other accompanying drawings can be obtained based on these drawings.

[0022] Figure 1 is a flowchart of a flash control method provided by an embodiment of the present application;

[0023] Figure 2 is a schematic diagram of the depth of field principle provided by an embodiment of the present application;

[0024] Figure 3 is a flowchart of a method for obtaining control parameters of a flash of a camera provided by an embodiment of the present application;

[0025] Figure 4 is a block diagram of a flash control device provided by an embodiment of the present application;

[0026] Figure 5 is a block diagram of a flash control device provided by another embodiment of the present application;

[0027] Figure 6 is a schematic diagram of the structure of a device provided by an embodiment of the present application. Detailed implementation manners

[0028] To make the objectives, technical solutions, and advantages of the present application clearer, the following will further describe the embodiments of the present application in detail with reference to the accompanying drawings.

[0029] For the method provided by the embodiments of the present application, the execution entity of each step may be a shooting device, and the shooting device may be any electronic device with a fill light shooting function. For example, the shooting device may be a device such as a camera, a video camera, a surveillance camera, etc. Optionally, the execution entity of each step may also be other devices communicatively connected to the shooting device. For example, the other device may be a server or a device such as a mobile phone, a PC (Personal Computer), or a tablet computer.

[0030] When the execution entity of each step is the shooting device, the shooting device collects an environmental preview image from the shooting environment, determines the control parameters of the flash based on the environmental preview image, and then controls the flash to flash according to the control parameters.

[0031] When the execution entity of each step is another device communicatively connected to the imaging device, the imaging device captures an environmental preview image from the shooting environment and sends the environmental preview image to the other device. The other device determines the control parameters of the flash based on the environmental preview image, and feeds back the control parameters to the imaging device. The imaging device controls the flash to emit light according to the control parameters.

[0032] Exemplarily, take a mobile phone communicatively connected to a camera as an example. When the light in the shooting environment is insufficient or backlit, the flash of the camera can be turned on to fill light for the target object when taking a picture of the target object, so as to obtain a better shooting effect. For example, the mobile phone calculates the depth information of the environmental preview image collected by the camera, as well as the predicted relative position of the target object and the camera at the shooting moment, and then determines reasonable control parameters (such as current parameters) of the flash at the shooting moment, and sends the control parameters to the camera. The camera controls the flash to emit light according to the control parameters to obtain a reasonable flash brightness, so that the brightness of the captured image falls within a relatively appropriate brightness range, and a better shooting effect is obtained. The method provided by the embodiments of the present application can solve the problems of overfilling light or insufficient filling light during the shooting process.

[0033] Since the method for the imaging device or another device communicatively connected to the imaging device to obtain the control parameters is the same, the method provided by the embodiments of the present application will be introduced below taking the imaging device as an example. It should be understood that the detailed description taking the imaging device as an example is only exemplary and explanatory, and does not limit the present application.

[0034] Please refer to Figure 1 , which shows a flowchart of a flash control method provided by an embodiment of the present application. The above-introduced imaging device is used as the execution entity of each step of this method for introduction. The method may include the following steps (101 to 103):

[0035] Step 101, obtaining the depth information of the environmental preview image collected at the first moment, where the environmental preview image is an image obtained by the camera focusing on the first target object in the shooting environment.

[0036] In the embodiments of the present application, an application program with a shooting function may be installed in the imaging device, such as a camera application program, a video shooting application program, etc. Through the above application program, components such as the camera and the flash in the imaging device can be turned on, so that the camera can capture the picture in the shooting environment, and the flash can fill light for the shooting environment (or the first target object in the shooting environment). Optionally, the environmental preview image refers to the picture captured by the camera from the shooting environment at the current perspective, and the environmental preview image can be displayed in the shooting preview interface of the imaging device for the user to refer to and select the shooting picture.

[0037] Optionally, the first target object may refer to any target object on which the user focuses through the camera. The target object may refer to any object or person in the shooting environment, such as flowers, grass, pedestrians, cats, etc., or may refer to a part of any object or a part of a person in the shooting environment, such as a person's eyes, a cat's tail, a tree's leaves, etc. The embodiments of the present application do not limit this here.

[0038] The first moment may be any moment during the shooting process, that is, after the camera is turned on, the camera captures the picture in the shooting environment in real time. Optionally, the first moment may be defined as the moment before the shooting moment (that is, the moment corresponding to the previous frame of the environmental preview picture of the captured image), and the second moment below may be defined as the shooting moment, and the shooting moment is determined by the user according to actual needs.

[0039] The depth information can be used to represent the distance between each position point corresponding to the image in the shooting environment and the focus of the camera. For example, the depth information of the environmental preview picture can be used to describe the distance between each position point corresponding to the environmental preview picture in the shooting environment and the focus of the camera. In the embodiments of the present application, the depth information may refer to the depth value of the first focal plane below, or may refer to the average value of the depth values of each pixel point in the depth map corresponding to the environmental preview picture. The embodiments of the present application do not limit this here. Among them, the first focal plane is a virtual plane passing through the focus and perpendicular to the optical axis of the camera. In response to the user focusing on the first target object, the picture corresponding to the first target object is projected onto the first focal plane. For example, referring to Figure 2 , in response to the user focusing on the first target object 201, the picture corresponding to the first target object 201 is projected onto the first focal plane, and the first focal plane is a virtual plane passing through the focus 202 and perpendicular to the optical axis of the camera.

[0040] In one example, the process of obtaining the depth information of the environmental preview picture may be as follows: obtain the depth map corresponding to the environmental preview picture; determine the depth value of the first focal plane based on the depth map, where the first focal plane is the focal plane containing the first target object; determine the depth information according to the depth value of the first focal plane.

[0041] Among them, the environmental preview picture can be subjected to conversion processing to obtain the depth map of the environmental preview picture. For example, the environmental preview picture can be subjected to conversion processing based on a deep neural network to obtain the depth features (i.e., depth values) of each pixel point in the environmental preview picture, and form the depth map of the environmental preview picture. Among them, the depth value can be used to represent the distance between the actual position point corresponding to the pixel point in the shooting environment and the focus of the camera.

[0042] When the photographing device is equipped with a 3D camera, while obtaining the environmental preview image, the depth map of the environmental preview image can be obtained. Optionally, the photographing device can construct the depth map of the environmental preview image in a preset depth map construction manner. For example, the depth map of the environmental preview image can be constructed based on the time from the emission to the return of the laser pulse; the depth map of the environmental preview image can be constructed by collecting images from multiple different positions based on binocular or multi-camera vision. The embodiments of the present application do not limit the depth map construction method herein.

[0043] Optionally, the process of obtaining the depth value of the first focal plane can be as follows: determine at least one pixel point of the first target object in the depth map; based on the depth values of the at least one pixel point, determine the depth value of the first target object; and determine the depth value of the first target object as the depth value of the first focal plane.

[0044] Among them, the environmental preview image can be first subjected to target recognition processing to identify the first target object and determine at least one pixel point of the first target object in the environmental preview image. Then, based on the at least one pixel point, the depth values of the at least one pixel point are determined from the depth map of the environmental preview image. Finally, an average calculation is performed on the depth values corresponding to the at least one pixel point to obtain the depth value of the first target object. It is also possible to directly determine at least one pixel point corresponding to the first target object and the depth values of the at least one pixel point from the depth map of the environmental preview image, and then perform an average calculation on the depth values corresponding to the at least one pixel point to obtain the depth value of the first target object. Optionally, the minimum depth value among the depth values corresponding to the at least one pixel point can also be determined as the depth value of the first target object, or the maximum depth value among the depth values corresponding to the at least one pixel point can be determined as the depth value of the first target object. The embodiments of the present application do not limit this herein.

[0045] Determine the depth value of the first target object as the depth value of the first focal plane, and determine the depth value of the first focal plane as the depth information of the environmental preview image.

[0046] Optionally, the process of obtaining the depth value of the first focal plane can also be as follows: obtain the focusing parameters of the camera at the first moment, and the focusing parameters include parameters such as depth of field, front depth of field, and rear depth of field. Among them, the depth of field is the sum of the front depth of field and the rear depth of field, and the dividing point between the front depth of field and the rear depth of field corresponds to the position of the focus. In this way, the depth value of the first focal plane can be obtained according to the proportion of the front depth of field or the rear depth of field in the depth of field. For example, refer to Figure 2, in response to the user focusing on the first target object 201, the image of the first target object 201 is projected onto the first focal plane. The position of the first target object 201 corresponding to the depth of field corresponds to the focal point 202, that is, it corresponds to the position of the first focal plane in the depth map. The depth value types included in the depth map of the environmental preview image are sorted in ascending order, and the depth value type corresponding to the ratio of the front depth of field to the depth of field is determined as the depth value of the first focal plane. For example, if the ratio of the front depth of field to the depth of field is 1 / 3, and the depth value types included in the depth map are: 1, 2, 3, 4, 5, and 6, then 2 can be used as the depth value of the first focal plane.

[0047] In a feasible example, the process of obtaining the depth information of the environmental preview image can also be as follows: Determine the second focal plane and the third focal plane based on the first focal plane; wherein, the second focal plane and the third focal plane are distributed on both sides of the first focal plane, and the distance between the second focal plane and the first focal plane is equal to the distance between the third focal plane and the first focal plane; Based on the image area of the first target object in the first focal plane, the image area of the second target object in the second focal plane, and the image area of the third target object in the third focal plane, determine the weight parameters corresponding to the first focal plane, the second focal plane, and the third focal plane respectively; Based on the weight parameters corresponding to the first focal plane, the second focal plane, and the third focal plane respectively, perform weighted summation on the depth value of the first focal plane, the depth value of the second focal plane, and the depth value of the third focal plane to obtain a comprehensive depth value; Determine the depth information according to the comprehensive depth value.

[0048] Exemplarily, referring to Figure 2 , within the range of the depth of field, the second focal plane and the third focal plane are determined respectively on the front and back sides of the first focal plane (i.e., based on the focal point 202), that is, the second target object and the third target object are the target objects that the user captures secondarily before and after the first target object 201. For example, the depth value of the second focal plane is 1, the depth value of the third focal plane is 3, the depth value of the first focal plane is 2, the image area of the second target object in the second focal plane is B, the image area of the third target object in the third focal plane is C, and the image area of the first target object 201 in the first focal plane is A. Then the weight parameter of the first focal plane is A / (A + B + C), the weight parameter of the second focal plane is B / (A + B + C), and the weight parameter of the third focal plane is C / (A + B + C). Then the depth information of the environmental preview image is 2A / (A + B + C) + B / (A + B + C) + 3C / (A + B + C) = (2A + B + 3C) / (A + B + C).

[0049] Optionally, according to actual requirements, the distance between the second focal plane and the first focal plane or the distance between the third focal plane and the first focal plane can be adjusted adaptively.

[0050] Thus, when focusing on the first target object, by combining the second target object and the third target object before and after the first target object to determine the depth information, the fill light effect of the secondary target object can be improved while ensuring the fill light effect of the first target object.

[0051] Step 102: Obtain the predicted relative position of the first target object and the camera at the second moment.

[0052] The predicted relative position is the position obtained by predicting the relative position of the first target object and the camera at the second moment. For example, by predicting the relative position of the first target object and the camera at the shooting moment, the predicted relative position of the first target object and the camera at the shooting moment is obtained.

[0053] In one example, the process of obtaining the predicted relative position can be as follows: Based on the depth values of the first target object at the first moment and at least one historical moment before the first moment, determine the predicted motion trend parameter of the first target object from the first moment to the second moment; Based on the depth value of the first target object at the first moment and the predicted motion trend parameter of the first target object, determine the predicted position of the first target object at the second moment; Based on the predicted position of the first target object at the second moment and the predicted position of the camera at the second moment, determine the predicted relative position of the first target object and the camera at the second moment.

[0054] Among them, the difference between the predicted position of the first target object at the second moment and the predicted position of the camera at the second moment can be determined as the predicted relative position of the first target object and the camera at the second moment.

[0055] Optionally, the process of determining the predicted position of the first target object at the second moment can be as follows: Based on the depth values of the first target object at the first moment and at least one historical moment before the first moment, determine the predicted motion trend parameter of the first target object from the first moment to the second moment; Based on the depth value of the first target object at the first moment and the predicted motion trend parameter of the first target object, determine the predicted position of the first target object at the second moment.

[0056] Optionally, the process of determining the predicted position of the camera at the second moment can be as follows: Based on the motion parameters of the camera at the first moment and at least one historical moment before the first moment, determine the predicted motion trend parameter of the camera from the first moment to the second moment; Based on the predicted motion trend parameter of the camera, determine the predicted position of the camera at the second moment.

[0057] Among them, the predicted motion trend parameter is used to describe the predicted motion trend of an object, and the predicted motion parameter may include a predicted motion direction, a predicted motion distance, a predicted motion speed, etc. The motion parameters of the camera can be obtained according to the data collected by motion sensors, such as a G-sensor (acceleration sensor), a Gyro-sensor (gyroscope sensor), etc.

[0058] Exemplarily, in the case of determining the predicted motion trend parameter by combining the first moment and the historical moments adjacent to the first moment, based on the depth difference between the depth value of the first target object at the first moment and the depth values of the historical moments adjacent to or near the first moment, the motion trend of the first target object is determined, and then the predicted motion trend parameter of the first target object from the first moment to the second moment is determined. In the case of determining the predicted motion trend parameter by combining the first moment and multiple historical moments before the first moment, the first moment and the multiple historical moments before the first moment are sorted in chronological order, and the depth differences between adjacent moments in the sorting are obtained to obtain a depth difference sequence. Based on the depth difference sequence, the motion trend of the first target object is determined, and thus the predicted motion trend parameter of the first target object from the first moment to the second moment is determined. By using the same method, the predicted motion trend parameter of the camera from the first moment to the second moment is obtained.

[0059] Based on the depth value of the first target object at the first moment, the relative position between the first target object and the camera at the first moment is determined. For example, with the position of the camera as a reference, the position of the first target object at the first moment is determined, and then combined with the predicted motion trend parameter of the first target object from the first moment to the second moment, the predicted position of the first target object at the second moment can be determined. With the position of the camera as a reference, combined with the predicted motion trend parameter of the camera from the first moment to the second moment, the predicted position of the camera at the second moment can be determined, and then based on the predicted position of the first target object at the second moment and the predicted position of the camera at the second moment, the predicted relative position can be obtained.

[0060] Step 103: Based on the depth information and the predicted relative position, determine the control parameter of the flash at the second moment, where the control parameter is used to control the brightness of the flash.

[0061] The above control parameter may refer to a current parameter, a voltage parameter, etc. Optionally, the control parameter is positively correlated with the brightness of the flash.

[0062] In one example, the process of determining the control parameter of the flash at the second moment can be as follows: Based on the depth information and the predicted relative position, the parameter determination model determines the control parameter of the flash at the second moment; wherein, the parameter determination model is a mathematical model or a machine learning model constructed based on multiple sets of experimental data, and each set of experimental data includes the corresponding relationship between a set of depth information, the predicted relative position, and the control parameter under the optimal shooting effect.

[0063] When the parameter determination model is a mathematical model, the parameter determination model can be a mathematical model with the depth information as the first variable, the predicted relative position as the second variable, and the control parameter as the output variable. Optionally, the relationship between the depth information, the predicted relative position, and the control parameter can be a linear relationship or a non-linear relationship, which is not limited in the embodiments of the present application.

[0064] When the parameter determination model is a machine learning model, first, the parameter determination model is trained with the depth information and the predicted relative position as training samples and the control parameter under the optimal shooting effect as labels to obtain a trained parameter determination model, and then the trained parameter determination model is applied to the shooting device. By inputting the depth information and the predicted relative position into the trained parameter determination model, the control parameter of the flash at the second moment can be obtained.

[0065] Among them, the shooting effect can be adjusted with the depth information and the predicted relative position under different settings to obtain the corresponding relationship between the depth information, the predicted relative position, and the control parameter under the optimal shooting effect. Further, interpolation processing of the control parameter can be performed according to the changes in the depth information and the predicted relative position, so that the control parameter is a continuous value rather than being limited to a finite number of discontinuous values, to improve the above experimental data. Optionally, the brightness value of the ambient light in the shooting environment can also be combined to determine the control parameter. For example, a parameter determination model is constructed based on multiple sets of experimental data including the corresponding relationship between the depth information, the predicted relative position, the brightness value of the ambient light, and the control parameter under the optimal shooting effect. In this way, a more reasonable control parameter can be obtained, so that the flash brightness is closer to the shooting environment, further improving the shooting effect.

[0066] Optionally, the control parameter of the flash at the second moment can also be obtained by querying according to a mapping table based on the depth information and the predicted relative position. The mapping table includes the corresponding relationship between the depth information, the predicted relative position, and the control parameter under the optimal shooting effect.

[0067] Exemplarily, after obtaining the control parameter of the flash at the shooting moment, in response to the user's shooting operation, the flash is controlled to flash according to the control parameter to fill light for the first target object, and a captured image focused on the first target object is obtained.

[0068] In one example, the flash of the flash lamp can also be controlled based on the control parameters in combination with the flash mode of the flash lamp. The determination process of the flash mode can be as follows: Obtain the characteristic parameters of the shooting environment, where the characteristic parameters include at least one of the following: environmental depth information, environmental type, and ambient light parameters; Based on the characteristic parameters of the shooting environment, determine the predicted convergence duration, where the predicted convergence duration is used to represent the optimal duration required for the camera to capture the first target object with the assistance of the flash lamp; Based on the predicted convergence duration, determine the flash mode of the flash lamp, where the flash mode is used to control the duration of a single flash of the flash lamp.

[0069] Among them, the environmental depth information can refer to the average value of the depth values of all pixel points in the image corresponding to the shooting environment relative to the camera. The environmental type can be types such as daytime, rainy day, night, etc. The ambient light parameters can include parameters such as the intensity and brightness of the ambient light.

[0070] Optionally, the AEC (Acoustic Echo Cancellation) algorithm can be used to determine the predicted convergence duration based on the characteristic parameters of the shooting environment, so as to dynamically obtain the flash mode of the flash lamp.

[0071] Optionally, in the case of controlling the flash of the flash lamp in combination with the control parameters and the flash mode, determine the magnitude of the current parameter based on the control parameters, and determine the continuous output duration of the current parameter based on the flash mode. Among them, the magnitude of the current parameter controls the flash brightness, and the continuous output duration of the current parameter controls the duration of a single flash of the flash lamp.

[0072] In summary, the technical solution provided by the embodiments of the present application determines the control parameters of the flash lamp at the shooting moment based on the depth information of the environmental preview image and the predicted relative position between the target object and the camera at the shooting moment, realizing the dynamic adjustment of the flash brightness at the shooting moment based on the depth information of the environmental preview image and the predicted relative position between the target object and the camera at the shooting moment, so as to obtain a reasonable flash brightness, rather than being limited to a limited brightness level (such as the flash brightness remaining unchanged), thereby solving the problem of excessive fill light or insufficient fill light during the shooting process and improving the rationality of fill light.

[0073] In addition, since the predicted relative position between the target object and the camera at the shooting moment is combined to determine the control parameters, the influence of the change in the shooting distance on the shooting effect can be reduced in the case of a change in the shooting distance. At the same time, the obtained control parameters can also determine the lighting timing of the flash lamp, thereby improving the accuracy of fill light.

[0074] In addition, by combining the brightness value of the ambient light to determine the control parameter, for different shooting environments, the dynamic adjustment of the control parameter can be realized, so that the flash brightness matches the shooting environment, and the rationality of fill light is improved.

[0075] In an exemplary embodiment, the acquisition process of the control parameter of the flash lamp during the shooting process of the camera is taken as an example for introduction. Please refer to Figure 3 , which shows the flowchart of the method for acquiring the control parameter of the flash lamp of the camera provided by an embodiment of the present application.

[0076] Step 301, in response to the user's operation of turning on the shooting function of the camera, the camera turns on the shooting function and displays a shooting preview interface, and the shooting preview interface can display an environmental preview image corresponding to the current shooting environment. Optionally, in the shooting preview interface, the user can select a shooting mode, such as an image shooting mode or a video shooting mode.

[0077] Step 302, in response to the user's focusing operation on the first target object in the shooting environment, the camera captures an environmental preview image focused on the first target object and displays the environmental preview image focused on the first target object in the shooting preview interface. Among them, the environmental preview image can refer to the image data in RAW (unprocessed) format.

[0078] Step 303, the camera performs recognition processing on the environmental preview image, recognizes the first target object, and at least one pixel point of the first target object in the environmental preview image.

[0079] Step 304, perform conversion processing on the environmental preview image to obtain the depth map of the environmental preview image, obtain the depth values of at least one pixel point from the depth map of the environmental preview image based on at least one pixel point, determine the average value of the depth values of at least one pixel point as the depth value of the first target object, and determine the depth value of the first target object as the depth value of the first focal plane. Among them, the first focal plane refers to a virtual plane passing through the current focus of the camera lens and perpendicular to the optical axis of the camera.

[0080] Optionally, the depth value of the first focal plane can also be obtained from the depth map of the environmental preview image based on the focusing parameter of the camera at the current moment. For example, the focusing parameters include parameters such as focus, depth of focus, depth of field, and front depth of field. Based on the ratio of the front depth of field to the depth of field, the depth value of the first focal plane is obtained from the depth map of the environmental preview image.

[0081] Step 305: Based on the depth value of the first target object at the current moment and the depth values at at least one historical moment before the current moment, determine the predicted motion trend parameters of the first target object from the current moment to the next moment. Based on the motion parameters of the camera at the current moment and the motion parameters at at least one historical moment before the current moment, determine the predicted motion trend parameters of the camera from the current moment to the next moment. Based on the depth value of the first target object at the current moment, determine the relative position between the first target object and the camera at the current moment. Then, combining the predicted motion trend parameters of the first target object from the current moment to the next moment and the predicted motion trend parameters of the camera from the current moment to the next moment, determine the predicted relative position between the first target object and the camera at the next moment.

[0082] Step 306: Based on the depth value of the first focal plane and the predicted relative position between the first target object and the camera at the next moment, determine the control parameters of the flash at the next moment.

[0083] Step 307: If the next moment is the shooting moment, control the flash to flash according to the control parameters of the flash at the next moment to fill light for the first target object, so as to assist the camera in shooting and obtain a photo with better shooting effect.

[0084] Step 308: In response to the user's shooting operation, the camera acquires a shooting image.

[0085] In an exemplary embodiment, the flash control method provided in the embodiments of the present application can also be applied to the HDR (High-Dynamic Range) image shooting scenario. During the process of performing different exposure processes, according to the depth information of different shooting areas, adjust the brightness of the flash (i.e., adjust the control parameters of the flash) respectively to fill light for different shooting areas with appropriate flash brightness, so as to obtain shooting images with better effects in different shooting areas. Then, perform synthesis processing on multiple shooting images to obtain a high-dynamic range image with better effects. Optionally, based on the depth information of different shooting areas, in combination with the predicted relative position between the target object and the camera in different shooting areas, the control parameters of the flash can be adjusted to obtain a better light filling effect.

[0086] In summary, the technical solution provided by the embodiments of the present application determines the control parameters of the flash at the shooting moment based on the depth information of the environmental preview image and the predicted relative position between the target object and the camera at the shooting moment, realizing the dynamic adjustment of the flash brightness at the shooting moment based on the depth information of the environmental preview image and the predicted relative position between the target object and the camera at the shooting moment, so as to obtain a reasonable flash brightness, rather than being limited to a limited number of brightness levels (such as the flash brightness remaining unchanged), thereby solving the problems of overexposure or insufficient fill light during the shooting process and improving the rationality of fill light.

[0087] In addition, since the predicted relative position between the target object and the camera at the shooting moment is combined to determine the control parameters, the influence of the change in the shooting distance on the shooting effect can be reduced in the case of a changing shooting distance. At the same time, the obtained control parameters can also determine the lighting timing of the flash, thereby improving the accuracy of fill light.

[0088] The following is an embodiment of the device of the present application, which can be used to execute the method embodiment of the present application. For the details not disclosed in the device embodiment of the present application, please refer to the method embodiment of the present application.

[0089] Please refer to Figure 4 , which shows a block diagram of a flash control device provided by an embodiment of the present application. The device has the function of implementing the above-mentioned flash control method example, and the function can be implemented by hardware or by hardware executing corresponding software. The device can be a shooting device or other devices communicatively connected to the shooting device. The device 400 may include: a depth information acquisition module 401, a predicted position acquisition module 402, and a control parameter determination module 403.

[0090] The depth information acquisition module 401 is configured to acquire the depth information of the environmental preview image acquired at the first moment, where the environmental preview image is an image obtained by the camera focusing on the first target object in the shooting environment.

[0091] The predicted position acquisition module 402 is configured to acquire the predicted relative position between the first target object and the camera at the second moment.

[0092] The control parameter determination module 403 is configured to determine the control parameters of the flash at the second moment based on the depth information and the predicted relative position, where the control parameters are used to control the brightness of the flash.

[0093] In an exemplary embodiment, the depth information acquisition module 401 is configured to:

[0094] Acquire the depth map corresponding to the environmental preview image;

[0095] Determine the depth value of the first focal plane based on the depth map, where the first focal plane is the focal plane containing the first target object;

[0096] Determine the depth information according to the depth value of the first focal plane.

[0097] In an exemplary embodiment, the depth information acquisition module 401 is further configured to:

[0098] Determine at least one pixel point of the first target object in the depth map;

[0099] Based on the depth values of the at least one pixel point, determine the depth value of the first target object;

[0100] Determine the depth value of the first target object as the depth value of the first focal plane.

[0101] In an exemplary embodiment, the depth information acquisition module 401 is further configured to:

[0102] Based on the first focal plane, determine a second focal plane and a third focal plane; wherein, the second focal plane and the third focal plane are distributed on both sides of the first focal plane, and the distance between the second focal plane and the first focal plane is equal to the distance between the third focal plane and the first focal plane;

[0103] Based on the image area of the first target object in the first focal plane, the image area of the second target object in the second focal plane, and the image area of the third target object in the third focal plane, determine the weight parameters corresponding to the first focal plane, the second focal plane, and the third focal plane respectively;

[0104] Based on the weight parameters corresponding to the first focal plane, the second focal plane, and the third focal plane respectively, perform weighted summation on the depth value of the first focal plane, the depth value of the second focal plane, and the depth value of the third focal plane to obtain a comprehensive depth value;

[0105] Determine the depth information according to the comprehensive depth value.

[0106] In an exemplary embodiment, the predicted position acquisition module 402 is configured to:

[0107] Based on the depth values of the first target object at the first moment and at least one historical moment before the first moment, determine the predicted position of the first target object at the second moment;

[0108] Determine the predicted position of the camera at the second moment based on the motion parameters of the camera at the first moment and at least one historical moment before the first moment;

[0109] Based on the predicted position of the first target object at the second moment and the predicted position of the camera at the second moment, determine the predicted relative position between the first target object and the camera at the second moment.

[0110] In an exemplary embodiment, the predicted position acquisition module 402 is further configured to:

[0111] Determine the predicted motion trend parameters of the first target object from the first moment to the second moment based on the depth values of the first target object at the first moment and at least one historical moment before the first moment;

[0112] Determine the predicted position of the first target object at the second moment based on the depth value of the first target object at the first moment and the predicted motion trend parameters of the first target object.

[0113] In an exemplary embodiment, the predicted position acquisition module 402 is further configured to:

[0114] Determine the predicted motion trend parameters of the camera from the first moment to the second moment based on the motion parameters of the camera at the first moment and at least one historical moment before the first moment;

[0115] Determine the predicted position of the camera at the second moment based on the predicted motion trend parameters of the camera.

[0116] In an exemplary embodiment, the control parameter determination module 403 is configured to:

[0117] Determine the control parameter of the flash at the second moment through a parameter determination model based on the depth information and the predicted relative position;

[0118] Wherein, the parameter determination model is a mathematical model or a machine learning model constructed based on multiple sets of experimental data, and each set of experimental data includes the corresponding relationship between a set of depth information, predicted relative position, and control parameters under the optimal shooting effect.

[0119] In an exemplary embodiment, referring to Figure 5 , the apparatus 400 further includes: a feature parameter acquisition module 404, a convergence duration determination module 405, and a flash mode determination module 406.

[0120] A feature parameter acquisition module 404 is configured to acquire feature parameters of the shooting environment, where the feature parameters include at least one of the following: environmental depth information, environmental type, and environmental light parameters.

[0121] A convergence duration determination module 405 is configured to determine a predicted convergence duration based on the feature parameters of the shooting environment, where the predicted convergence duration is used to represent an optimal duration required for the camera to capture the first target object with the assistance of the flash.

[0122] A flash mode determination module 406 is configured to determine a flash mode of the flash based on the predicted convergence duration, where the flash mode is used to control a duration of a single flash of the flash.

[0123] In summary, the technical solution provided in the embodiment of the present application determines control parameters of the flash at the shooting moment based on the depth information of the environmental preview image and the predicted relative position between the target object and the camera at the shooting moment, realizes dynamic adjustment of the flash brightness at the shooting moment based on the depth information of the environmental preview image and the predicted relative position between the target object and the camera at the shooting moment, so as to obtain a reasonable flash brightness, rather than being limited to a limited brightness level (such as a constant flash brightness), thereby solving the problem of overexposure or insufficient fill light during shooting and improving the rationality of fill light.

[0124] In addition, since the predicted relative position between the target object and the camera at the shooting moment is combined to determine the control parameters, the influence of the change in the shooting distance on the shooting effect can be reduced when the shooting distance changes. At the same time, the obtained control parameters can also determine the lighting timing of the flash, thereby improving the accuracy of fill light.

[0125] It should be noted that when the device provided in the above embodiment realizes its functions, only the above-mentioned division of each functional module is used for illustration. In actual applications, the above functions can be allocated to different functional modules according to needs, that is, the internal structure of the device is divided into different functional modules to complete all or part of the functions described above. In addition, the device provided in the above embodiment and the method embodiment belong to the same concept, and the specific implementation process is detailed in the method embodiment, which will not be elaborated here.

[0126] Please refer to Figure 6 , which shows a schematic structural diagram of a device 600 provided in an embodiment of the present application. The device 600 may be the above-mentioned shooting device or other devices communicatively connected to the shooting device, and the device 600 may be used to execute the above-mentioned flash control method. Specifically: The device 600 may include: a processor 601, a receiver 602, a transmitter 603, a memory 604, and a bus 605.

[0127] The processor 601 includes one or more processing cores. The processor 601 executes various functional applications and information processing by running software programs and modules.

[0128] The receiver 602 and the transmitter 603 can be implemented as a transceiver 606, and the transceiver 606 can be a communication chip.

[0129] The memory 604 is connected to the processor 601 through the bus 605.

[0130] The memory 604 can be used to store a computer program, and the processor 601 is used to execute the computer program to implement each step performed by the device in the above method embodiments.

[0131] In addition, the memory 604 can be implemented by any type of volatile or non-volatile storage device or a combination thereof. The volatile or non-volatile storage devices include, but are not limited to: RAM (Random-Access Memory), ROM (Read-Only Memory), EPROM (Erasable Programmable Read-Only Memory), EEPROM (Electrically Erasable Programmable Read-Only Memory), flash memory or other solid-state storage technologies, CD-ROM (Compact Disc Read-Only Memory), DVD (Digital Video Disc) or other optical storage, cassette tapes, magnetic tapes, disk storage or other magnetic storage devices. Among them:

[0132] The processor 601 is configured to obtain the depth information of the environmental preview image collected at the first moment, and the environmental preview image is an image obtained by the camera focusing on the first target object in the shooting environment;

[0133] Obtain the predicted relative position between the first target object and the camera at the second moment;

[0134] Based on the depth information and the predicted relative position, determine the control parameter of the flash at the second moment, and the control parameter is used to control the brightness of the flash.

[0135] For the details not described in detail in this embodiment, reference can be made to the introduction in the above embodiments, and details will not be repeated here.

[0136] An embodiment of the present application also provides a computer-readable storage medium, in which a computer program is stored, and the computer program is used to be executed by a processor of a device to implement the above-mentioned flash control method.

[0137] Optionally, the computer-readable storage medium may include: ROM (Read-Only Memory), RAM (Random-Access Memory), SSD (Solid State Drives), or optical discs, etc. Among them, the random access memory may include ReRAM (Resistance Random Access Memory) and DRAM (Dynamic Random Access Memory).

[0138] An embodiment of the present application also provides a chip, which includes programmable logic circuits and / or program instructions, and is used to implement the above-mentioned flash control method when the chip runs on a device.

[0139] An embodiment of the present application also provides a computer program product or a computer program. The computer program product or the computer program includes computer instructions, and the computer instructions are stored in a computer-readable storage medium. A processor of a device reads and executes the computer instructions from the computer-readable storage medium to implement the above-mentioned flash control method.

[0140] It should be understood that "a plurality" mentioned herein refers to two or more. "And / or" describes the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B may represent: A exists alone, A and B exist simultaneously, and B exists alone. The character " / " generally represents an "or" relationship between the associated objects before and after. In addition, the step numbers described herein only exemplarily show a possible execution sequence between steps. In some other embodiments, the above steps may not be executed in the order of the numbers. For example, two steps with different numbers are executed simultaneously, or two steps with different numbers are executed in the reverse order of the illustration. The embodiments of the present application do not limit this.

[0141] The above are only exemplary embodiments of the present application, and are not intended to limit the present application. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. A flash control method, characterized in that, the method includes: obtaining depth information of an environmental preview image collected at a first moment, where the environmental preview image is an image obtained by a camera focusing on a first target object in a shooting environment; determining a predicted motion trend parameter of the first target object from the first moment to a second moment based on depth values of the first target object at the first moment and at least one historical moment before the first moment; determining a predicted position of the first target object at the second moment based on the depth value of the first target object at the first moment and the predicted motion trend parameter of the first target object, where the depth value at the first moment is used to indicate the relative position between the first target object and the camera at the first moment; determining a predicted position of the camera at the second moment based on motion parameters of the camera at the first moment and at least one historical moment before the first moment; determining a predicted relative position between the first target object and the camera at the second moment based on the predicted position of the first target object at the second moment and the predicted position of the camera at the second moment; determining a control parameter of the flash at the second moment based on the depth information and the predicted relative position, where the control parameter is used to control the brightness of the flash.

2. The method according to claim 1, characterized in that, the obtaining depth information of the environmental preview image collected at the first moment includes: obtaining a depth map corresponding to the environmental preview image; determining a depth value of a first focal plane based on the depth map, where the first focal plane is a focal plane containing the first target object; determining the depth information according to the depth value of the first focal plane.

3. The method according to claim 2, characterized in that, the determining a depth value of a first focal plane based on the depth map includes: determining at least one pixel point of the first target object in the depth map; determining a depth value of the first target object based on depth values of the at least one pixel point; determining the depth value of the first target object as the depth value of the first focal plane.

4. The method according to claim 2, characterized in that, the method further includes: determining a second focal plane and a third focal plane based on the first focal plane; wherein, the second focal plane and the third focal plane are distributed on both sides of the first focal plane, and the distance between the second focal plane and the first focal plane is equal to the distance between the third focal plane and the first focal plane; determining weight parameters corresponding to the first focal plane, the second focal plane and the third focal plane respectively based on an image area of the first target object in the first focal plane, an image area of a second target object in the second focal plane and an image area of a third target object in the third focal plane. Based on the weight parameters corresponding to the first focal plane, the second focal plane, and the third focal plane respectively, perform a weighted sum of the depth values of the first focal plane, the depth values of the second focal plane, and the depth values of the third focal plane to obtain a comprehensive depth value; Determine the depth information according to the comprehensive depth value.

5. The method according to claim 1, wherein, the determining the predicted position of the camera at the second moment based on the motion parameters of the camera at the first moment and at least one historical moment before the first moment includes: Based on the motion parameters of the camera at the first moment and at least one historical moment before the first moment, determine the predicted motion trend parameters of the camera from the first moment to the second moment; Based on the predicted motion trend parameters of the camera, determine the predicted position of the camera at the second moment.

6. The method according to claim 1, wherein, the determining the control parameters of the flash at the second moment based on the depth information and the predicted relative position includes: Determine the control parameters of the flash at the second moment through a parameter determination model based on the depth information and the predicted relative position; wherein, the parameter determination model is a mathematical model or a machine learning model constructed based on multiple sets of experimental data, and each set of experimental data includes a corresponding relationship between a set of depth information, a predicted relative position, and control parameters under an optimal shooting effect.

7. The method according to claim 1, wherein, the method further includes: Obtain the characteristic parameters of the shooting environment, and the characteristic parameters include at least one of the following: environmental depth information, environmental type, and environmental light parameters; Based on the characteristic parameters of the shooting environment, determine a predicted convergence duration, where the predicted convergence duration is used to represent the optimal duration required for the camera to capture the first target object with the assistance of the flash; Based on the predicted convergence duration, determine the flash mode of the flash, where the flash mode is used to control the duration of a single flash of the flash.

8. A flash control device, wherein, the device includes: A depth information acquisition module, configured to acquire the depth information of an environmental preview image acquired at a first moment, where the environmental preview image is an image obtained by the camera focusing on a first target object in the shooting environment; A predicted position acquisition module, configured to determine the predicted motion trend parameters of the first target object from the first moment to the second moment based on the depth values of the first target object at the first moment and at least one historical moment before the first moment; The predicted position acquisition module is further configured to determine the predicted position of the first target object at the second moment based on the depth value of the first target object at the first moment and the predicted motion trend parameters of the first target object, where the depth value at the first moment is used to indicate the relative position between the first target object and the camera at the first moment; The predicted position obtaining module is further configured to determine the predicted position of the camera at the second moment based on the motion parameters of the camera at the first moment and at least one historical moment before the first moment; The predicted position obtaining module is further configured to determine the predicted relative position between the first target object and the camera at the second moment based on the predicted position of the first target object at the second moment and the predicted position of the camera at the second moment; The predicted position obtaining module is further configured to obtain the predicted relative position between the first target object and the camera at the second moment; The control parameter determining module is configured to determine the control parameter of the flash lamp at the second moment based on the depth information and the predicted relative position, and the control parameter is used to control the brightness of the flash lamp.

9. A device, Characterized in that, The device includes a processor and a memory, and a computer program is stored in the memory and executed by the processor to implement the flash lamp control method according to any one of claims 1 to 7.

10. A computer-readable storage medium, Characterized in that, A computer program is stored in the computer-readable storage medium, and the computer program is used to be executed by a processor to implement the flash lamp control method according to any one of claims 1 to 7.

11. A chip, Characterized in that, The chip includes programmable logic circuits and / or program instructions, and when the chip runs, it is used to implement the flash lamp control method according to any one of claims 1 to 7.

12. A computer program product, Characterized in that, The computer program product includes computer instructions, the computer instructions are stored in a computer-readable storage medium, and a processor of a device reads and executes the computer instructions from the computer-readable storage medium to implement the flash lamp control method according to any one of claims 1 to 7.

Images