Photographing method and device, equipment, chip, storage medium and product

Abstract

The invention discloses a photographing method and device, equipment, a chip, a storage medium and a product, and the method comprises the steps: controlling a first flash lamp to be turned on in response to a photographing event; obtaining at least one frame of a first type of image frame, wherein the first type of image frame is an image frame captured under the condition that the first flash lamp is turned on; after the first flash lamp is turned off, controlling the second flash lamp to be turned on; obtaining at least one second type of image frame, wherein the second type of image frame is an image frame captured under the condition that a second flash lamp is turned on; and carrying out fusion processing on the at least one first type of image frame and the at least one second type of image frame to obtain an image after fusion processing. According to the method provided by the embodiment of the invention, when fusion processing is carried out, the image frame captured under the condition that the first flash lamp is turned on and the image frame captured under the condition that the second flash lamp is turned on are fused, so that the image after fusion processing contains richer exposure information.

Description

Technical Field

[0001] This application relates to, but is not limited to, the field of cameras, and in particular, to a photographing method, apparatus, device, chip, storage medium, and product. Background Art

[0002] Flash photography technology can provide the required physical light source for the current photographing scene, thereby avoiding the picture being too dark. Currently, the commonly used flash firing method includes the pre-flash main-flash method. That is, after the user triggers a photograph, the pre-flash lamp first flashes to obtain exposure information with 3A (i.e., automatic exposure (AE), autofocus (AF), and automatic white balance (AWB)) convergence and stability; after the pre-flash, the main flash adjusts its output according to the exposure information obtained in the pre-flash stage to perform formal photographing.

[0003] During the photographing process in which the flash flashes in the pre-flash main-flash manner, the exposure information of the main flash is usually used for the photographed image. However, since the exposure information that the main flash can provide is limited, it is difficult to restore rich scene information in a dark environment. Summary of the Invention

[0004] This application provides at least one photographing method, apparatus, device, chip, storage medium, and product.

[0005] The technical solution of this application is implemented as follows:

[0006] In a first aspect, this application provides a photographing method, which is applied to a camera device. The method includes: in response to a photographing event, controlling a first flash to turn on; obtaining at least one frame of first-type image frames, where the first-type image frames are image frames captured when the first flash is on; after the first flash is turned off, controlling a second flash to turn on; obtaining at least one frame of second-type image frames, where the second-type image frames are image frames captured when the second flash is on; and performing fusion processing on the at least one frame of first-type image frames and the at least one frame of second-type image frames to obtain a fused image.

[0007] In a second aspect, an embodiment of this application provides a photographing apparatus. The apparatus includes: a first control unit, configured to control a first flash to turn on in response to a photographing event; a first obtaining unit, configured to obtain at least one frame of first-type image frames, where the first-type image frames are image frames captured when the first flash is on; a second control unit, configured to control a second flash to turn on after the first flash is turned off; a second obtaining unit, configured to obtain at least one frame of second-type image frames, where the second-type image frames are image frames captured when the second flash is on; and a fusion unit, configured to perform fusion processing on the at least one frame of first-type image frames and the at least one frame of second-type image frames to obtain a fused image.

[0008] In a third aspect, an embodiment of the present application provides a photographing device, which includes a memory and a processor. The memory is used to store computer-executable instructions, and the processor is connected to the memory and is configured to implement the method described in the first aspect by executing the computer-executable instructions.

[0009] In a fourth aspect, an embodiment of the present application provides a chip, which includes a processor configured to call and run a computer program from a memory, so that a device installed with the chip executes the method described in the first aspect.

[0010] In a fifth aspect, an embodiment of the present application provides a computer-readable storage medium storing a computer program, which when executed by at least one processor implements the method described in the first aspect.

[0011] In a sixth aspect, an embodiment of the present application provides a computer program product including a computer program or instructions, which when executed by a processor implements the method described in the first aspect.

[0012] In the embodiment of the present application, at least one first type of image frame and at least one second type of image frame can be obtained. The first type of image frame is an image frame captured when a first flash is turned on, and the second type of image frame is an image frame captured when a second flash is turned on. Further, the at least one first type of image frame and the at least one second type of image frame can be subjected to fusion processing to obtain a fused image.

[0013] According to the method of the embodiment of the present application, when performing fusion processing, the image frames captured when the first flash is turned on can be fused, and the image frames captured when the second flash is turned on can also be fused. Since the first flash and the second flash can provide different exposure information (or brightness information), this method can make the fused image contain richer exposure information (or brightness information), which is beneficial to restoring rich scene information in a dark environment.

[0014] It should be understood that the above general description and the following detailed description are only exemplary and explanatory, and do not limit the technical solution of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] The accompanying drawings herein are incorporated into the specification and constitute a part of this specification. These drawings illustrate embodiments consistent with the present application and are used together with the specification to explain the technical solution of the present application.

[0016] Figure 1 It is a schematic flowchart of a photographing method provided by an embodiment of the present application;

[0017] Figure 2 It is a schematic diagram of a possible implementation process of the photographing method provided by the embodiment of the present application;

[0018] Figure 3 It is a schematic diagram of the correspondence between the currently requested frame and the currently effective frame;

[0019] Figure 4 It is an example schematic diagram of obtaining a photographing frame in the photographing method provided by the embodiment of the present application;

[0020] Figure 5 It is a schematic diagram of the composition structure of a photographing device provided by the embodiment of the present application;

[0021] Figure 6 It is a schematic diagram of a hardware entity of a photographing device in the embodiment of the present application. Detailed implementation manners

[0022] In order to be able to understand the features and technical content of the embodiment of the present application in more detail, the implementation of the embodiment of the present application will be described in detail below with reference to the accompanying drawings. The accompanying drawings are only for reference and explanation, and are not used to limit the embodiment of the present application.

[0023] Unless otherwise defined, all technical and scientific terms used in the embodiments of the present application have the same meaning as those commonly understood by those skilled in the technical field to which the present application belongs. The terms used in the embodiments of the present application are only for the purpose of describing the embodiments of the present application, and are not intended to limit the present application.

[0024] In the following description, reference is made to "some embodiments", which describe a subset of all possible embodiments. However, it can be understood that "some embodiments" can be the same subset or different subsets of all possible embodiments, and can be combined with each other without conflict. It should also be noted that the terms "first / second / third" involved in the embodiments of the present application are only used to distinguish similar objects, and do not represent a specific order for the objects. It can be understood that "first / second / third" can be interchanged with a specific order or sequence when allowed, so that the embodiments of the present application described here can be implemented in an order other than that illustrated or described here.

[0025] It should be understood that the term " / and" in the embodiments of the present application is only a description of the association relationship of associated objects, indicating that there can be three relationships. For example, A and / or B can represent: A exists alone, A and B exist simultaneously, and B exists alone. In addition, the character " / " in this article generally represents an "or" relationship between the associated objects before and after.

[0026] As an essential function of current mobile photography devices, flash photography technology is characterized in that the flash can provide the required physical light source for the current photography scene, thus avoiding the picture from being too dark and enabling users to take good-looking photos even in a dark environment, thereby enhancing the photography experience of users in a dark environment.

[0027] Currently, the flash firing methods are mainly divided into two types: one is the flash always-on method (Torch method), and the other is the pre-flash main-flash method. There are also certain differences in the corresponding photography processes for these two methods. For example, in the Torch method, the flash is driven to turn on with a specific current during photography to provide additional light source information to complete the photography; the pre-flash main-flash method is divided into a pre-flash stage and a main-flash stage during photography. In the pre-flash stage, the pre-flash lamp flashes to obtain the exposure information for the current 3A convergence and stabilization. Among them, 3A refers to Auto Exposure (AE), Auto Focus (AF), and Auto White Balance (AWB). In the main-flash stage, the main flash lamp flashes. During this process, the exposure information of the main flash lamp can be used for taking pictures to complete the photography.

[0028] In some scenarios, the pre-flash lamp in the embodiments of the present application can also be understood as the flash lamp that performs the pre-flash, and the main flash lamp can also be understood as the flash lamp that performs the main flash. In some embodiments, the flash lamp that performs the pre-flash and the flash lamp that performs the main flash can also be the same flash lamp.

[0029] In some scenarios, the "frame" and "image frame" in the embodiments of the present application can be replaced with each other.

[0030] During the photography process when the flash flashes in the pre-flash main-flash mode, the exposure information of the main flash lamp is usually used for taking pictures. However, due to the limited exposure information that the main flash lamp can provide, it is difficult to restore rich scene information in a dark environment.

[0031] In view of this, the embodiments of the present application provide a photography method, device, equipment, chip, storage medium, and product. In this method, at least one frame of first-type image frames and at least one frame of second-type image frames can be obtained; wherein, the first-type image frames are the image frames captured when the first flash lamp is turned on, and the second-type image frames are the image frames captured when the second flash lamp is turned on; further, the at least one frame of first-type image frames and the at least one frame of second-type image frames can be fused to obtain a fused image.

[0032] According to the method of the embodiments of the present application, when performing fusion processing, image frames captured when the first flash is turned on can be fused, and image frames captured when the second flash is turned on can also be fused. Since the first flash and the second flash can provide different exposure information (or brightness information), this method can enable the image after fusion processing to contain richer exposure information (or brightness information), thereby facilitating the restoration of rich scene information in a dark environment.

[0033] In some scenarios, the first flash in the embodiments of the present application can also be understood as the flash that performs the first flash (such as performing a pre-flash), and the second flash can also be understood as the flash that performs the second flash (such as performing the main flash). Exemplarily, the flash that performs the first flash and the flash that performs the second flash can be the same flash.

[0034] The embodiments of the present application will be described in detail below with reference to the accompanying drawings.

[0035] The embodiments of the present application provide a photographing method applied to a camera device. As Figure 1 shown, the method may include:

[0036] S101, in response to a photographing event, control the first flash to turn on.

[0037] Exemplarily, the photographing event can be understood as an event in which a user triggers the camera device to take a photo. For example, the user presses the shutter.

[0038] In this step, the camera device can control the first flash to turn on in response to the photographing event. For example, after the user presses the shutter, the application layer (APP) of the camera device can send a request to start triggering the flash to the Hardware Abstract Layer (HAL), and then HAL can control the first flash to turn on. Here, the first flash turning on can also be understood as the first flash flashing, or it can also be understood as the flash performing the first flash.

[0039] In some embodiments, the first flash is a pre-flash.

[0040] S102, obtain at least one first type of image frame, where the first type of image frame is an image frame captured when the first flash is turned on.

[0041] After the first flash is turned on, at least one first type of image frame (that is, an image frame captured when the first flash is turned on) can be obtained. The at least one first type of image frame will be used for subsequent fusion processing to obtain an image after fusion processing.

[0042] In some embodiments, obtaining at least one first - type image frame may include: obtaining at least one first - type image frame captured after a first moment, where the first moment is the moment when the first camera parameters of the camera device reach a stable state. That is to say, the at least one first - type image frame obtained in this step may be at least one first - type image frame captured after the first camera parameters of the camera device reach a stable state.

[0043] In some embodiments, obtaining at least one first - type image frame captured after a first moment may include: obtaining all first - type image frames captured after the first moment. That is to say, the at least one first - type image frame obtained in this step may be all first - type image frames captured after the first camera parameters of the camera device reach a stable state.

[0044] In some embodiments, after the first camera parameters of the camera device reach a stable state, the application layer of the camera device may send a photographing request to the hardware abstraction layer. After receiving the photographing request, the hardware abstraction layer can trigger a formal photographing. In this case, obtaining at least one first - type image frame captured after the first moment may include: obtaining at least one first - type image frame captured after the application layer of the camera device sends the photographing request.

[0045] In some embodiments, obtaining at least one first - type image frame captured after the application layer of the camera device sends the photographing request may include: obtaining all first - type image frames captured after the application layer of the camera device sends the photographing request. That is to say, the at least one first - type image frame obtained in this step may be all first - type image frames captured after the application layer of the camera device sends the photographing request.

[0046] According to the method of this embodiment, at least one first - type image frame captured after the first moment can be obtained for fusion processing. In this way, there is no need to cache the image frames captured before the first moment, thus saving cache resources. In addition, since the image frames captured after the first moment are all the image frames captured after the first camera parameters of the camera device reach a stable state, the adverse effects of image frames with unstable first camera parameters on the fusion result can be avoided.

[0047] In some embodiments, the first camera parameters may include at least one of the following: automatic exposure (AE) parameter, automatic focus (AF) parameter, automatic white balance (AWB) parameter. In this case, the first camera parameters reaching a stable state can also be understood as reaching a 3A convergence state, or can also be understood as the AE, AF, and AWB functions working together to finally reach a stable and accurate state.

[0048] In some embodiments, the method may further include: determining exposure amount information to be used by the camera device during the photographing process according to the second camera parameters of the camera device in the preview state; determining a second moment according to the exposure amount information; and controlling the first flash to turn off at the second moment.

[0049] Among them, the first flash turning off can also be understood as the first flash stopping flashing, or can also be understood as the flash ending the first flash.

[0050] Exemplarily, the second camera parameters may include sensitivity, illuminance, exposure amount, etc.

[0051] Exemplarily, the exposure amount information to be used by the camera device during the photographing process can also be understood as the exposure amount information to be used by the camera device in the formal photographing stage, that is, after the application layer of the camera device issues a photographing request, the exposure amount information required for the camera device to take a photograph.

[0052] In some scenarios, the "exposure amount" and "exposure value (i.e., EV value)" in the embodiments of the present application can be interchanged.

[0053] In one implementation, before the user triggers photographing (such as before pressing the shutter button), the camera device can enter the preview state. Thus, according to the second camera parameters (such as sensitivity, illuminance, exposure amount, etc.) in the preview state, the exposure amount information to be used by the camera device in the subsequent photographing process can be calculated. According to the exposure amount information, a second moment (that is, the moment to control the first flash to turn off) can be determined.

[0054] As one implementation, the exposure amount information to be used by the camera device during the photographing process can be represented by an exposure value list (EVList). That is, according to the second camera parameters in the preview state, the EVList can be obtained. For example, according to the second camera parameters in the preview state, the EVList (0 0 0 0 0 -80 0 0) can be obtained.

[0055] Among them, each value in the EVList can represent the exposure value (i.e., EV value) corresponding to one frame. For example, for the EVList (0 0 0 0 0 -8 0 0 0), the first 5 values "0" indicate that the EV values corresponding to the first 5 frames are 0. Thus, in the formal photographing stage, the camera device can calculate the exposure gain and exposure time required for capturing the first 5 frames based on the EV value "0"; the 6th value "-8" indicates that the EV value corresponding to the 6th frame is -8. Thus, in the formal photographing stage, the camera device can calculate the exposure gain and exposure time required for capturing the 6th frame based on the EV value "-8".

[0056] Further, the second moment may be determined according to the exposure amount information, and the first flash lamp may be controlled to turn off at the second moment.

[0057] In some embodiments, controlling the first flash lamp to turn off at the second moment may also be understood as triggering PostPreFlash (waiting for the lamp to turn off) at the second moment. In one implementation, after triggering PostPreFlash, the first flash lamp may be turned off officially after a short delay. That is to say, after triggering PostPreFlash, the action of turning off the first flash lamp may be delayed to take effect.

[0058] According to the method of this embodiment, the moment to turn off the first flash lamp may be determined according to the exposure amount information required during the photographing process. In this way, the timing of turning off the first flash lamp can be adapted to the exposure situation during photographing, which is beneficial to improving the effect of flash photography.

[0059] In some embodiments, determining the second moment according to the exposure amount information (that is, the exposure amount information required during the photographing process) may include: determining the time interval between the second moment and the moment when the application layer of the camera device issues a photographing request according to the exposure amount information; determining the second moment according to the moment when the application layer of the camera device issues a photographing request and the time interval.

[0060] Exemplarily, assuming that the exposure amount information required during the photographing process is EVList, then the time interval between the second moment and the moment when the application layer of the camera device issues a photographing request may be determined according to EVList.

[0061] For example, for EVList(0 0 0 0 0 -8 0 0 0), the first negative value (i.e., "-8") is the 6th value in this EVList. Then, the time interval between the second moment and the moment when the application layer of the camera device issues a photographing request may be 6 frames. That is to say, the second moment may be the 7th frame after the application layer of the camera device issues a photographing request. In other words, starting from the moment when the application layer of the camera device issues a photographing request, after 6 frames, the first flash lamp can be controlled to turn off. Or rather, at the 7th frame after the application layer of the camera device issues a photographing request, the first flash lamp can be controlled to turn off.

[0062] According to the method of this embodiment, the time interval between the second moment (i.e., the moment when the first flash is turned off) and the moment when the camera device's application layer issues a photographing request can be determined based on the exposure amount information required during the photographing process. Furthermore, after the camera device's application layer issues a photographing request, the moment to turn off the first flash can be determined according to this time interval. In addition, the moment to turn off the first flash can be after the moment when the camera device's application layer issues a photographing request. That is to say, within a period of time after the camera device's application layer issues a photographing request, the first flash will remain on. In this way, it can be ensured that after the camera device's application layer issues a photographing request, an image frame captured with the first flash on can be obtained.

[0063] In one implementation, determining the time interval between the second moment and the moment when the camera device's application layer issues a photographing request according to the exposure amount information may include: determining a first sequence according to the exposure amount information; determining the time interval between the second moment and the moment when the camera device's application layer issues a photographing request according to the first sequence.

[0064] Exemplarily, the first sequence may include m values (such as m "1"s), and m can be determined according to the position where a negative value first appears in the EVList. For example, for the EVList (0 0 0 0 0 -8 0 0 0), the first negative value (i.e., "-8") is the 6th value in the EVList. Then, the value of m can be equal to 6, that is, the first sequence may include 6 values (such as 6 "1"s).

[0065] Exemplarily, each value "1" in the first sequence may represent that the corresponding flash state of one frame is the on state of the first flash. If the first sequence includes m "1"s, it can represent that after the camera device's application layer issues a photographing request, there are m frames with the corresponding flash state being the on state of the first flash.

[0066] In one implementation, the first sequence may be included in the SnapshotTriggerList, and the first sequence may be the first subsequence in the SnapshotTriggerList.

[0067] Furthermore, the time interval between the second moment and the moment when the camera device's application layer issues a photographing request can be determined according to the length of the first sequence. For example, assuming the first sequence includes 6 values (such as 6 "1"s), that is, the length of the first sequence is 6, then the time interval between the second moment and the moment when the camera device's application layer issues a photographing request can be determined to be 6 frames.

[0068] In some embodiments, the time interval (or the first sequence) between the second moment and the moment when the application layer of the camera device issues a photographing request may also be preconfigured. For example, the time interval (or the first sequence) between the second moment and the moment when the application layer of the camera device issues a photographing request may be preconfigured according to the current application requirements.

[0069] For example, assume that the current application requirement is to keep the first flash lamp on for a relatively long time. Then, the time interval between the second moment and the moment when the application layer of the camera device issues a photographing request may be configured to be relatively long (that is, the number of frames corresponding to this time interval may be configured to be relatively large), or the first sequence may be configured to be relatively long (that is, the number of values included in the configured first sequence may be relatively large).

[0070] S103. After the first flash lamp is turned off, control the second flash lamp to turn on.

[0071] After the first flash lamp is turned off, the camera device may control the second flash lamp to turn on. Herein, the second flash lamp turning on may also be understood as the second flash lamp flashing, or may also be understood as the flash lamp performing a second flash.

[0072] In some embodiments, the second flash lamp is the main flash lamp.

[0073] In some embodiments, controlling the second flash lamp to turn on may include: controlling the second flash lamp to turn on at a third moment, and there is at least one-frame time interval between the third moment and the moment when the first flash lamp is turned off.

[0074] That is to say, after the first flash lamp is turned off, the camera device may control the second flash lamp to turn on at the third moment, and there is at least one-frame time interval between the third moment and the moment when the first flash lamp is turned off.

[0075] In some embodiments, the time interval between the third moment and the moment when the first flash lamp is turned off may be determined based on the length of the second sequence. That is to say, the time interval between the third moment and the moment when the first flash lamp is turned off may be determined according to the length of the second sequence, and then the third moment (that is, the moment to control the second flash lamp to turn on) may be determined according to this time interval.

[0076] For example, assume that the second sequence includes 2 values (such as 2 "3"), that is, the length of the second sequence is 2. Then, it may be determined that the time interval between the third moment and the moment when the first flash lamp is turned off is 2 frames.

[0077] In one implementation, the second sequence may include at least one numerical value "3", where each numerical value "3" may represent that the corresponding flash state of a frame is a transition state. If the second sequence includes n "3"s, it may represent that after the first flash is turned off, there are n frames with the corresponding flash state being the transition state. Here, the transition state can also be understood as the transition state between the first flash being on and the second flash being on.

[0078] In one implementation, the second sequence may be included in the SnapshotTriggerList, and the second sequence may be the second subsequence in the SnapshotTriggerList.

[0079] In some embodiments, the time interval (or the second sequence) between the third moment and the moment when the first flash is turned off is pre-configured. For example, the time interval (or the second sequence) between the third moment and the moment when the first flash is turned off may be pre-configured according to the current application requirements.

[0080] For example, assume that the current application requirement is to quickly turn on the second flash after the first flash is turned off. Then, the time interval between the third moment and the moment when the first flash is turned off can be configured to be relatively short (that is, the number of frames corresponding to this time interval can be configured to be relatively small), or the second sequence can be configured to be relatively short (that is, the number of numerical values included in the configured second sequence can be relatively small).

[0081] In some embodiments, the time interval (or the second sequence) between the third moment and the moment when the first flash is turned off may be fixed. For example, after the time interval (or the second sequence) between the third moment and the moment when the first flash is turned off is pre-configured, it will no longer change.

[0082] In some embodiments, the time interval (or the second sequence) between the third moment and the moment when the first flash is turned off may be dynamically changed. For example, whenever the application requirements change, the time interval (or the second sequence) between the third moment and the moment when the first flash is turned off can be re-pre-configured.

[0083] S104, obtain at least one second type of image frame, where the second type of image frame is an image frame captured when the second flash is on.

[0084] In this step, at least one second type of image frame (that is, an image frame captured when the second flash is on) can be obtained. Furthermore, in S105, the at least one first type of image frame obtained previously and the at least one second type of image frame can be fused to obtain a fused image.

[0085] In some embodiments, the duration for which the second flash is turned on is M frames; obtaining at least one second type of image frame may include: obtaining at least one second type of image frame within the M frames; where M is less than or equal to a first threshold.

[0086] Wherein, the duration for which the second flash is turned on is M frames, which can also be understood as that after the second flash is turned on, the duration of maintaining the on state is M frames.

[0087] Since the working current of the second flash (such as the main flash) is usually large, therefore, limiting the duration for which the second flash is turned on to M frames can save the power consumption caused by the flash to a certain extent.

[0088] As an implementation, M = 1. That is, the duration for which the second flash is turned on is 1 frame, and in this way, the power consumption caused by the flash can be saved as much as possible.

[0089] In some embodiments, the second flash can flash with a relatively large current so as to be able to obtain scene information with sufficient brightness. According to the method of this embodiment, an image frame captured when the first flash is turned on can be obtained, and an image frame captured when the second flash is turned on can also be obtained. Therefore, both the exposure information (or brightness information) provided by the first flash and the exposure information (or brightness information) provided by the second flash can be obtained. Furthermore, by fusing the image frame captured when the first flash is turned on and the image frame captured when the second flash is turned on, the fused image can contain richer exposure information (or brightness information).

[0090] S105, perform a fusion process on the at least one first type of image frame and the at least one second type of image frame to obtain a fused image.

[0091] In this step, a fusion process can be performed on the at least one first type of image frame obtained in S102 and the at least one second type of image frame obtained in S104 to obtain a fused image. This fused image is also the photo obtained from this photo-taking.

[0092] According to the method of this embodiment, when performing the fusion process, at least one first type of image frame (that is, the image frame captured when the first flash is turned on) can be fused, and at least one second type of image frame (that is, the image frame captured when the second flash is turned on) can also be fused. Since the first flash and the second flash can provide different exposure information (or brightness information), this method can make the fused image contain richer exposure information (or brightness information), thereby being beneficial to restoring rich scene information in a dark environment.

[0093] In some embodiments, the method may further include: obtaining the image frames (denoted as the third type of image frames) between the first type of image frames and the second type of image frames. In this case, when performing the fusion process, in addition to being able to fuse the at least one first type of image frame and the at least one second type of image frame, the third type of image frames may also be fused.

[0094] It can be understood that since the first type of image frames and the second type of image frames are not continuous, therefore, when performing the fusion process, if the third type of image frames are not fused, "ghosts" may appear in the fused image. After fusing the third type of image frames, the "ghosts" can be eliminated to a certain extent.

[0095] To facilitate the understanding of the embodiments of the present application, a possible implementation process of the photographing method provided by the embodiments of the present application is introduced below.

[0096] Figure 2 It is a schematic diagram of a possible implementation process of the photographing method provided by the embodiments of the present application. As Figure 2 shown, the implementation process may include a preview stage and a photographing and frame-taking stage.

[0097] Figure 2 Figure (a) in

[0098] S211, the APP sends preview information to the HAL.

[0099] Exemplarily, the preview information may include the size, data format, etc. of the preview image.

[0100] S212, the HAL sends information on whether to flash to the APP.

[0101] In this step, the HAL may perform a preview according to the preview information sent by the APP, and obtain information such as the current preview sensitivity (IOS), illuminance (LuxIndex), exposure amount, etc. Further, the HAL may determine whether the flash needs to flash according to the current preview sensitivity, illuminance, exposure amount, etc., and send information on whether to flash to the APP to inform the APP whether the flash needs to flash.

[0102] In one implementation, the HAL may also send information such as the current preview sensitivity, illuminance, exposure amount, etc. to the APP.

[0103] S213, the APP sends information on whether to flash to the Algorithm Process Service (APS) module (abbreviated as APS).

[0104] After the APP receives the information on whether to flash sent by the HAL, it can further send this information to the APS to inform the APS whether the flash needs to be fired.

[0105] In one implementation, after the APP receives information such as the sensitivity, illuminance, and exposure amount of the current preview, it can also send the information such as the sensitivity, illuminance, and exposure amount of the current preview to the APS.

[0106] S214, the APS sends the flash decision information and the information required for flash photography to the APP.

[0107] In this step, the APS can decide whether the flash needs to be fired during subsequent photography based on the information on whether to flash sent by the APP, and send the flash decision information to the APP. The flash decision information is used to inform the APP whether the flash needs to be fired during subsequent photography.

[0108] Furthermore, if the decision result of the APS is that the flash needs to be fired during subsequent photography, then the APS can also send the information required for flash photography to the APP. For example, a list of exposure values (EVList) required for flash photography, etc. Among them, the EVList can be generated based on information such as the sensitivity, illuminance, and exposure amount of the current preview.

[0109] Furthermore, after the user triggers photography (such as pressing the shutter), the APP can send a request to start triggering the flash to the HAL. Subsequently, the HAL can control the pre-flash to fire.

[0110] Figure 2 Figure (b) in [] is a schematic flowchart of the photo-taking frame acquisition stage, and this stage may include the following steps:

[0111] S221, the APP sends a photography request to the HAL.

[0112] After 3A convergence, the APP can send a photography request to the HAL. Among them, the photography request may carry the information required for flash photography.

[0113] Exemplarily, the information required for flash photography may include the following customized APS photography information: EVList(0 0 00 0 -8 0 0 0), SnapshotTriggerList(1 1 1 1 1 1 3 3 2). Each value in the EVList represents the exposure value corresponding to one frame (i.e., the EV value). In the AE module, the EV value will be converted into the exposure time and gain; each value in the SnapshotTrigerList represents the flash state corresponding to one frame. As an example, 1 represents pre-flash, 2 represents main flash, 3 represents transition frame, and 0 represents turning off the light.

[0114] In one implementation, the part in the SnapshotTriggerList indicating pre-flash can be determined according to the EVList. For example, if the 6th value in the EVList is negative, that is, the EV value corresponding to the 6th frame is negative, then in the SnapshotTriggerList, the first 6 values should be 1, that is, the flash states corresponding to the first 6 frames are pre-flash.

[0115] S222, the HAL sends the captured photo data to the APP.

[0116] After the HAL receives the photo-taking request, it can trigger the official photo-taking. During the official photo-taking process, the HAL can control the flash to flash according to the information required for flash photography (such as EVList, SnapshotTriggerList). For example, during the official photo-taking process, according to the EVList (0 0 0 0 0 -8 0 0 0) and the SnapshotTriggerList (1 1 1 1 1 1 3 32), the HAL can control the pre-flash to flash in the first 6 frames, and the EV values corresponding to the first 6 frames are 0, 0, 0, 0, 0, -8 respectively; the 7th and 8th frames are transition frames and do not flash, and the EV values corresponding to the 7th and 8th frames are 0, 0 respectively; subsequently, the HAL can control the main flash to flash in the 9th frame, and the EV value corresponding to the 9th frame is 0. In some scenarios, during the official photo-taking process, the exposure amount in the main flash stage can also be adjusted according to the exposure amount in the pre-flash stage to ensure that the image frames with superimposed flashes are not too bright or too dark.

[0117] After the photo-taking ends, the HAL can send the captured photo data to the APP, which includes the photo-taking frames obtained during the official photo-taking process.

[0118] Exemplarily, the "preview frame" in the embodiments of the present application can also be understood as the image frame captured during the preview stage before the official photo-taking (that is, before the APP sends the photo-taking request); the "photo-taking frame" can also be understood as the image frame captured during the official photo-taking stage.

[0119] In the embodiments of the present application, the flash control process may include the following four stages 1) to 4):

[0120] 1) PreCapture: The HAL receives the start trigger flash request sent from the upper layer, and the HAL prepares to turn on the pre-flash.

[0121] 2) PreFlash: The pre-flash starts to flash. After the 3A convergence ends, until the current frame is a transition frame, it triggers PostPreFlash.

[0122] Exemplarily, after the pre-flash starts flashing, it can remain on until the HAL receives a photographing request. After the HAL receives the photographing request, the HAL can determine whether the current frame is a transition frame according to the SnapshotTriggerList. For example, in the SnapshotTriggerList (1 1 1 1 1 1 3 3 2), the 7th and 8th values are "3", that is, the 7th and 8th frames represent transition frames. Then, if the current frame is the 7th frame after the HAL receives the photographing request, the current frame can be considered a transition frame. In this case, PostPreFlash can be triggered. If the current frame is not a transition frame, the pre-flash can remain on, and the current state that needs to turn off the light can be recorded until PostPreFlash is triggered when the current frame is a transition frame.

[0123] According to the method of this embodiment, when reaching the 3A convergence state, PostPreFlash will not be immediately triggered. Instead, after a period of time interval, it will be triggered when the current frame is a transition frame. In this way, it can be ensured that after 3A convergence, an image frame captured with the pre-flash on can be obtained.

[0124] 3) PostPreFlash: Wait for turning off the light. After triggering PostPreFlash, turning off the light can take effect with a delay.

[0125] Take Figure 3 as an example. The first row represents the currently requested frame (i.e., the stt frame of the current req), and the second row represents the currently effective frame (i.e., the stt frame being processed). There is a corresponding relationship between the currently requested frame and the currently effective frame. For example, the 100th frame in the first row (i.e., the frame labeled #100) corresponds to the 98th frame in the second row (i.e., the frame labeled #98), the 101st frame in the first row (i.e., the frame labeled #101) corresponds to the 99th frame in the second row (i.e., the frame labeled #99), and so on.

[0126] In Figure 3 's example, assume that the 105th frame in the first row (i.e., the frame labeled #105) is a transition frame. Then, PostPreFlash can be triggered at the 105th frame in the first row. After triggering PostPreFlash, turning off the light can take effect with a delay. For example, it is necessary to wait until the currently requested frame and the currently effective frame are the same before turning off the pre-flash. Take Figure 3For example, assume that PostPreFlash is triggered when the current requested frame is the 105th frame. Then, the pre-flashlight needs to be turned off until the currently effective frame is also the 105th frame. The reason is that the capture frame that should originally include the pre-flash brightness before the transition frame. If the pre-flashlight is turned off in advance, it cannot be guaranteed to obtain the capture frame with the pre-flash brightness.

[0127] 4) MainFlash: The main flashlight starts to flash.

[0128] Exemplarily, after the HAL receives a capture request, it can determine whether the flashlight state corresponding to the current frame is the main flash according to the SnapshotTriggerList. For example, in SnapshotTriggerList(1 1 1 1 1 1 33 2), the 9th value is "2", that is, the flashlight state corresponding to the 9th frame is the main flash. Then, if the current frame is the 9th frame after the HAL receives the capture request, it can be considered that the flashlight state corresponding to the current frame is the main flash. In this case, the HAL can turn on the main flashlight.

[0129] In one implementation, the main flashlight can be turned off after flashing for one frame, thus solving the power consumption problem caused by the long duration of the main flashlight.

[0130] According to the method of this embodiment, since the pre-flashlight will continue to flash after it is turned on until PostPreFlash is triggered when the current frame is a transition frame, during the formal capture process, not only can the image frame captured when the main flashlight flashes be obtained, but also the image frame captured when the pre-flashlight flashes can be obtained.

[0131] Furthermore, the HAL can send capture data to the APP, which includes the capture frames obtained during the formal capture process (including the capture frames captured when the pre-flashlight flashes and the capture frames captured when the main flashlight flashes).

[0132] Figure 4 This is a schematic diagram of obtaining a capture frame in the capture method provided by the embodiment of the present application. As Figure 4 shown, assume that the capture request is issued at the end of preview frame #1. Then, the frames after preview frame #1 are capture frames. In Figure 4 the example, the pre-flashlight starts to flash from preview frame #1 and continues until capture frame #6; the main flashlight starts to flash from capture frame #9 and is turned off after capture frame #9. After the capture is completed, the HAL can send capture data (that is, the obtained capture frames) to the APP, which includes the capture frames captured when the pre-flashlight flashes (that is, capture frames #1 to capture frame #6) and the capture frames captured when the main flashlight flashes (that is, capture frame #9). In some embodiments, the capture data sent by the HAL to the APP may also include capture frame #7 and capture frame #8.

[0133] S523. The APP sends the packaged photo-taking data to the APS.

[0134] After receiving the photo-taking data sent by the HAL, the APP can package the photo-taking data and send it to the APS.

[0135] S524. The APS sends the photo-taking result to the APP.

[0136] After receiving the packaged photo-taking data, the APS can perform fusion processing on the photo-taking frames included therein, and return the result of the fusion processing as the photo-taking result to the APP.

[0137] According to the method of this embodiment, since the photo-taking data includes not only the image frames captured when the main flash fires, but also the image frames captured when the pre-flash fires, diverse exposure information (or brightness information) can be provided for the photo-taking result, which is conducive to restoring rich scene information in a dark environment.

[0138] In the embodiments of the present application, by controlling the flash state through customized parameter information, multiple photo-taking frames can be obtained during the pre-flash to provide input images of the pre-flash brightness. During the main flash, input images with a larger flash current brightness can be obtained to provide sufficient brightness scene information, so that the information in the dark scene can be restored to a great extent. Finally, rich scene information under the flash can be restored through the multi-frame fusion method. In addition, the embodiments of the present application reduce the on-time and power consumption of the main flash.

[0139] The photo-taking method provided by the embodiments of the present application can bring at least the following three benefits:

[0140] Benefit 1: Controlling the flash firing state through customized parameter information provides guarantee for the input of the image fusion algorithm.

[0141] Benefit 2: Solving the power consumption problem caused by the long duration of the main flash when there are multiple photo-taking frames.

[0142] Benefit 3: In a dark scene, more scene information can be obtained through the pre-flash and main-flash scheme, and useful scene information can be obtained under the exposure of one main-flash frame.

[0143] Exemplarily, the method of the embodiments of the present application is applicable to scenes that require stronger exposure, such as urban night scenes, indoor night scenes, mixed lighting, blue tones, sunsets, etc. Through the method of the embodiments of the present application, rich scene information can be restored in the above scenes, thus bringing a better photo-taking experience to users.

[0144] The preferred embodiments of the present application have been described in detail above in conjunction with the accompanying drawings. However, the present application is not limited to the specific details in the above embodiments. Within the scope of the technical concept of the present application, various simple modifications can be made to the technical solutions of the present application, and these simple modifications all fall within the protection scope of the present application. For example, in the various specific technical features described in the above specific embodiments, they can be combined in any suitable manner without contradiction. To avoid unnecessary repetition, the present application will not separately describe various possible combination methods. Again, for example, any combination can be made between various different embodiments of the present application as long as it does not violate the idea of the present application, and it should also be regarded as the content disclosed in the present application. Again, for example, on the premise of no conflict, each embodiment described in the present application and / or the technical features in each embodiment can be arbitrarily combined with the prior art, and the technical solutions obtained after combination should also fall within the protection scope of the present application.

[0145] Based on the foregoing embodiments, embodiments of the present application provide a corresponding photographing device. The device includes each module / unit included therein, as well as each sub-module / sub-unit included in each module / unit, and can be implemented by a processor in a computer device with information processing capabilities; of course, it can also be implemented by specific logic circuits; during the implementation process, the processor can be a central processing unit (CPU), a microprocessor unit (MPU), a digital signal processor (DSP), or a field programmable gate array (FPGA), etc.

[0146] Embodiments of the present application provide a photographing device. As Figure 5 shown, the photographing device 500 may include:

[0147] A first control unit 510, configured to control the first flash to turn on in response to a photographing event; a first acquisition unit 520, configured to acquire at least one frame of first-type image frames, where the first-type image frames are image frames captured when the first flash is on; a second control unit 530, configured to control the second flash to turn on after the first flash is turned off; a second acquisition unit 540, configured to acquire at least one frame of second-type image frames, where the second-type image frames are image frames captured when the second flash is on; a fusion unit 550, configured to perform a fusion process on the at least one frame of first-type image frames and the at least one frame of second-type image frames to obtain a fused image.

[0148] In some embodiments, the first acquisition unit 520 is specifically configured to: acquire at least one first type of image frame captured after a first moment, where the first moment is the moment when the first camera parameters of the camera device reach a stable state.

[0149] In some embodiments, the photographing device 500 further includes: a first determination unit, configured to determine exposure amount information to be used by the camera device during photographing according to second camera parameters of the camera device in a preview state; a second determination unit, configured to determine a second moment according to the exposure amount information; and a third control unit, configured to control the first flash to turn off at the second moment.

[0150] In some embodiments, the second determination unit is specifically configured to: determine a time interval between the second moment and the moment when a photographing request is sent from the application layer of the camera device according to the exposure amount information; and determine the second moment according to the moment when the photographing request is sent from the application layer of the camera device and the time interval.

[0151] In some embodiments, the second control unit 530 is specifically configured to: control the second flash to turn on at a third moment, where there is a time interval of at least one frame between the third moment and the moment when the first flash turns off.

[0152] In some embodiments, the duration for which the second flash is on is M frames; the second acquisition unit 540 is specifically configured to: acquire at least one second type of image frame within the M frames; where M is less than or equal to a first threshold.

[0153] The description of the above device embodiments is similar to the description of the above method embodiments and has similar beneficial effects to those of the method embodiments. In some embodiments, the functions or modules / units included in the device provided in the embodiments of the present application can be used to execute the methods described in the above method embodiments. For technical details not disclosed in the device embodiments of the present application, please refer to the description of the method embodiments of the present application for understanding.

[0154] It should be noted that in the embodiments of the present application, if the above-mentioned method is implemented in the form of software function modules and sold or used as an independent product, it can also be stored in a computer-readable storage medium. Based on such an understanding, the technical solution of the embodiments of the present application, in essence or the part that contributes to the related technology, can be embodied in the form of a software product. The software product is stored in a storage medium and includes several instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the methods described in the various embodiments of the present application. The aforementioned storage medium includes: various media such as USB flash drives, mobile hard disks, read-only memories (ROMs), magnetic disks, or optical discs that can store program codes. In this way, the embodiments of the present application are not limited to any specific hardware, software, or firmware, or any combination among hardware, software, and firmware.

[0155] The embodiments of the present application further provide a photographing device, including a memory and a processor. The memory stores a computer program that can run on the processor, and when the processor executes the program, it implements some or all of the steps in the above-mentioned method.

[0156] The embodiments of the present application further provide a chip. The chip includes: a processor for calling and running a computer program from a memory, so that a device installed with the chip executes some or all of the steps in the above-mentioned method.

[0157] The embodiments of the present application further provide a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, it implements some or all of the steps in the above-mentioned method. The computer-readable storage medium can be transient or non-transient.

[0158] The embodiments of the present application further provide a computer program, including computer-readable code. When the computer-readable code runs in a device, the processor in the device executes some or all of the steps in the above-mentioned method.

[0159] The embodiments of the present application further provide a computer program product. The computer program product includes a non-transient computer-readable storage medium storing a computer program. When the computer program is read and executed by a computer, it implements some or all of the steps in the above-mentioned method. The computer program product can be specifically implemented in a manner of hardware, software, or a combination thereof. In some embodiments, the computer program product is specifically embodied as a computer storage medium. In other embodiments, the computer program product is specifically embodied as a software product, such as a software development kit (SDK), etc.

[0160] It should be noted here that the descriptions of the above embodiments tend to emphasize the differences between the embodiments, and their similarities can be referred to each other. The descriptions of the above embodiments of the device, chip, storage medium, computer program, and computer program product are similar to the descriptions of the above method embodiments and have similar beneficial effects to those of the method embodiments. For the technical details not disclosed in the embodiments of the device, chip, storage medium, computer program, and computer program product of the present application, please refer to the descriptions of the method embodiments of the present application for understanding.

[0161] An embodiment of the present application provides a photographing device. As Figure 6 shown, the photographing device 600 (hereinafter simply referred to as the device 600) includes a processor 610. The processor 610 can call and run a computer program from a memory to implement the method in the embodiment of the present application.

[0162] In some embodiments, as Figure 6 shown, the device 600 may further include a memory 620. Among them, the processor 610 can call and run a computer program from the memory 620 to implement the method in the embodiment of the present application. Among them, the memory 620 can be a separate device independent of the processor 610 or integrated in the processor 610.

[0163] In some embodiments, as Figure 6 shown, the device 600 may further include a transceiver 630. The processor 610 can control the transceiver 630 to communicate with other devices. Specifically, it can send information or data to other devices or receive information or data sent by other devices. Among them, the transceiver 630 can include a transmitter and a receiver. The transceiver 630 may further include an antenna, and the number of antennas can be one or more.

[0164] It should be understood that the "one embodiment", "an embodiment" or "some embodiments" mentioned throughout the specification means that the specific features, structures or characteristics related to the embodiment are included in at least one embodiment of the present application. Therefore, the appearances of "in one embodiment", "in an embodiment" or "in some embodiments" throughout the specification do not necessarily refer to the same embodiment. In addition, these specific features, structures or characteristics can be combined in one or more embodiments in any suitable manner. It should be understood that in various embodiments of the present application, the magnitudes of the serial numbers of the above steps / processes do not mean the order of execution. The execution order of each step / process should be determined by its function and internal logic, and should not constitute any limitation to the implementation process of the embodiments of the present application. The serial numbers of the embodiments of the present application above are only for description and do not represent the advantages or disadvantages of the embodiments.

[0165] It should be noted that, in this document, the terms "include", "comprise" or any other variant thereof are intended to cover non-exclusive inclusion, such that a process, method, article or device that includes a series of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article or device. Without further limitation, an element defined by the phrase "including a..." does not exclude the presence of additional identical elements in the process, method, article or device that includes such element.

[0166] In several embodiments provided in this application, it should be understood that the disclosed devices and methods can be implemented in other ways. The device embodiments described above are merely illustrative. For example, the division of the units or modules is only a logical functional division, and there can be other division methods in actual implementation. For example, multiple units or components can be combined, or can be integrated into another system, or some features can be ignored, or not executed. In addition, the coupling, direct coupling, or communication connection between the various components shown or discussed can be through some interfaces. The indirect coupling or communication connection of devices or units can be electrical, mechanical or other forms.

[0167] The units described above as separate components may or may not be physically separated, and the components shown as units may or may not be physical units; they can be located in one place or distributed to multiple network units; some or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0168] In addition, each functional unit in the embodiments of this application can be all integrated in a processing unit, or each unit can be separately a unit, or two or more units can be integrated in a unit; the above-mentioned integrated units can be implemented in the form of hardware, or in the form of a combination of hardware and software functional units.

[0169] Those of ordinary skill in the art can understand that all or part of the steps to implement the above method embodiments can be completed by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it executes the steps including the above method embodiments; and the aforementioned storage medium includes: various media such as removable storage devices, read-only memory (ROM), magnetic disks or optical discs that can store program codes.

[0170] Alternatively, if the above integrated units of the present application are implemented in the form of software functional modules and sold or used as independent products, they can also be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, in essence or the part that contributes to the related technology, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the methods described in the various embodiments of the present application. The aforementioned storage medium includes: various media that can store program codes, such as removable storage devices, ROMs, magnetic disks, or optical discs.

[0171] As described above, the above are only the implementation manners of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art within the technical scope disclosed by the present application can easily think of changes or substitutions, which should all be covered within the protection scope of the present application.

Claims

1. A photographing method, characterized in that: Applied to a camera device, the method comprises: In response to a photo taking event, controlling the first flash to turn on; Acquire at least one first-category image frame, where the first-category image frame is an image frame captured when the first flash is turned on; After the first flash light is turned off, controlling the second flash light to turn on; Acquire at least one second-type image frame, where the second-type image frame is an image frame captured when the second flash is turned on; The at least one first-category image frame and the at least one second-category image frame are fused to obtain a fused image.

2. The method according to claim 1, characterized in that The acquiring of at least one first-category image frame comprises: At least one first-category image frame captured after a first moment is acquired, where the first moment is a moment when a first camera parameter of the camera device reaches a stable state.

3. The method according to claim 1 or 2, characterized in that: The method further comprises: Determining exposure information to be used by the camera device during photographing according to a second camera parameter of the camera device in a preview state; determining a second moment according to the exposure information; At the second moment, the first flash light is controlled to be turned off.

4. The method according to claim 3, characterized in that The determining the second moment according to the exposure information comprises: Determine, according to the exposure information, a time interval between the second moment and a moment when the application layer of the camera device sends a photo request; The second moment is determined according to the moment when the application layer of the camera device sends the photo request and the time interval.

5. The method according to claim 1 or 2, characterized in that: The controlling the second flash light to turn on includes: The second flash light is controlled to be turned on at a third moment, and there is a time interval of at least one frame between the third moment and the moment when the first flash light is turned off.

6. The method according to claim 1 or 2, characterized in that: The second flash light is turned on for M frames; The acquiring of at least one second-category image frame comprises: At least one second-category image frame is acquired within the M frames; wherein M is less than or equal to a first threshold.

7. A photographing device, characterized in that: The device comprises: A first control unit, configured to control the first flash to turn on in response to a photo taking event; A first acquisition unit, configured to acquire at least one first-type image frame, where the first-type image frame is an image frame captured when the first flash is turned on; A second control unit, used for controlling the second flash light to turn on after the first flash light is turned off; A second acquisition unit, configured to acquire at least one second-type image frame, where the second-type image frame is an image frame captured when the second flash is turned on; The fusion unit is used to perform fusion processing on the at least one first-category image frame and the at least one second-category image frame to obtain a fused image.

8. A photographing device, characterized in that: The device comprises: A memory for storing computer executable instructions; A processor, connected to the memory, configured to implement the method according to any one of claims 1 to 6 by executing the computer executable instructions.

9. A chip, characterized in that: The chip comprises: A processor, configured to call and run a computer program from a memory, so that a device equipped with the chip executes the method according to any one of claims 1 to 6.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by at least one processor, the method according to any one of claims 1 to 6 is implemented.

11. A computer program product comprising a computer program or instructions, characterized in that When the computer program or instruction is executed by a processor, the method according to any one of claims 1 to 6 is implemented.

Images