Shooting mode determination method, device, electronic device, and storage medium

Abstract

The present disclosure relates to a shooting mode determination method, device, electronic device, and storage medium. [Solution] The method includes the steps of obtaining at least two consecutive images captured by an image capture device, determining whether the at least two consecutive images have a similarity less than a preset threshold, and setting the capture mode of the image capture device to a high dynamic range mode if the at least two consecutive images have a similarity less than the preset threshold, and when the at least two consecutive images have a similarity less than the preset threshold, the image capture mode of the image capture device is in the high dynamic range mode, and the image captured and output by the image capture device is a high dynamic range image. Thus, the embodiment of the present disclosure provides an effective detection mechanism for the image capture device to reasonably determine whether to turn on the high dynamic range mode, optimizing the detection result, and at the same time, avoiding the phenomenon of "ghosting" in images captured by a user subsequently due to image similarity detection, thereby ensuring image capture quality. [Representative diagram] Figure 1

Description

[Technical Field]

[0001] The present disclosure relates to the technical field of image processing, and in particular to a shooting mode determination method, device, electronic device, and storage medium.

[0002] CROSS-REFERENCE TO RELATED APPLICATIONS This disclosure claims priority to a Chinese patent application filed with the State Intellectual Property Office of the People's Republic of China on December 17, 2020, bearing application number 202011502690.2 and entitled "Method, device, electronic device and storage medium for determining photographing mode," the entire contents of which are incorporated herein by reference. [Background technology]

[0003] High-Dynamic Range (HDR) images can provide more dynamic range and image detail because more ambient lighting information is recorded.

[0004] Currently, during the shooting process, users usually manually turn on the high dynamic range mode of the image capturing device according to the specific shooting environment. This not only increases the number of times the user manually operates the image capturing device, but also makes it impossible to achieve an intelligent shooting effect because the turning on of the high dynamic range mode lacks an effective automatic trigger mechanism. Summary of the Invention [Problem to be solved by the invention]

[0005] To solve the above technical problems or to solve at least part of the above technical problems, embodiments of the present disclosure provide a shooting mode determination method, device, electronic device, and storage medium. [Means for solving the problem]

[0006] According to a first aspect, an embodiment of the present disclosure provides a shooting mode determination method, comprising: acquiring at least two frames of consecutive images captured by an image capture device; determining whether there are two consecutive frames of images whose similarity is less than a preset threshold value; and if there are no two frames of images whose similarity is less than a preset threshold value among the at least two frames of consecutive images, setting the image capture device to a high dynamic range mode; When the imaging mode of the image capturing device is in the high dynamic range mode, the image captured and output by the image capturing device is a high dynamic range image.

[0007] According to a second aspect, an embodiment of the present disclosure further provides a shooting mode determination device, an image capture module for capturing at least two frames of consecutive images captured by an image capture device; a similarity determination module for determining whether there are two frames of images in the at least two consecutive frames whose similarity is less than a preset threshold; a mode setting module for setting a photographing mode of the image photographing device to a high dynamic range mode when there are no two frames of images whose similarity is less than a preset threshold value among the at least two frames of consecutive images; When the imaging mode of the image capturing device is in the high dynamic range mode, the image captured and output by the image capturing device is a high dynamic range image.

[0008] According to a third aspect, an embodiment of the present disclosure further provides an electronic device, the electronic device including a processor, a memory for storing executable instructions of the processor, and a camera for collecting images, the processor reading the executable instructions from the memory and executing the executable instructions to realize any of the shooting mode determination methods provided in the embodiment of the present disclosure.

[0009] According to a fourth aspect, an embodiment of the present disclosure further provides a computer-readable storage medium having a computer program stored therein, the computer program realizing any of the shooting mode determination methods provided in the embodiments of the present disclosure when executed by a processor.

[0010] The technical solution provided in the embodiments of the present disclosure has the following advantages over the prior art: When it is determined that at least two consecutive frames of images captured by the image capturing device do not contain two frames whose similarity is less than a preset threshold, the embodiment of the present disclosure sets the image capturing device to a high dynamic range mode to capture a high dynamic range image, thereby providing an effective detection mechanism for the image capturing device to rationally determine whether to turn on the high dynamic range mode, optimizing the detection result, solving the problem of the conventional image capturing process requiring the user to manually trigger the image capturing mode, which lacks intelligence, and ensuring the accuracy of determining when to turn on the high dynamic range mode. At the same time, image similarity detection is performed before turning on the high dynamic range mode, avoiding the "ghost" phenomenon that occurs in images captured after the user, and ensuring the quality of image capture. [Brief explanation of the drawings]

[0011] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the principles of the disclosure.

[0012] In order to more clearly describe the embodiments of the present disclosure or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art are briefly introduced below. Obviously, those skilled in the art can further obtain other drawings based on these drawings without any creative efforts. [Figure 1] 1 is a flowchart of a shooting mode determination method provided in an embodiment of the present disclosure. [Figure 2]10 is a flowchart of another shooting mode determination method provided in an embodiment of the present disclosure. [Figure 3] FIG. 1 is a structural schematic diagram of a shooting mode determination device provided in an embodiment of the present disclosure; [Figure 4] 1 is a structural schematic diagram of an electronic device provided in an embodiment of the present disclosure. DETAILED DESCRIPTION OF THE INVENTION

[0013] In order to make the above objects, features, and advantages of the present disclosure more clearly understood, the solutions of the present disclosure are further described below. It should be noted that, if not contradictory, the embodiments of the present disclosure and the features in the embodiments can be combined with each other.

[0014] In the following description, numerous specific details are set forth in order to provide a thorough understanding of the present disclosure; however, the present disclosure may be implemented in other ways different from those described herein, and it is apparent that the embodiments in the specification are merely some of the embodiments of the present disclosure, but not all of the embodiments.

[0015] 1 is a flowchart of a shooting mode determination method provided in an embodiment of the present disclosure, which can be applied to a case where an image capture device determines whether to turn on a high dynamic range mode to capture an image. The method provided in the embodiment of the present disclosure can be performed by a shooting mode determination device, which can be realized in software and / or hardware, and can be integrated into any electronic device with computing capabilities, such as a device integrated with an image capture device, such as a mobile terminal or tablet, or an image capture device dedicated to image capture, such as a camera device.

[0016] To ensure the accuracy of detection regarding the shooting mode of the image capture device, the method provided in the embodiment of the present disclosure can perform detection repeatedly and continuously according to a fast execution cycle in a preview mode of the image capture device, where the execution cycle is usually in units of seconds. The preview mode may refer to a mode in which, after the image capture device is turned on, the image capture device collects and captures images (e.g., contents displayed in the viewfinder frame of the camera) in real time for the current shooting environment, so that the user can preview the image effect before triggering a shooting command.

[0017] In addition, the method provided in the embodiments of the present disclosure can be applied to a shooting scene where the moving speed of the object is greater than a speed threshold, and the speed threshold can be determined according to the shooting scene, such as a moving train, which contributes to improving the shooting quality of dynamic objects.

[0018] As shown in FIG. 1, the shooting mode determination method provided in the embodiment of the present disclosure may include S101 to S103.

[0019] S101: At least two frames of consecutive images are acquired by an image capturing device.

[0020] For example, an image stream (i.e., a preview stream) may be generated in real time in a preview mode of the image capture device, and at least two consecutive frames of images captured by the image capture device, for example, at least two consecutive frames of images with the current image as the end frame, may be acquired from the image stream. Then, by calculating image similarity, it may be determined whether the image capture device in the current shooting environment is set to a high dynamic range mode, and a high dynamic range image may be output in the high dynamic range mode. The number of frames acquired for consecutive images is not specifically limited in the embodiments of the present disclosure and may be reasonably set as needed.

[0021] S102: Determine whether there are at least two consecutive frames of images with a similarity smaller than a preset threshold.

[0022] Specifically, for at least two consecutive frames of images acquired, a similarity between each pair of images can be calculated to determine whether the at least two consecutive frames of images contain two frames whose similarity is less than a preset threshold. A preset number of images can be screened from the at least two consecutive frames of images according to a preset image screening policy, and the similarity between each pair of images can be calculated based on the screened images to determine whether the screened images contain two frames whose similarity is less than the preset threshold. The preset number of frames can be at least two. The image screening policy can include a policy for screening a preset number of images from the at least two consecutive frames of images acquired according to a preset frame interval, and can further include a policy for screening images corresponding to at least two designated frames among the at least two consecutive frames of images acquired. The preset frame interval value and the designated frames can both be set as appropriate while ensuring the accuracy of detection of the shooting mode of the image capturing device, and are not particularly limited in the embodiments of the present disclosure.

[0023] A preset threshold value of the similarity can also be set as appropriate. The similarity calculation can be realized by any available image similarity calculation method in the prior art, such as Euclidean distance calculation, cosine similarity calculation, etc., and the embodiments of the present disclosure are not limited thereto.

[0024] By calculating the similarity between two frames based on at least two consecutive frames of acquired images, the accuracy of determining whether two frames of images have a similarity below a preset threshold can be guaranteed. Calculating the similarity between two frames based on a preset number of screened images contributes to improving the efficiency of similarity calculation. Additionally, considering that images are acquired continuously and in real time in the preview mode of the image capture device, changes in the current state of the subject can be reflected continuously and in real time. Therefore, by determining whether there are two frames of images with a similarity below a preset threshold among the acquired consecutive images of at least two frames based on the screened multiple frames of images, high accuracy of determination can be ensured.

[0025] For example, the current subject is a moving train, and an image stream of the moving train can be captured in real time using the preview mode of the image capture device. Assuming that at least two consecutive frames of images captured within one execution period (usually within a few seconds) of the solution provided in the embodiments of the present disclosure include four frames of images, the difference between the first and fourth frames of images is relatively maximum because the train's position is constantly changing. If the similarity between the first and fourth frames of images is greater than a preset threshold, the similarity between any two frames of images among the at least two consecutive frames of images is also greater than the preset threshold. That is, there are no two frames of images among the at least two consecutive frames of images whose similarity is less than the preset threshold. Therefore, calculating the similarity between specific two frames of images not only improves calculation efficiency but also ensures judgment accuracy. If the similarity between the images of the first and fourth frames is less than the preset threshold, regardless of whether there are two frames of images whose similarity is less than the preset threshold in the images of the remaining number of frames, a judgment result may already be obtained, i.e., there are two frames of images whose similarity is less than the preset threshold in the consecutive images of at least two frames acquired, and in this case, the image capturing device is not set to the high dynamic range mode.

[0026] Furthermore, in the process of calculating the similarity between two frame images, The method may include the steps of: dividing the image of each frame involved in the similarity calculation to obtain image blocks belonging to the image of each frame, the size of the image blocks being set as appropriate, and embodiments of the present disclosure are not limited thereto; calculating sub-similarity between image blocks belonging to different images of the two frame images; and determining the similarity between the images of the two frames according to the sub-similarity between the image blocks. Exemplarily, the step of determining the similarity between the images of the two frames according to the sub-similarity between the image blocks includes the steps of: calculating a proportion of image blocks in any one of the two frame images whose sub-similarity is smaller than a sub-similarity threshold, the sub-similarity threshold being set according to a similarity calculation request; and determining the similarity between the images of the two frames according to the calculated proportion. If the proportion of the image occupied by an image block whose sub-similarity is smaller than the sub-similarity threshold is greater than the proportion threshold, the proportion threshold can also be flexibly set, for example, if set to 80%, it can be determined that the similarity of the images of the two frames is smaller than the preset threshold, otherwise it can be determined that the similarity of the images of the two frames is equal to or greater than the preset threshold.In addition, the similarity of the images of the two frames can also be obtained by calculating the sub-similarity between the image blocks, for example, by performing a weighted addition calculation, to obtain a single overall similarity value.

[0027] Dividing an image into blocks and calculating the similarity between them contributes to improving the accuracy of calculating the similarity between images. In addition, image blocks with low similarity can be used to determine the position of the subject in each of the two frames of image.

[0028] S103: If there are no two consecutive frames of images with a similarity smaller than the preset threshold, set the image capturing mode of the image capturing device to a high dynamic range mode.

[0029] When the imaging mode of the image capturing device is in a high dynamic range mode, the image captured and output by the image capturing device is a high dynamic range image.

[0030] If it is determined that there are no two consecutive frames whose similarity is less than the preset threshold, then it is considered that there is a high degree of similarity between the two consecutive frames. Furthermore, in the process of capturing images using the high dynamic range mode of the current image capturing device, there is no "ghost" in the high dynamic range image obtained by image synthesis. Therefore, it is possible to set the image capturing device to the high dynamic range mode. If it is determined that there are two consecutive frames whose similarity is less than the preset threshold, then it is considered that there is a certain difference between the two consecutive frames. Furthermore, in the process of capturing images using the high dynamic range mode of the current image capturing device, there is a possibility that there is a "ghost" in the high dynamic range image obtained by image synthesis. Therefore, it is not appropriate to set the image capturing device to the high dynamic range mode. "Ghost" refers to the phenomenon in which interference images are present in the synthesized image when there is a large difference between the two frames involved in synthesis, and this is an issue regarding the quality of the synthesized image.

[0031] The embodiments of the present disclosure provide an effective detection mechanism for the image capture device to rationally determine whether to turn on the high dynamic range mode by setting the capture mode of the image capture device to high dynamic range mode when it is determined that there are no two frames of images with a similarity smaller than a preset threshold among at least two consecutive frames captured by the image capture device, thereby optimizing the detection result, solving the problem of the user having to manually trigger the capture mode in the traditional capture process, which lacks intelligence, and ensuring the accuracy of determining when to turn on the high dynamic range mode. At the same time, image similarity detection is performed before turning on the high dynamic range mode, which avoids the phenomenon of "ghosting" in images subsequently captured by the user and ensures the quality of image capture.

[0032] Optionally, before the step of setting the capture mode of the image capture device to a high dynamic range mode: determining whether or not there is an image whose luminance information satisfies target luminance information in at least two consecutive frames of images, the target luminance information being used to define an image luminance condition that triggers a high dynamic range mode; Accordingly, when there are no two frames of images having a similarity smaller than the preset threshold value among at least two consecutive frames of images, the step of setting the photographing mode of the image photographing device to a high dynamic range mode includes: The method further includes setting the photographing mode of the image photographing device to a high dynamic range mode when at least two consecutive frames of images have luminance information that satisfies the target luminance information and at least two consecutive frames of images do not have two frames of images whose similarity is lower than a preset threshold. At the same time, the method further ensures the accuracy and rationality of the timing of turning on the high dynamic range mode of the image photographing device by determining whether to turn on the high dynamic range mode based on the image similarity and image luminance information.

[0033] Specifically, the step of determining whether or not there is an image whose luminance information satisfies the target luminance information among at least two consecutive frames of images may include the steps of: determining an image histogram for each frame of the at least two consecutive frames of images; determining luminance information for each corresponding frame based on the image histogram; and determining whether or not there is an image whose luminance information satisfies the target luminance information among at least two consecutive frames of images based on the determined image luminance information. Alternatively, the step of screening a preset number of images from at least two consecutive frames of images in accordance with an image screening policy, where the current preset number of frames may be at least one, determining an image histogram for the screening image, determining luminance information for the corresponding image based on the image histogram for the screening image, and determining whether or not there is an image whose luminance information satisfies the target luminance information among the screened images based on the determined image luminance information. The image screening policy may include a policy for screening a preset number of images from at least two consecutive frames of images acquired in accordance with an interval of the preset frame number, and may include a policy for setting an image corresponding to at least one designated frame among the acquired at least two consecutive frames of images as the screened image.

[0034] In addition, the image screening policy adopted in the process of determining image luminance information may be the same as the image screening policy adopted in the process of calculating the similarity between images, that is, the images involved in the two operation processes may be the same images. For example, from at least two consecutive frames of images, two frames of images with an interval of a target number of frames are determined, and it is determined whether there is an image whose luminance information satisfies the target luminance information among the two frames of images with an interval of the target number of frames.

[0035] Optionally, the step of determining whether or not there is an image whose luminance information satisfies target luminance information in at least two frames of consecutive images comprises: The method may include determining a first image region having an image brightness less than a first brightness threshold and determining a second image region having an image brightness greater than a second brightness threshold for each frame of at least two consecutive images or each frame of images screened based on the image screening policy, wherein the first brightness threshold is less than the second brightness threshold, and the specific values ​​thereof may be determined appropriately.

[0036] Determine whether the ratio value between the first image region and the second image region is within a region threshold interval, which may be determined according to a situation that meets the image luminance region distribution when the high dynamic range mode of the image capturing device is turned on.

[0037] If there are at least two consecutive frames of images in which the ratio value between the first image region and the second image region is within the region threshold interval, or if there are at least two consecutive frames of images in which the ratio value between the first image region and the second image region is within the region threshold interval, it indicates that there are at least two consecutive frames of images in which the luminance information satisfies the target luminance information. In other words, there are images in which the luminance distribution is uneven, or conversely, there are no images in which the luminance distribution is uneven.

[0038] Optionally, the step of determining whether or not there is an image whose luminance information satisfies target luminance information in at least two frames of consecutive images comprises: The method may include determining whether at least two consecutive frames of images contain an image whose brightness value is greater than a third brightness threshold or less than a fourth brightness threshold, or determining whether images screened based on the image screening policy contain an image whose brightness value is greater than the third brightness threshold or less than a fourth brightness threshold. The fourth brightness threshold is smaller than the third brightness threshold and can be used to screen images with non-uniform brightness distribution, and a specific value may be determined appropriately.

[0039] If at least two consecutive frames of images have a luminance value greater than the third luminance threshold or less than the fourth luminance threshold, or if images screened based on the image screening policy have a luminance value greater than the third luminance threshold or less than the fourth luminance threshold, it indicates that at least two consecutive frames of images have luminance information that meets the target luminance information. In other words, an image with uneven luminance distribution exists, or conversely, does not exist.

[0040] Image brightness information is used as the basis for determining whether the image capturing device should turn on the high dynamic range mode. In the prior art solutions, the realization principle of determining whether to turn on the high dynamic range mode of the image capturing device based on image brightness can be referred to, and the above examples should not be understood as specific limitations on the embodiments of the present disclosure.

[0041] In addition, with regard to the step of determining whether at least two consecutive frames of images have an image whose luminance information satisfies the target luminance information and the step of determining whether at least two consecutive frames of images have two frames whose similarity is lower than a preset threshold, there is no strict limit on the order of the two operations, i.e., image luminance information detection may be performed before or after image similarity detection. As a preferred solution, image luminance information detection can be performed after image similarity detection, thereby improving the efficiency of detecting whether to turn on the high dynamic range mode while ensuring the accuracy of determining when to turn on the high dynamic range mode.

[0042] Furthermore, when images are screened from at least two consecutive frames of images based on the image screening policy, if the image luminance information detection operation is performed after the image similarity detection operation, at least one more frame of images can be screened from the images involved in the image similarity calculation, and it can be determined whether or not there is an image whose luminance information satisfies the target luminance information in the at least one frame of images currently screened. For example, any one frame of images from the two frames of images involved in the similarity calculation is selected as the target for image luminance information detection, i.e., it is determined whether or not the luminance information of the arbitrary one frame of image satisfies the target luminance information. If yes, it is considered that there is an image whose luminance information satisfies the target luminance information in at least two consecutive frames of images, thereby further improving the efficiency of detecting whether to turn on the high dynamic range mode.

[0043] 2 is a flowchart of another shooting mode determination method provided in an embodiment of the present disclosure, which can be further optimized and extended based on the above technical solution and combined with each of the above optional embodiments. As shown in FIG. 2, the method can include S201 to S208. S201: At least two frames of consecutive images are acquired by an image capturing device. S202: From at least two consecutive frames of images, two target frames spaced apart by a target number of frames are determined.

[0044] The target frame number may be set appropriately, for example, to three or four frames. For example, if the frame interval is set to three frames, five frames of images are included in at least two consecutive frames, and the images involved in the similarity calculation are the first and fifth frames, thereby improving calculation efficiency. Additionally, considering that image acquisition is continuous and real-time, changes in the current state of the subject can be continuously reflected in real time. Therefore, even if two target frames separated by the target frame number are used to determine whether two frames of images with a similarity lower than a preset threshold are present in the acquired at least two consecutive frames, high determination accuracy can be ensured.

[0045] S203: Determine whether the similarity of the target images of the two frames is less than a preset threshold.

[0046] If no, that is, the similarity of the two frames of the target image is equal to or greater than the preset threshold, execute operation S204 and operation S205; if yes, execute operation S208.

[0047] S204: Set the number of image synthesis frames in the process of synthesizing a high dynamic range image in high dynamic range mode based on at least one frame of the two target image frames.

[0048] In the high dynamic range mode of the image capturing device, the final high dynamic range image captured is obtained by combining multiple frames of images automatically captured by the image capturing device. Before capturing an image using the high dynamic range mode of the image capturing device, the number of image composite frames that the image capturing device currently needs to capture can be rationally determined, thereby improving the rationality of capturing high dynamic range images and ensuring the capture quality.

[0049] Specifically, the image capture device uses a mapping relationship between the predetermined image brightness information and the upper and lower limits of an exposure parameter (Exposure Value (EV)) for the image capture device based on the brightness information of at least two consecutive frames or the brightness information of at least one of the two target frames to determine the upper and lower limits of the exposure parameter in the current high dynamic range mode of the image capture device, i.e., the value range of the exposure parameter. Then, the number of composite frames of the high dynamic range image can be determined according to the determined upper and lower limits of the exposure parameter, where one exposure parameter corresponds to one frame of image. For example, if the upper limit of the exposure parameter of the image capture device is determined to be +1 and the lower limit is determined to be -1, the values ​​of the exposure parameter of the image capture device are +1, 0, and -1, and the corresponding number of composite frames involved in the synthesis of the high dynamic range image is three frames. That is, the image capture device needs to capture three frames of images in the process of obtaining the high dynamic range image, and then synthesize the high dynamic range image.

[0050] For example, the mapping relationship between the upper and lower limit values ​​of the exposure parameters of the image capturing device may be utilized, whereby the predetermined image brightness information is used to determine the upper and lower limit values ​​of multiple sets of exposure parameters in the high dynamic range mode of the image capturing device based on the brightness information of each frame of at least two consecutive frames of images, and then the average values ​​of the upper and lower limit values ​​of the exposure parameters may be calculated based on the determined upper and lower limit values ​​of the multiple sets of exposure parameters, and finally the number of composite frames of the high dynamic range image may be determined based on the average values ​​of the upper and lower limit values ​​of the exposure parameters.

[0051] For example, according to the image screening policy, a preset number of images may be screened from at least two consecutive images, and the current preset number of frames may be at least one frame. The upper and lower limit values ​​of at least one set of exposure parameters in the high dynamic range mode of the image capturing device may be determined based on the luminance information of the screened images with the preset number of frames. Then, based on the determined upper and lower limit values ​​of the exposure parameters, average values ​​of the upper and lower limit values ​​of the exposure parameters may be calculated, and finally, the number of composite frames of the high dynamic range image may be determined based on the average values ​​of the upper and lower limit values ​​of the exposure parameters. Of course, if only the upper and lower limit values ​​of one set of exposure parameters are determined, the corresponding average values ​​are the upper and lower limit values ​​of the exposure parameters themselves.

[0052] In addition, the image screening policy adopted in the process of determining the number of composite frames of a high dynamic range image may be the same as the image screening policy adopted in the process of calculating the similarity between images, and the images involved in the two operation processes may be the same images.

[0053] S205: It is determined that there are no two consecutive frames of images whose similarity is less than a preset threshold value.

[0054] S206: The photographing mode of the image photographing device is set to a high dynamic range mode.

[0055] S207: In response to the shooting command, the image shooting device is called to shoot the same number of images as the number of image synthesis frames, and the same number of images as the number of image synthesis frames are synthesized to output a high dynamic range image.

[0056] The capture command may be triggered by a user touching a capture control on an electronic device, for example, a user touching a shutter of an image capture device to trigger the capture command. How the image capture device captures a high dynamic range image in high dynamic range mode can be realized by referring to conventional high dynamic range image capture logic, and is not particularly limited in the embodiments of the present disclosure.

[0057] S208: Reject setting the imaging mode of the image capturing device to the high dynamic range mode.

[0058] If it is determined that the similarity of the two target images is less than the preset threshold, it is determined that there are at least two consecutive images with a similarity less than the preset threshold, and there is a certain difference between the at least two consecutive images. Furthermore, in the process of capturing images using the high dynamic range mode of the current image capturing device, there may be "ghosts" in the high dynamic range image obtained by image synthesis, so it is not appropriate to set the image capturing device to the high dynamic range mode.

[0059] Based on the above technical solution, optionally, the step of setting an image synthesis frame number in the process of synthesizing a high dynamic range image in a high dynamic range mode based on at least one frame of the two target images includes: inputting at least one frame of the two target images into an exposure parameter determination model and outputting upper and lower limit values ​​of the exposure parameters of the image capturing device, the exposure parameter determination model being a pre-trained model for determining upper and lower limit values ​​of the exposure parameters of the image capturing device; and setting the number of image synthesis frames in the process of synthesizing a high dynamic range image in high dynamic range mode according to the upper and lower limit values ​​of the exposure parameters.

[0060] In addition, each frame image of the acquired consecutive images of at least two frames, or a preset number of frames of images screened from the consecutive images of at least two frames, may be used as input to an exposure parameter determination model, and upper and lower limit values ​​of the exposure parameters of the image capturing device may be output.

[0061] Furthermore, the training process of the exposure parameter determination model includes the following steps: A step of obtaining a sample image and annotation results of the upper and lower limit values ​​of the exposure parameters of the image capturing device corresponding to the sample image, where, for example, a user takes a plurality of frames of images of the same subject in advance, each frame using a different exposure parameter of the image capturing device, and uses the image taken when the exposure parameter is 0 as the sample image, and determines the upper and lower limit values ​​of the exposure parameters according to the exposure parameters of the image capturing device when taking the remaining images, and uses these as the exposure annotation results of the sample image; and The step of training an exposure parameter determination model is to take a sample image as input, annotate the upper and lower limit values ​​of the exposure parameters of the image capture device corresponding to the sample image as output, and train the model to obtain the exposure parameter determination model. That is, in the model training process, image brightness information is automatically extracted based on the sample image, and a mapping relationship between the image brightness information and the upper and lower limit values ​​of the exposure parameters of the image capture device can be learned.

[0062] The embodiment of the present disclosure does not limit the model algorithm used in the model training process, and may include, but is not limited to, a linear classification support vector machine (linearSVC), etc. In addition, the machine learning model trained based on the linear classification support vector machine has fast operation efficiency and can meet the real-time requirements of the present solution, and at the same time, has high output accuracy and can ensure the accuracy of the upper and lower limit values ​​of the exposure parameters in the current shooting environment of the image capturing device, thereby ensuring the accurate determination of the number of composite frames of a high dynamic range image.

[0063] Furthermore, in the embodiments of the present disclosure, the type of the pre-trained exposure parameter determination model may include a multi-task model and a single-task model. A multi-task model refers to a model that can simultaneously output upper and lower limit values ​​of the exposure parameters of an image capture device, thereby improving the efficiency of outputting exposure parameters. A single-task model refers to a model that can only output upper or lower limit values ​​of the exposure parameters of an image capture device, in which case the model training process is simple and easy to implement. The type of model can be determined according to requirements in specific applications, and the embodiments of the present disclosure are not limited thereto.

[0064] For the single-task model type, the exposure parameter determination model in the embodiment of the present disclosure may include an upper exposure limit determination model and a lower exposure limit determination model. The upper exposure limit determination model is used to determine the upper limit value of the exposure parameter of the image capture device, and the lower exposure limit determination model is used to determine the lower limit value of the exposure parameter of the image capture device. The training implementation principles of the two models are the same as the model training process described above, and will not be described again here.

[0065] The embodiments of the present disclosure provide an effective detection mechanism for rationally determining whether to turn on the high dynamic range mode by setting the image capture device to a high dynamic range mode when it is determined that there are no two consecutive frames of images captured by the image capture device whose similarity is less than a preset threshold. The detection result is optimized to solve the problem of the traditional image capture process, in which the user must manually trigger the image capture mode, resulting in a lack of intelligence. This ensures the accuracy of determining when to turn on the high dynamic range mode, avoids the "ghost" phenomenon that occurs in images captured after the user, and ensures the quality of the image capture. At the same time, before setting the image capture device to the high dynamic range mode (or before using the high dynamic range mode of the image capture device to capture an image), the number of composite frames involved in synthesizing the high dynamic range image can be rationally determined, ensuring the rationality of capturing the high dynamic range image and reducing the number of unnecessary captured frames. The number of captured frames of the image capture device can be dynamically determined during the process of obtaining the high dynamic range image, further improving the intelligence of the image capture.

[0066] 3 is a structural schematic diagram of a photographing mode determination device provided in an embodiment of the present disclosure. The embodiment of the present disclosure may be applied to a case where an image capture device determines whether to turn on a high dynamic range mode to capture an image. The device provided in the embodiment of the present disclosure may be implemented in software and / or hardware and integrated into any electronic device with computing capabilities, such as a mobile terminal or tablet integrated with an image capture device, or a dedicated image capture device such as a camera device.

[0067] As shown in FIG. 3 , the photographing mode determination device provided in the embodiment of the present disclosure includes an image acquisition module 301 , a similarity determination module 302 , and a mode setting module 303 . The image acquisition module 301 acquires at least two frames of consecutive images taken by an image capture device; The similarity determination module 302 determines whether there are at least two consecutive frames of images whose similarity is less than a preset threshold; The mode setting module 303 sets the photographing mode of the image photographing device to a high dynamic range mode when there are no two consecutive images with a similarity smaller than a preset threshold value among at least two consecutive images; When the imaging mode of the image capturing device is in a high dynamic range mode, the image captured and output by the image capturing device is a high dynamic range image.

[0068] Optionally, the similarity determination module 302: a target image determining unit for determining two target images from at least two consecutive images, the two target images being spaced apart by a target number of frames; a first similarity determination unit for determining whether the similarity of the two frames of the target image is less than a preset threshold; and a second similarity determining unit for determining, when the similarity of the two frames of the target image is equal to or greater than the preset threshold, that there are no two frames of images in the at least two frames of consecutive images whose similarity is less than the preset threshold.

[0069] Optionally, the shooting mode determination device provided in the embodiments of the present disclosure comprises: The method further includes a synthesis frame number determination module for setting the number of image synthesis frames in the process of synthesizing a high dynamic range image in high dynamic range mode based on at least one of the two target images when the similarity of the two target images is greater than or equal to a preset threshold (i.e., the similarity of the two target images is greater than or equal to a preset threshold).

[0070] Optionally, the synthesis frame number determination module: a parameter upper and lower limit determination unit for inputting at least one frame of the two target images into an exposure parameter determination model and outputting upper and lower limit values ​​of the exposure parameters of the image capture device; a synthesis frame number determination unit for setting the number of image synthesis frames in the process of synthesizing a high dynamic range image in high dynamic range mode according to the upper and lower limit values ​​of the exposure parameters;

[0071] Optionally, the shooting mode determination device provided in the embodiments of the present disclosure comprises: a sample acquisition module for acquiring a sample image and annotation results of upper and lower limit values ​​of exposure parameters of an image capture device corresponding to the sample image; The method further includes a model training module that receives a sample image as input, outputs annotation results of upper and lower limit values ​​of exposure parameters of an image capturing device corresponding to the sample image, and performs training to obtain an exposure parameter determination model.

[0072] Optionally, the shooting mode determination device provided in the embodiments of the present disclosure comprises: The image output module further includes an image capture device that captures the same number of images as the number of image synthesis frames in response to a capture command, synthesizes the same number of images as the number of image synthesis frames, and outputs a high dynamic range image.

[0073] Optionally, the shooting mode determination device provided in the embodiments of the present disclosure comprises: The method further includes a luminance image determination module that determines whether or not there is an image whose luminance information satisfies target luminance information in at least two consecutive frames of images, the target luminance information being used to define an image luminance condition that triggers a high dynamic range mode; In response, the mode setting module 303 specifically: If there are images in at least two consecutive frames whose luminance information satisfies the target luminance information and there are no images in at least two consecutive frames whose similarity is smaller than the preset threshold, the imaging mode of the image capturing device is set to a high dynamic range mode.

[0074] The photographing mode determination device provided in the embodiments of the present disclosure can execute any photographing mode determination method provided in the embodiments of the present disclosure, and has corresponding functional modules and beneficial effects of the executed method. For content not described in detail in the apparatus embodiments of the present disclosure, reference can be made to the description in any method embodiments of the present disclosure.

[0075] 4 is a structural schematic diagram of an electronic device provided in an embodiment of the present disclosure, for illustrating and explaining an electronic device that executes a shooting mode determination method in an embodiment of the present disclosure. The electronic device in an embodiment of the present disclosure may include, but is not limited to, mobile terminals such as mobile phones, laptops, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablets), PMPs (portable multimedia players), and in-vehicle terminals (e.g., in-vehicle navigation terminals), as well as fixed terminals such as digital TVs and desktop computers. The electronic device shown in FIG. 4 is merely an example and should not impose any limitations on the functionality and scope of use of the embodiment of the present disclosure.

[0076] As shown in FIG. 4 , the electronic device 400 may include a processing unit (e.g., a central processor, a graphics processor, etc.) 401, which may perform various appropriate operations and processes according to a program stored in a read-only memory (ROM) 402 or loaded from a storage device 408 into a random access memory (RAM) 403. The RAM 403 further stores various programs and data necessary for the operation of the electronic device 400. The processing unit 401, the ROM 402, and the RAM 403 are connected to each other via a bus 404. An input / output (I / O) interface 405 is also connected to the bus 404. The electronic device 400 may further include a camera 410 for collecting multimedia data, such as images and videos, and the camera 410 may also be connected to other devices or modules via the bus 404.

[0077] Typically, input devices 406, including, for example, a touch screen, touch pad, keyboard, mouse, microphone, accelerometer, gyroscope, etc.; output devices 407, including, for example, a liquid crystal display (LCD), speaker, vibrator, etc.; storage devices 408, including, for example, a magnetic tape, hard disk, etc.; and communication devices 409 may be connected to the I / O interface 405. A camera may also be connected to the I / O interface 405 as a type of input device 406. The communication devices 409 enable the electronic device 400 to exchange data with other devices through wireless or wired communication. While FIG. 4 illustrates the electronic device 400 having various devices, it should be understood that not all of the devices shown are required to be implemented or included. Instead, more or fewer devices may be implemented or included.

[0078] In particular, according to embodiments of the present disclosure, the processes described above with reference to the flowcharts may be implemented as a computer software program. For example, embodiments of the present disclosure include a computer program product, which includes a computer program carried on a non-transitory computer-readable medium, the computer program including program code for performing the methods illustrated in the flowcharts. In such embodiments, the computer program may be downloaded and installed from a network via the communication device 409, or may be installed from the storage device 408, or may be installed from the ROM 402. When the computer program is executed by the processing device 401, it performs the functions defined in the methods of the embodiments of the present disclosure.

[0079] It should be noted that the computer-readable medium described above in this disclosure may be a computer-readable signal medium, a computer-readable storage medium, or any combination thereof. The computer-readable storage medium may be, for example, but is not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of the computer-readable storage medium may include, but are not limited to, an electrical connection having one or more wires, a portable computer magnetic disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In embodiments of the present disclosure, the computer-readable storage medium may be any tangible medium that contains or stores a program, which may be used by or in connection with an instruction execution system, apparatus, or device. In embodiments of the present disclosure, a computer-readable signal medium may include a propagated data signal, either in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals may take a variety of forms, including, but not limited to, electromagnetic signals, optical signals, or any suitable combination of the above. The computer-readable signal medium may be any computer-readable medium other than a computer-readable storage medium. The computer-readable signal medium may transmit, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device.The program code contained on the computer readable medium may be transmitted by any suitable medium, including, but not limited to, electrical wire, fiber optic cable, RF (radio frequency), etc., or any suitable combination of the above.

[0080] In some embodiments, clients and servers may communicate using any network protocol now known or later developed, such as HyperText Transfer Protocol (HTTP), and may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include local area networks (LANs), wide area networks (WANs), the World Wide Web (e.g., the Internet), end-to-end networks (e.g., ad-hoc end-to-end networks), and other networks now known or later developed.

[0081] The computer-readable medium may be included in the electronic device, or may be separate from the electronic device.

[0082] When the computer-readable medium carries one or more programs and the one or more programs are executed by the electronic device, the electronic device includes the steps of acquiring at least two consecutive frames of images captured by an image capturing device, determining whether there are two consecutive frames of images whose similarity is less than a preset threshold value among the at least two consecutive frames of images, and setting the shooting mode of the image capturing device to a high dynamic range mode if there are no two consecutive frames of images whose similarity is less than the preset threshold value among the at least two consecutive frames of images, and when the shooting mode of the image capturing device is in the high dynamic range mode, the image captured and output by the image capturing device is a high dynamic range image.

[0083] Computer program code for carrying out the operations of the present disclosure may be written in one or more programming languages, or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and further including, but not limited to, conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may execute entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. When referring to a remote computer, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0084] The flowcharts and block diagrams in the figures illustrate architecture, functions, and operations that can be implemented in accordance with systems, methods, and computer program products according to various embodiments of the present application. In this regard, each block in a flowchart or block diagram may represent a module, program segment, or portion of code, which includes one or more executable instructions for implementing a given logical function. It should be noted that, in some alternative implementations, the functions shown in the blocks may occur in an order different from that shown in the figures. For example, two blocks shown in succession may actually be executed essentially in parallel, or in some cases, may be executed in the reverse order, depending on the functionality involved. Furthermore, each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, may be implemented in a system using dedicated hardware that performs a given function or operation, or in a combination of dedicated hardware and computer instructions.

[0085] The modules and units described in the embodiments of the present disclosure may be implemented in a software or hardware manner. The names of the modules and units may not necessarily be limited to the modules or units themselves. For example, an image acquisition module may be further described as "a module for acquiring at least two frames of consecutive images captured by an image capture device."

[0086] The functions described herein above may be performed, at least in part, by one or more hardware logic elements. For example, exemplary hardware logic elements that may be used include, but are not limited to, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), etc.

[0087] In this disclosure, a machine-readable medium may be a tangible medium that contains or stores a program that may be used by or in combination with an instruction execution system, device, or apparatus. The machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. The machine-readable medium includes, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or apparatus, or any suitable combination of the above. More specific examples of machine-readable storage media include an electrical connection of one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact magnetic disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0088] It should be noted that, in this specification, when combined with specific context, relational terms such as "first" and "second" are used to distinguish one entity or operation from another, and do not necessarily require or imply the existence of any such actual relationship or ranking between those entities or operations. Furthermore, the terms "comprise," "comprises," or any other variations thereof are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that includes a set of elements further includes those elements as well as other elements not expressly listed or inherent in such process, method, article, or apparatus. Absent further limitations, an element defined by the phrase "comprises ..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes the element.

[0089] The foregoing are merely specific embodiments of the present disclosure, intended to enable those skilled in the art to understand and practice the present disclosure. Various modifications to these examples will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other examples without departing from the spirit or scope of the present disclosure. Therefore, the present disclosure is not intended to be limited to these examples, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A shooting mode determination method executed by a shooting mode determination device, comprising: acquiring at least two frames of consecutive images captured by an image capture device; determining whether there are two consecutive frames of images whose similarity is less than a preset threshold value; a step of determining at least one first image frame from images that are involved in the calculation of the similarity and satisfy the condition that the similarity is equal to or greater than the preset threshold, when the at least two consecutive images of the ... If there are no two frames of images whose similarity is less than a preset threshold value among the at least two consecutive frames of images and there is an image whose luminance information satisfies target luminance information among the at least one frame of the first image, setting the image capturing mode of the image capturing device to a high dynamic range mode, When the imaging mode of the image capturing device is in the high dynamic range mode, the image captured and output by the image capturing device is a high dynamic range image. A method characterized by:

2. The step of determining whether there are two frames of images whose similarity is less than a preset threshold value among the at least two frames of consecutive images includes: determining, from the at least two consecutive frames of images, two target frames of images spaced apart by a target number of frames; determining whether the similarity of the two frames of the target image is less than the preset threshold; and determining, when the similarity of the two frames of the target image is equal to or greater than the preset threshold, that there are no two frames of images in the at least two frames of consecutive images whose similarity is less than the preset threshold.

2. The method of claim 1.

3. After the step of determining whether the similarity of the two frames of the target image is less than the preset threshold, and further comprising: when the similarity of the two target images is equal to or greater than the preset threshold, setting an image synthesis frame number in the process of synthesizing the high dynamic range image in the high dynamic range mode based on at least one of the two target images.

3. The method according to claim 2.

4. The step of setting the number of image synthesis frames in the process of synthesizing the high dynamic range image in the high dynamic range mode based on at least one frame of the two target images, inputting at least one of the two target image frames into an exposure parameter determination model to output upper and lower limit values ​​for exposure parameters of the image capture device; and setting an image synthesis frame number in the process of synthesizing the high dynamic range image in the high dynamic range mode according to the upper and lower limit values ​​of the exposure parameter.

4. The method according to claim 3.

5. The training process of the exposure parameter determination model includes: obtaining a sample image and annotation results of upper and lower limit values ​​of exposure parameters of an image capture device corresponding to the sample image; and obtaining the exposure parameter determination model by training the sample image as an input and the annotation results of the upper and lower limit values ​​of the exposure parameters of the image capture device corresponding to the sample image as an output.

5. The method according to claim 4.

6. After the step of setting the imaging mode of the image capturing device to a high dynamic range mode, and further comprising the step of calling the image capture device in response to a capture command, capturing images in the same number as the number of image synthesis frames, synthesizing the images in the same number as the number of image synthesis frames, and outputting the high dynamic range image.

4. The method according to claim 3.

7. The step of determining whether or not an image whose luminance information satisfies target luminance information exists in the first image of at least one frame includes: for each of the at least one frame of first images, determining a first image region having an image brightness less than a first brightness threshold and determining a second image region having an image brightness greater than a second brightness threshold, the first brightness threshold being less than the second brightness threshold; determining whether a ratio value of the area between the first image region and the second image region is within an area threshold interval; determining that an image whose luminance information satisfies target luminance information is present in the at least two frames of consecutive images when the value of the ratio of the area between the first image region and the second image region is within the area threshold interval; The method of claim 1 , comprising:

8. A photography mode determination device, an image capture module for capturing at least two frames of consecutive images captured by an image capture device; a similarity determination module for determining whether there are two frames of images in the at least two consecutive frames whose similarity is less than a preset threshold; a luminance image determination module that, when there are no two consecutive images of which similarity is less than a preset threshold value, determines at least one first image from images that are involved in the calculation of the similarity and satisfy the similarity being equal to or greater than a preset threshold value, and determines whether or not there is an image of which luminance information satisfies target luminance information among the at least one first image of the frame, the target luminance information being used to define an image luminance condition that triggers a high dynamic range mode; a mode setting module for setting a photographing mode of the image photographing device to a high dynamic range mode when there are no two frames of images whose similarity is less than a preset threshold value among the at least two frames of consecutive images and there is an image whose luminance information satisfies target luminance information among the at least one frame of the first image; When the imaging mode of the image capturing device is the high dynamic range mode, the image captured and output by the image capturing device is a high dynamic range image. An apparatus characterized in that

9. a processor; a memory for storing executable instructions for said processor; a camera for collecting images; The processor reads the executable instructions from the memory and executes the executable instructions to implement the method of any one of claims 1 to 7. An electronic device characterized by:

10. A computer-readable storage medium having a computer program stored thereon; When the computer program is executed by a processor, Implementing the method according to any one of claims 1 to 7, A storage medium comprising:

Images