Reference image determination method and device, electronic equipment and storage medium

Abstract

The invention provides a reference image determination method and device, electronic equipment and a storage medium, and the method comprises the steps: obtaining a plurality of frames of images collected at a plurality of continuous collection moments, and determining whether a shooting scene of the plurality of frames of images is a motion scene or not according to the motion data corresponding to each frame of image in the plurality of frames of images, the motion data represents motion data of a target object in the multiple frames of images, and / or motion data of the electronic equipment collecting the multiple frames of images, responding to a shooting scene of the multiple frames of images as a motion scene, determining a reference image from the multiple frames of images according to a shutter moment, sharpness information of each frame of image in the multiple frames of images and a corresponding collection moment, according to the invention, further recognition of the motion scene is realized, and then the reference image of which the sharpness and the time both meet the shooting requirements is screened from the multiple frames of images based on the sharpness information and the time information, so that the accuracy of determining the reference image is improved, and the shooting quality is improved.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to a reference image determination method, device, electronic device and storage medium. Background Art

[0002] When an electronic device takes photos, in order to improve the clarity of the captured image, a reference image is determined from multiple frames of images captured, and multiple frames of images are synthesized based on the reference image to obtain a displayable target image.

[0003] In actual scenarios, the electronic device or the photographed object may be in motion. In scenarios with motion, the reference image is usually determined based on the sharpness information of the image. However, the reference image determined in this way has low accuracy, which reduces the imaging quality. Summary of the invention

[0004] The present application aims to solve one of the technical problems in the related art at least to some extent.

[0005] To this end, the present application proposes a reference image determination method, device, electronic device and storage medium, which identify whether it is a motion scene based on the motion data corresponding to the image frame. In a motion scene, the reference image is determined from multiple frames of images based on the shutter moment, sharpness information of multiple frames of images and the corresponding acquisition moment, thereby meeting the sharpness requirements and imaging requirements, improving the accuracy of reference image determination, and thereby improving imaging quality.

[0006] In one aspect, an embodiment of the present application provides a method for determining a reference image, including:

[0007] Acquire multiple frames of images acquired at multiple consecutive acquisition moments;

[0008] Determining whether a shooting scene of the multiple frames of images is a motion scene according to motion data corresponding to each frame of the multiple frames of images; wherein the motion data represents motion data of a target object in the multiple frames of images and / or motion data of an electronic device that collects the multiple frames of images;

[0009] In response to the shooting scene of the multiple frames of images being a motion scene, a reference image is determined from the multiple frames of images according to a shutter moment, sharpness information of each frame of the multiple frames of images, and a corresponding acquisition moment.

[0010] Another aspect of the present application provides a reference image determination device, including:

[0011] An acquisition module, used to acquire multiple frames of images acquired at multiple consecutive acquisition moments;

[0012] An identification module, configured to determine whether a shooting scene of the multiple frames of images is a motion scene according to motion data corresponding to each frame of the multiple frames of images; wherein the motion data represents motion data of a target object in the multiple frames of images and / or motion data of an electronic device that collects the multiple frames of images;

[0013] A determination module is used to determine a reference image from the multiple frames of images in response to the shooting scene of the multiple frames of images being a motion scene, according to a shutter moment, sharpness information of each frame of the multiple frames of images and a corresponding acquisition moment.

[0014] Another aspect of the present application provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the method described in the above aspect is implemented.

[0015] Another aspect of the present application provides a non-transitory computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the method described in the aforementioned aspect is implemented.

[0016] Another aspect of the present application provides a computer program product on which a computer program is stored. When the program is executed by a processor, the method described in the above aspect is implemented.

[0017] The reference image determination method, device, electronic device and storage medium proposed in the present application obtain multiple frames of images acquired at multiple consecutive acquisition moments, and determine whether the shooting scene of the multiple frames of images is a motion scene based on the motion data corresponding to each frame in the multiple frames of images, wherein the motion data represents the motion data of the target object in the multiple frames of images, and / or the motion data of the electronic device that acquires the multiple frames of images. In response to the shooting scene of the multiple frames of images being a motion scene, the reference image is determined from the multiple frames of images based on the shutter moment, the sharpness information of each frame in the multiple frames of images and the corresponding acquisition moment, thereby achieving further identification of the motion scene. Furthermore, based on the sharpness information and the time information, the reference image whose sharpness and time meet the shooting requirements is screened out from the multiple frames of images, thereby improving the accuracy of the reference image determination and thus improving the shooting quality.

[0018] Additional aspects and advantages of the present application will be given in part in the description below, and in part will become apparent from the description below, or will be learned through the practice of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] The above and / or additional aspects and advantages of the present application will become apparent and easily understood from the following description of the embodiments in conjunction with the accompanying drawings, in which:

[0020] Figure 1A schematic diagram of a scene image with image jump display provided in an embodiment of the present application;

[0021] Figure 2 A schematic diagram of a flow chart of a reference image determination method provided in an embodiment of the present application;

[0022] Figure 3 A schematic flow chart of another reference image determination method provided in an embodiment of the present application;

[0023] Figure 4A A schematic diagram of a flow chart of a method for determining a reference image in a scenario provided by an embodiment of the present application;

[0024] Figure 4B A schematic diagram of the structure of a reference image determination device provided in an embodiment of the present application;

[0025] Figure 5 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0026] Embodiments of the present application are described in detail below, and examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to be used to explain the present application, and should not be construed as limiting the present application.

[0027] The following describes a reference image determination method, device, electronic device, and storage medium according to embodiments of the present application with reference to the accompanying drawings.

[0028] In the related art, in a scenario where a reference image needs to be determined for multi-frame synthesis, when determining the reference image, the multiple frames of images are sorted according to the values ​​of the sharpness information of each frame, and the frame with the largest sharpness is used as the reference image of the multi-frame synthesis algorithm. Then, the multiple frames of images and the reference image are used for image synthesis to obtain a frame of target image. However, for motion scenes, the image content of the reference image determined based on the sharpness information may be quite different from the image content of the preview image at the moment the shutter is clicked to take a picture. Then, the target image synthesized based on the reference image, also called the film image, also has a quite different image content from the preview image at the moment the shutter is clicked to take a picture. As an example, Figure 1 As shown, Figure 1 In the figure, the image contents in the preview image on the left and the finished image on the right are different, resulting in a visual jump between the imaged image and the preview image in the user's perception, which cannot meet the shooting quality requirements.

[0029] To this end, an embodiment of the present application provides a method for determining a reference image, which obtains multiple frames of images acquired at multiple consecutive acquisition moments, and determines whether the shooting scene of the multiple frames of images is a motion scene based on motion data corresponding to each frame in the multiple frames of images, wherein the motion data represents motion data of a target object in the multiple frames of images, and / or motion data of an electronic device that acquires the multiple frames of images. In response to the shooting scene of the multiple frames of images being a motion scene, a reference image is determined from the multiple frames of images based on a shutter moment, sharpness information of each frame in the multiple frames of images, and the corresponding acquisition moment, thereby achieving accurate recognition of the motion scene. Furthermore, based on the sharpness information and time information, a reference image whose sharpness and time meet the shooting requirements is screened out from the multiple frames of images, thereby improving the accuracy of determining the reference image and thus improving the shooting quality.

[0030] Figure 2 A flowchart of a reference image determination method provided in an embodiment of the present application.

[0031] The executor of the reference image determination method in the embodiment of the present application is a reference image determination device, which can be set in an electronic device. The electronic device can be a smart phone, a smart watch, smart glasses, a Pad, a handheld computer, etc., which is not limited in this embodiment.

[0032] like Figure 2 As shown, the method may include the following steps:

[0033] Step 201 : acquiring multiple frames of images acquired at multiple consecutive acquisition moments.

[0034] In the embodiment of the present application, multiple frames of images captured by the sensor of the electronic device at multiple consecutive capture times are obtained, and each frame of the image carries a corresponding timestamp, i.e., the capture time. As an implementation method, the multiple frames of images can be multiple frames of images captured when the shutter of the electronic device is pressed, and the time when the shutter is pressed, i.e., the shutter time, is also stored as the shutter timestamp.

[0035] Among them, the multi-frame images collected by the embodiment of the present application, as one implementation method, can be the multi-frame images collected by the sensor sensor starting from clicking the shutter to take pictures, that is, the multi-frame data collected at multiple consecutive collection moments after the shutter moment; as a second implementation method, it is the multi-frame images that have been cached when the shutter is clicked, that is, the multi-frame data collected at multiple consecutive collection moments stored before the shutter moment. As a third implementation method, it is the multi-frame images collected during the process of clicking the shutter, and the multi-frame images collected are, for example, 8 frames of images.

[0036] Step 202 : determining whether the shooting scene of the multiple frames of images is a motion scene according to the motion data corresponding to each frame of the multiple frames of images.

[0037] Wherein, the motion data represents the motion data of the target object in the multiple frames of images, and / or the motion data of the electronic device that collects the multiple frames of images. The motion data corresponding to each frame of images represents the motion data of the object in each frame of images, wherein, when there are multiple objects in the multiple frames of images, the target object is the moving object in the multiple frames of images, and if there are multiple moving objects, it can be any one moving object, or the object with the fastest moving speed among the multiple moving objects.

[0038] The motion data may be a velocity value or a displacement value.

[0039] In one implementation of the embodiment of the present application, the motion data is a speed value. According to the speed value corresponding to each frame image in the multi-frame image, it is determined whether the shooting scene of the multi-frame image is a motion scene. In the embodiment of the present application, the speed value corresponding to each frame image output when the Auto scene Detection (ASD) algorithm is started for scene detection is obtained. In one scenario, the electronic device for collecting images is in motion, then the speed value corresponding to each frame image is the speed value of the electronic device for collecting images at the time of collecting each frame image; in another scenario, the target object in the multi-frame image is in motion, then the speed value corresponding to each frame image is the speed value of the target object in the multi-frame image at the time of collecting each frame image; in another scenario, the electronic device for collecting images is in motion, and the target object in the multi-frame image is in motion, then the speed value corresponding to each frame image is the sum of the speed value of the target object at the time of collecting each frame image and the speed value of the electronic device for collecting multi-frame images at the time of collecting each frame image.

[0040] As an implementation method, the speed value corresponding to each frame image is compared with the speed threshold, and in response to the existence of at least one frame image corresponding to a speed value greater than the speed threshold, the shooting scene of the multiple frames of images is determined to be a moving scene. That is to say, if there is at least one frame corresponding to a speed value greater than the speed threshold, it means that the electronic device that collects the image or the target object in the multiple frames of images is moving, and the shooting scene of the multiple frames of images is determined to be a moving scene.

[0041] On the contrary, in response to the speed values ​​corresponding to each frame image being less than or equal to the speed threshold, the shooting scene of the multiple frames of images is determined to be a non-motion scene. The speed threshold can be set as required, for example, 20 meters per second. As another implementation method, the multiple frames of images are sorted according to the acquisition time of each frame image, and the speed difference between the speed values ​​corresponding to the two adjacent frames of images in the sorting is determined. In response to the existence of at least one speed difference greater than the set threshold, the shooting scene of the multiple frames of images is determined to be a motion scene. That is to say, if the speed difference between the two adjacent frames is large, it means that there is a motion state between the two adjacent frames, which means that the shooting scene of the multiple frames of images is a motion scene.

[0042] On the contrary, in response to each speed difference being less than or equal to the set threshold, the shooting scene of the multiple frames of images is determined to be a non-motion scene. The set threshold can be set according to the accuracy requirement and is not limited in this embodiment. For example, the set threshold is 10 m / s.

[0043] In another implementation of the embodiment of the present application, the motion data is a displacement value. According to the displacement values ​​corresponding to each frame image in the multi-frame image, it is determined whether the shooting scene of the multi-frame image is a motion scene. In one scenario, the displacement value corresponding to the Nth frame image is the offset of the position of the electronic device that collects the multi-frame image at the Nth frame relative to the position at the N-1 frame, that is, the displacement value. In another scenario, the displacement value corresponding to the Nth frame image is the offset of the position of the target object in the multi-frame image at the Nth frame relative to the position at the N-1 frame, that is, the displacement value, which can be used to indicate the movement of the object. Among them, the position of the target object at the Nth frame can be the position of the target object in the image of the Nth frame, or the position in the three-dimensional space at the acquisition time corresponding to the image of the Nth frame. As an example, if it is determined that object 1 in the nth frame image is at position A in the frame image, and in the n-1th frame image, object 1 is at position B in the frame image, the distance between position A and position B is the displacement value of object 1 in the Nth frame. If the displacement value is greater than the set distance threshold, it is considered that object 1 has displaced in space, and it is considered that object 1 is in motion, that is, object 1 is in motion; otherwise, it is considered that object 1 is not in motion, that is, object 1 is not in motion. Similarly, the principle of determining whether an electronic device is in motion based on its displacement value is the same and will not be repeated here.

[0044] Step 203, in response to determining that the scene is in motion, determining a reference image from the multiple image frames according to the shutter time, the sharpness information of each image frame in the multiple image frames and the corresponding acquisition time.

[0045] In an embodiment of the present application, in response to determining that a motion scene is in progress, the method of determining a reference image based only on the sharpness information of an image in the related art cannot be used. Instead, based on the sharpness information of multiple frames of images, multiple frames of candidate images whose sharpness information is greater than a set threshold or whose sharpness information ranks high are selected, thereby increasing the number of candidate images selected based on the sharpness information. Furthermore, a comparison is made based on the shutter moment and the acquisition moments corresponding to the multiple frames of candidate images whose sharpness information meets the set requirements, so as to determine the target acquisition moment closest to the shutter moment from the acquisition moments corresponding to the candidate images, and use a frame of candidate images corresponding to the target acquisition moment as the target candidate image, and the target candidate image as the reference image. Since the image captured closer to the shutter moment is closer to the image previewed by the user, the sharpness requirements for the image are met, and a large difference between the image previewed by the user at the shutter moment is avoided, visual jumps are avoided, and the imaging requirements for outputting the target image based on the reference image are met.

[0046] In the reference image determination method of the embodiment of the present application, multiple frames of images captured by the camera at multiple acquisition moments are obtained, and whether they belong to a motion scene is determined based on motion data corresponding to each frame of the image, wherein the motion data represents motion data of an object in the image, or represents motion data of a lens that captures the image. In response to determining that it belongs to a motion scene, a reference image is determined from the multiple frames of images based on shutter moments, sharpness information of the multiple frames of images, and the corresponding acquisition moments, thereby achieving further recognition of the motion scene, and screening out reference images whose sharpness and image content meet the shooting requirements from the multiple frames of images based on sharpness information and time information, thereby improving the accuracy of reference image determination and thus improving shooting quality.

[0047] Based on the above embodiments, Figure 3 A flowchart of another method for determining a reference image provided in an embodiment of the present application is provided. In the embodiment of the present application, a super-resolution shooting scene in a multi-frame synthesis scene is used as an example for explanation. Figure 3 As shown, the method comprises the following steps:

[0048] Step 301: Acquire multiple frames of images acquired at multiple consecutive acquisition moments.

[0049] Step 302 determines whether the shooting scene of the multiple frames of images is a motion scene according to the motion data corresponding to each frame of the multiple frames of images.

[0050] The explanations in the aforementioned embodiments are also applicable to this embodiment, and the principles are the same, which will not be repeated here.

[0051] Step 303: In response to determining that the scene belongs to a motion scene, a candidate image is determined according to sharpness information of each frame of the multiple frames of images.

[0052] Among them, the candidate images are multiple frames.

[0053] In one implementation of the embodiment of the present application, multiple frames of images are sorted in descending order according to sharpness information of each frame, and a preset number of multiple frames of images in the order are determined as candidate images. For example, the first three frames of images with larger sharpness information are determined as candidate images from the order.

[0054] In another implementation of the embodiment of the present application, the sharpness information of each frame image in the multiple frame images is compared with a set threshold, and multiple frame candidate images whose sharpness information is greater than the set threshold are determined from the multiple frame images.

[0055] As an implementation method, the sharpness information may include contour sharpness to meet the requirement for shooting clarity.

[0056] As another implementation method, the sharpness information may also include contour sharpness and color sharpness, so as to meet both the clarity requirement and the color reproduction requirement.

[0057] It should be noted that the sharpness information may also include other sharpness information related to the imaging quality, which is not limited in this embodiment.

[0058] In an embodiment of the present application, when determining candidate images that meet the sharpness requirements, multiple frames of candidate images are determined. Compared with the situation in the related art where one frame of image with the largest sharpness information is determined, the number of candidate images is increased in the present application. For the subsequent determination of the benchmark image, the range that can be screened is increased, thereby improving the accuracy of determining the benchmark image while taking into account the shooting quality.

[0059] Step 304: Determine a reference image from the candidate images according to the shutter time and the acquisition time of each candidate image.

[0060] As an implementation method, the acquisition time of each candidate image is compared with the shutter time respectively, and the target candidate image with the smallest difference with the shutter time is determined, that is, a frame of image that is closest in time to the shutter time is determined from the candidate images as the target candidate image, so that the image content of the target candidate image is as close as possible to the image content seen by the user when clicking the shutter, and the target candidate image is used as the reference image to avoid visual jumps. In the embodiment of the present application, from multiple frames of candidate images that meet the sharpness requirements, a target moment as close as possible to the shutter moment is determined from the acquisition time of each candidate image, and the candidate image corresponding to the target moment is used as the target candidate image, and then, the target candidate image is determined as the reference image, which achieves both taking into account the sharpness requirements of the image and avoiding the difference between the preview image at the shutter time, improving the accuracy of the reference image determination, and then improving the quality of the target image obtained by subsequent processing.

[0061] Step 305: Send the reference image to the photographing application so as to be displayed in the photographing application as an album preview image.

[0062] In the embodiment of the present application, the selected reference image will be uploaded to the shooting application and placed on the album thumbnail cover as a display frame for quick preview. This is the thumbnail that the user sees in the lower left corner of the shooting application after clicking to take a photo, so that the user can quickly preview it in the form of a thumbnail. At the same time, since it takes a certain amount of time to perform image synthesis in the background, it can also avoid the user thinking that the processing time is long, thereby improving user satisfaction.

[0063] Step 306 , pixel synthesis is performed on multiple frames of images according to the reference image to obtain a frame of target image.

[0064] In an embodiment of the present application, the reference image and multiple frames of images are converted in color space, that is, the collected multiple frames of original RAW images are converted from the red, green, and blue RGB color space to the YUV color space, and processed into YUVnv12 or nv21 data for easy processing by the image processor. Then, the SR algorithm will perform pixel synthesis of multiple frames of YUV data based on the selected reference image to synthesize a frame of target image or film image with high clarity and close to the preview screen, and store it in the album of the captured image according to the set image format, such as jpeg format, to meet the shooting requirements.

[0065] Furthermore, the application converts the target image from the YUV color space to the RGB color space through hardware encoding so that the target image can be displayed on the display device of the electronic device. This is because the display device is generally based on the RGB color space. If the display device is based on other color spaces, the target image can be converted to the corresponding color space to meet the display requirements, which is not limited in this embodiment.

[0066] In the reference image determination method of the embodiment of the present application, in the super-resolution shooting scene, due to the existence of unrecognized motion scenes, the motion data corresponding to each frame image is used to identify whether it is a motion scene, and when it is identified as a motion scene, multiple frames of candidate images are screened out based on the sharpness information, and then the acquisition time and shutter time of each frame of the candidate image are compared to determine the target candidate image whose difference with the shutter time is less than a threshold as the reference image, thereby avoiding the jump between the preview of clicking the shutter time and the reference image caused by motion, and the poor clarity caused by the failure to accurately determine the reference image during the synthesis process, so that the final film has higher clarity and better effect.

[0067] Based on the above embodiments, Figure 4A A flowchart of a method for determining a reference image in a scenario provided by an embodiment of the present application is shown as follows: Figure 4AAs shown, the method includes:

[0068] After turning on the camera, the Auto Scene Detection (ASD) algorithm is started for scene detection. During preview, if it is identified based on the motion level and it is determined that the triggering motion capture algorithm is met, it is determined to be a motion capture scene, and the motion capture algorithm is used for multi-frame synthesis processing; if there is motion according to the motion level, that is, the electronic device that collects the image or the target object that has motion in the scene to be collected is in motion, but the motion capture algorithm is not successfully triggered, or it is in accordance with the triggering motion capture algorithm, but the scene fails to successfully trigger the motion capture algorithm, and the zoom parameter ZOOM is above 1.4x, or the zoom parameter ZOOM is above 1.4x, but the electronic device that collects the image or the target object that has motion in the scene to be collected are both in a stationary state, then the Super Resolution (SR) algorithm is triggered by default, that is, it is determined to be an SR shooting scene, that is, the SR algorithm is used for multi-frame synthesis processing; in the SR scene, including the scene where the electronic device that collects the image or the target object that has motion in the scene to be collected is in motion, therefore, the recognition of the motion scene is added to further identify the motion scene in the SR scene. As shown in Figure 4, in the case of identification as an SR scene, in response to detecting that the shutter is triggered, multiple frames of images acquired at multiple consecutive acquisition moments are acquired, and based on the speed values ​​corresponding to each frame in the multiple frames, it is determined whether the shooting scene of the multiple frames is a motion scene, wherein the motion data represents the speed value of the target object in the multiple frames, and / or the speed value of the electronic device that acquires the multiple frames.

[0069] If it is determined that the shooting scene of the multiple-frame images is a motion scene, the motion type value is set to 1. Then, the Anchor pick algorithm is used to select the first three frames of images with larger sharpness information based on the sharpness information, and then based on the shutter moment, that is, the shutter timestamp, select the frame closest to the shutter timestamp as the reference image. Specifically, the reference image is determined from the multiple-frame images according to the shutter moment, the sharpness information of each frame in the multiple-frame images and the corresponding acquisition time.

[0070] Then, the reference image is sent to the shooting application to be displayed as an album preview image in the shooting application.

[0071] And, according to the reference image, multiple frames of images are pixel-synthesized to obtain a frame of target image, that is, after conversion from RGB space to YUV space, synthesis processing is performed according to the SR algorithm.

[0072] In the embodiments of the present application, in motion scenes, the problem of jumps between the preview and the finished film when clicking the shutter due to motion, and the problem of poor clarity caused by failure to accurately determine the reference image during the synthesis process is solved, thereby improving the clarity of the finished film and providing better images.

[0073] It should be noted that in other motion scenes, for example, motion capture scenes, the reference image determination method of the above embodiment of the present application can also be used to determine the reference frame. The principle is the same and will not be repeated here.

[0074] Among them, the explanations in the aforementioned method embodiment are also applicable to this embodiment, the principles are the same, and will not be repeated here.

[0075] In order to implement the above embodiment, the embodiment of the present application further proposes a reference image determination device.

[0076] Figure 4B A schematic diagram of the structure of a reference image determination device provided in an embodiment of the present application.

[0077] like Figure 4B As shown, the device may include:

[0078] An acquisition module 41 is used to acquire multiple frames of images acquired at multiple consecutive acquisition moments;

[0079] The identification module 42 is used to determine whether the shooting scene of the multiple frames of images is a motion scene according to the motion data corresponding to each frame of the multiple frames of images; wherein the motion data represents the motion data of the target object in the multiple frames of images and / or the motion data of the electronic device that collects the multiple frames of images;

[0080] The determination module 43 is used to determine a reference image from the multiple frames of images in response to the shooting scene of the multiple frames of images being a motion scene according to the shutter time, the sharpness information of each frame of the multiple frames of images and the corresponding acquisition time.

[0081] Further, in an implementation of the embodiment of the present application, the motion data is a speed value, and the identification module 42 is specifically used to:

[0082] Comparing the speed value corresponding to each frame of the multiple frames of images with the speed threshold value respectively;

[0083] In response to the existence of a speed value corresponding to at least one frame of image being greater than a speed threshold, determining that the shooting scene of the multiple frames of image is a motion scene;

[0084] In response to the speed value corresponding to each frame of image being less than or equal to the speed threshold, it is determined that the shooting scene of the multiple frames of image is a non-motion scene.

[0085] In an implementation of the embodiment of the present application, the motion data is a speed value, and the identification module 42 is specifically configured to:

[0086] Sorting the multiple frames of images according to the acquisition time of each frame of image;

[0087] Determine a speed difference between the speed values ​​corresponding to the two adjacent frames of images according to the speed values ​​corresponding to the two adjacent frames of images in the sorting;

[0088] In response to at least one of the speed difference values ​​being greater than a set threshold, determining that the shooting scene of the multiple frames of images is a motion scene;

[0089] In response to each of the speed differences being less than or equal to the set threshold, it is determined that the shooting scenes of the multiple frames of images are non-motion scenes.

[0090] In one implementation of the embodiment of the present application, the determination module 43 is specifically configured to:

[0091] Determine a candidate image based on the sharpness information of each frame of the multiple frames; wherein the candidate images are multiple frames; and determine the reference image from the candidate images based on the shutter moment and the acquisition moment of each of the candidate images.

[0092] In one implementation of the embodiment of the present application, the determination module 43 is specifically configured to:

[0093] The acquisition time of each candidate image is compared with the shutter time respectively, and the target candidate image with the smallest difference with the shutter time is determined; and the target candidate image is used as the reference image.

[0094] In one implementation of the embodiment of the present application, the determination module 43 is specifically configured to:

[0095] Sorting the multiple frames of images in descending order according to the sharpness information of each frame of image;

[0096] From the ranking, a preset number of multiple frames of images ranked first are determined as the candidate images.

[0097] In one implementation of the embodiment of the present application, the device further includes:

[0098] The display module is used to send the reference image to the shooting application so as to be displayed as an album preview image in the shooting application.

[0099] In one implementation of the embodiment of the present application, the device further includes:

[0100] The processing module is used to perform pixel synthesis on the multiple frames of images according to the reference image to obtain a frame of target image.

[0101] It should be noted that the above explanation of the method embodiment is also applicable to the device of this embodiment and will not be repeated here.

[0102] An embodiment of the present application provides a reference image determination device, which obtains multiple frames of images acquired at multiple consecutive acquisition moments, and determines whether the shooting scene of the multiple frames of images is a motion scene based on motion data corresponding to each frame in the multiple frames of images, wherein the motion data represents motion data of a target object in the multiple frames of images, and / or motion data of an electronic device that acquires the multiple frames of images. In response to the shooting scene of the multiple frames of images being a motion scene, a reference image is determined from the multiple frames of images based on a shutter moment, sharpness information of each frame in the multiple frames of images, and the corresponding acquisition moment, thereby achieving further recognition of the motion scene. Furthermore, based on the sharpness information and time information, a reference image whose sharpness and time meet the shooting requirements is screened out from the multiple frames of images, thereby improving the accuracy of reference image determination and thus improving the shooting quality.

[0103] In order to implement the above embodiments, the present application also proposes an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the method described in the above method embodiments is implemented.

[0104] In order to implement the above embodiments, the present application also proposes a non-temporary computer-readable storage medium on which a computer program is stored. When the program is executed by a processor, the method described in the above method embodiments is implemented.

[0105] In order to implement the above embodiments, the present application also proposes a computer program product on which a computer program is stored. When the computer program is executed by a processor, the method described in the above method embodiments is implemented.

[0106] Figure 5 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. For example, the electronic device 800 may be a mobile phone, a computer, a digital broadcast terminal, a message transceiver device, a game console, a tablet device, a medical device, a fitness device, a personal digital assistant, etc.

[0107] Reference Figure 5 , the electronic device 800 may include one or more of the following components: a processing component 802 , a memory 804 , a power component 806 , a multimedia component 808 , an audio component 810 , an input / output (I / O) interface 812 , a sensor component 814 , and a communication component 816 .

[0108] The processing component 802 generally controls the overall operation of the electronic device 800, such as operations associated with display, phone calls, data communications, camera operations, and recording operations. The processing component 802 may include one or more processors 820 to execute instructions to complete all or part of the steps of the above-mentioned method. In addition, the processing component 802 may include one or more modules to facilitate the interaction between the processing component 802 and other components. For example, the processing component 802 may include a multimedia module to facilitate the interaction between the multimedia component 808 and the processing component 802.

[0109] The memory 804 is configured to store various types of data to support operations on the electronic device 800. Examples of such data include instructions for any application or method operating on the electronic device 800, contact data, phone book data, messages, pictures, videos, etc. The memory 804 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk.

[0110] The power component 806 provides power to the various components of the electronic device 800. The power component 806 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power for the electronic device 800.

[0111] The multimedia component 808 includes a screen that provides an output interface between the electronic device 800 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touch screen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touch, slide, and gestures on the touch panel. The touch sensor may not only sense the boundaries of the touch or slide action, but also detect the duration and pressure associated with the touch or slide operation. In some embodiments, the multimedia component 808 includes a front camera and / or a rear camera. When the electronic device 800 is in an operating mode, such as a shooting mode or a video mode, the front camera and / or the rear camera may receive external multimedia data. Each front camera and rear camera may be a fixed optical lens system or have a focal length and optical zoom capability.

[0112] The audio component 810 is configured to output and / or input audio signals. For example, the audio component 810 includes a microphone (MIC), and when the electronic device 800 is in an operating mode, such as a call mode, a recording mode, and a voice recognition mode, the microphone is configured to receive an external audio signal. The received audio signal can be further stored in the memory 804 or sent via the communication component 816. In some embodiments, the audio component 810 also includes a speaker for outputting audio signals.

[0113] I / O interface 812 provides an interface between processing component 802 and peripheral interface modules, such as keyboards, click wheels, buttons, etc. These buttons may include but are not limited to: home button, volume button, start button, and lock button.

[0114] The sensor assembly 814 includes one or more sensors for providing various aspects of status assessment for the electronic device 800. For example, the sensor assembly 814 can detect the open / closed state of the electronic device 800, the relative positioning of the components, such as the display and keypad of the electronic device 800, and the sensor assembly 814 can also detect the position change of the electronic device 800 or a component of the electronic device 800, the presence or absence of contact between the user and the electronic device 800, the orientation or acceleration / deceleration of the electronic device 800, and the temperature change of the electronic device 800. The sensor assembly 814 may include a proximity sensor configured to detect the presence of a nearby object without any physical contact. The sensor assembly 814 may also include an optical sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, the sensor assembly 814 may also include an acceleration sensor, a gyroscope sensor, a magnetic sensor, a pressure sensor, or a temperature sensor.

[0115] The communication component 816 is configured to facilitate wired or wireless communication between the electronic device 800 and other devices. The electronic device 800 can access a wireless network based on a communication standard, such as WiFi, 4G or 5G, or a combination thereof. In an exemplary embodiment, the communication component 816 receives a broadcast signal or broadcast-related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component 816 also includes a near field communication (NFC) module to facilitate short-range communication. For example, the NFC module can be implemented based on radio frequency identification (RFID) technology, infrared data association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology and other technologies.

[0116] In an exemplary embodiment, the electronic device 800 may be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components to perform the above methods.

[0117] In an exemplary embodiment, a non-transitory computer-readable storage medium including instructions is also provided, such as a memory 804 including instructions, and the instructions can be executed by a processor 820 of an electronic device 800 to perform the above method. For example, the non-transitory computer-readable storage medium can be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, etc.

[0118] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present application. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine the different embodiments or examples described in this specification and the features of the different embodiments or examples, without contradiction.

[0119] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "first" and "second" may explicitly or implicitly include at least one of the features. In the description of this application, the meaning of "plurality" is at least two, such as two, three, etc., unless otherwise clearly and specifically defined.

[0120] Any process or method description in a flowchart or otherwise described herein may be understood to represent a module, fragment or portion of code comprising one or more executable instructions for implementing the steps of a custom logical function or process, and the scope of the preferred embodiments of the present application includes alternative implementations in which functions may not be performed in the order shown or discussed, including performing functions in a substantially simultaneous manner or in the reverse order depending on the functions involved, which should be understood by technicians in the technical field to which the embodiments of the present application belong.

[0121] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as an ordered list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by an instruction execution system, device or apparatus (such as a computer-based system, a system including a processor, or other system that can fetch instructions from an instruction execution system, device or apparatus and execute the instructions), or in combination with these instruction execution systems, devices or apparatuses. For the purpose of this specification, "computer-readable medium" can be any device that can contain, store, communicate, propagate or transmit a program for use by an instruction execution system, device or apparatus, or in combination with these instruction execution systems, devices or apparatuses. More specific examples of computer-readable media (a non-exhaustive list) include the following: an electrical connection with one or more wires (electronic device), a portable computer disk box (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable and programmable read-only memory (EPROM or flash memory), a fiber optic device, and a portable compact disk read-only memory (CDROM). In addition, the computer-readable medium may even be paper or other suitable medium on which the program is printed, since the program may be obtained electronically, for example, by optically scanning the paper or other medium and then editing, interpreting or processing in other suitable ways if necessary, and then stored in a computer memory.

[0122] It should be understood that the various parts of the present application can be implemented by hardware, software, firmware or a combination thereof. In the above-mentioned embodiments, multiple steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented by hardware, as in another embodiment, it can be implemented by any one of the following technologies known in the art or their combination: a discrete logic circuit having a logic gate circuit for implementing a logic function for a data signal, a dedicated integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0123] A person skilled in the art may understand that all or part of the steps in the method for implementing the above-mentioned embodiment may be completed by instructing related hardware through a program, and the program may be stored in a computer-readable storage medium, which, when executed, includes one or a combination of the steps of the method embodiment.

[0124] In addition, each functional unit in each embodiment of the present application may be integrated into a processing module, or each unit may exist physically separately, or two or more units may be integrated into one module. The above-mentioned integrated module may be implemented in the form of hardware or in the form of a software functional module. If the integrated module is implemented in the form of a software functional module and sold or used as an independent product, it may also be stored in a computer-readable storage medium.

[0125] The storage medium mentioned above may be a read-only memory, a magnetic disk or an optical disk, etc. Although the embodiments of the present application have been shown and described above, it can be understood that the above embodiments are exemplary and cannot be understood as limiting the present application. A person of ordinary skill in the art may change, modify, replace and modify the above embodiments within the scope of the present application.

Claims

1. A method for determining a reference image, characterized in that: include: Acquire multiple frames of images acquired at multiple consecutive acquisition moments; Determining whether a shooting scene of the multiple frames of images is a motion scene according to motion data corresponding to each frame of the multiple frames of images; wherein the motion data represents motion data of a target object in the multiple frames of images and / or motion data of an electronic device that collects the multiple frames of images; In response to the shooting scene of the multiple frames of images being a motion scene, a reference image is determined from the multiple frames of images according to a shutter moment, sharpness information of each frame of the multiple frames of images, and a corresponding acquisition moment.

2. The method according to claim 1, characterized in that The motion data is a speed value, and determining whether the shooting scene of the multiple frames of images is a motion scene according to the motion data corresponding to each frame of the multiple frames of images includes: Comparing the speed value corresponding to each frame of the multiple frames of images with the speed threshold value respectively; In response to the existence of a speed value corresponding to at least one frame of image being greater than a speed threshold, determining that the shooting scene of the multiple frames of image is a motion scene; In response to the speed value corresponding to each frame of image being less than or equal to the speed threshold, it is determined that the shooting scene of the multiple frames of image is a non-motion scene.

3. The method according to claim 1, characterized in that The motion data is a speed value, and determining whether the shooting scene of the multiple frames of images is a motion scene according to the motion data corresponding to each frame of the multiple frames of images includes: Sorting the multiple frames of images according to the acquisition time of each frame of image; Determine a speed difference between the speed values ​​corresponding to the two adjacent frames of images according to the speed values ​​corresponding to the two adjacent frames of images in the sorting; In response to at least one of the speed difference values ​​being greater than a set threshold, determining that the shooting scene of the multiple frames of images is a motion scene; In response to each of the speed differences being less than or equal to the set threshold, it is determined that the shooting scenes of the multiple frames of images are non-motion scenes.

4. The method according to claim 1, characterized in that Determining the reference image from the multiple frames of images according to the shutter time, the sharpness information of each frame of the multiple frames of images and the corresponding acquisition time includes: Determine a candidate image according to the sharpness information of each frame of the multiple frames of images; wherein the candidate image is multiple frames; The reference image is determined from the candidate images according to the shutter time and the acquisition time of each of the candidate images.

5. The method according to claim 4, characterized in that The step of determining the reference image from the candidate images according to the shutter moment and the acquisition moment of each of the candidate images comprises: Compare the acquisition time of each candidate image with the shutter time respectively, and determine the target candidate image with the smallest difference with the shutter time; The target candidate image is used as the reference image.

6. The method according to claim 4, characterized in that The step of determining the candidate image according to the sharpness information of each frame of the multiple frames of images comprises: Sorting the multiple frames of images in descending order according to the sharpness information of each frame of image; From the ranking, a preset number of multiple frames of images ranked first are determined as the candidate images.

7. The method according to any one of claims 1 to 6, characterized in that: The method further comprises: The reference image is sent to a photographing application to be displayed as an album preview image in the photographing application.

8. The method according to any one of claims 1 to 6, characterized in that: The method further comprises: According to the reference image, the multiple frames of images are pixel-synthesized to obtain a frame of target image.

9. A reference image determination device, characterized in that: include: An acquisition module, used to acquire multiple frames of images acquired at multiple consecutive acquisition moments; An identification module, configured to determine whether a shooting scene of the multiple frames of images is a motion scene according to motion data corresponding to each frame of the multiple frames of images; wherein the motion data represents motion data of a target object in the multiple frames of images and / or motion data of an electronic device that collects the multiple frames of images; A determination module is used to determine a reference image from the multiple frames of images in response to the shooting scene of the multiple frames of images being a motion scene, according to a shutter moment, sharpness information of each frame of the multiple frames of images and a corresponding acquisition moment.

10. An electronic device, characterized in that: The method comprises a memory, a processor and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the method according to any one of claims 1 to 8 is implemented.

11. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 8 is implemented.

Images