Image generation method and device

Abstract

The embodiment of the invention provides an image generation method and device. The method comprises the steps that N original image frames which are continuous in time are acquired, N is the number of the original image frames needed for synthesizing a high-definition image, and at least part of the image frames in the N original image frames are multiplexing image frames, the multiplexed image frames are original image frames which are adjacent to the current high-definition image generation and are used during the previous high-definition image generation, and synthesizing the N original image frames to generate a final high-definition image.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and in particular to an image generation method and device. Background Art

[0002] At present, when users take photos, N raw image frames are needed to synthesize a high-definition image. The raw image frames can be images generated by sensors, usually called raw images. In theory, the frame collection time of a photo sensor = (N / frame rate), and the next photo needs to be taken after the frame collection time of the previous photo is over. Therefore, when taking high-definition images, the time between two consecutive high-definition photos is limited by the frame collection time, and there is a long time interval between photos.

[0003] The current solution used by various manufacturers to reduce the shooting time interval is to convert multiple frames into a single frame (ie N = 1), thereby shortening the frame collection time for each shot and reducing the shooting time interval. However, the side effect is that the image quality is poor and cannot meet the requirements of shooting high-definition images. Summary of the invention

[0004] In order to overcome the problems existing in the related art, this specification provides an image generation method and device.

[0005] In a first aspect of any embodiment of this specification, a method for generating an image is provided, the method comprising:

[0006] Acquire N original image frames that are continuous in time, where N is the number of original image frames required to synthesize a high-definition image; at least some of the N original image frames are multiplexed image frames, and the multiplexed image frames are original image frames used in the previous high-definition image generation adjacent to the current high-definition image generation;

[0007] The N original image frames are synthesized to generate a final high-definition image.

[0008] According to a second aspect of any one of the embodiments of this specification, there is provided an image generating device, the device comprising:

[0009] An original image frame acquisition module is used to acquire N original image frames that are continuous in time, where N is the number of original image frames required to synthesize a high-definition image; at least some of the N original image frames are multiplexed image frames, and the multiplexed image frames are original image frames used in the previous high-definition image generation adjacent to the current high-definition image generation;

[0010] The high-definition image generation module is used to perform synthesis processing on the N original image frames to generate a final high-definition image.

[0011] According to a third aspect of any embodiment of this specification, there is provided an electronic device, including:

[0012] processor;

[0013] a memory for storing processor-executable instructions;

[0014] The processor implements the method described in any embodiment of this specification by running the executable instructions.

[0015] According to a fourth aspect of any embodiment of the present specification, a computer-readable storage medium is provided, on which computer instructions are stored, and when the instructions are executed by a processor, the steps of the method described in any embodiment of the present specification are implemented.

[0016] The technical solutions provided by the embodiments of this specification may have the following beneficial effects:

[0017] According to the above embodiments, by acquiring N original image frames that are consecutive in time, synthesizing the N original image frames to generate a final high-definition image, and using the N original image frames that are consecutive in time to synthesize to generate a high-definition image, the number of original image frames required for synthesizing the high-definition image is not reduced, and the image quality requirements of the photographed image can be guaranteed; at the same time, since the original image frames used in the last high-definition image generation are reused, the generation time of new original image frames can be reduced, and the frame collection time of the photograph can be shortened, thereby reducing the interval between photographing times without affecting the image quality.

[0018] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present specification. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0020] Figure 1 is a schematic diagram of frame receiving time in a traditional photographing method according to an exemplary embodiment of the present specification;

[0021] Figure 2 is a flowchart of an image generating method according to an exemplary embodiment of the present specification;

[0022] Figure 3 is a schematic diagram of obtaining N original image frames according to an exemplary embodiment of this specification;

[0023] Figure 4 is a schematic diagram of another method of acquiring N original image frames according to an exemplary embodiment of this specification;

[0024] Figure 5 is a flowchart of a method for synthesizing N original image frames according to an exemplary embodiment of the present specification;

[0025] Figure 6 is a schematic structural diagram of an electronic device according to an exemplary embodiment of the present specification;

[0026] Figure 7 It is a block diagram of an image generating device according to an exemplary embodiment of the present specification. DETAILED DESCRIPTION

[0027] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The implementations described in the following exemplary embodiments do not represent all implementations consistent with this specification. Instead, they are merely examples of devices and methods consistent with some aspects of this specification as detailed in the appended claims.

[0028] The terms used in this specification are for the purpose of describing specific embodiments only and are not intended to limit this specification. The singular forms "a", "the" and "the" used in this specification and the appended claims are also intended to include plural forms unless the context clearly indicates otherwise. It should also be understood that the term "and / or" used herein refers to and includes any or all possible combinations of one or more associated listed items.

[0029] It should be understood that although the terms first, second, third, etc. may be used in this specification to describe various information, this information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other. For example, without departing from the scope of this specification, the first information may also be referred to as the second information, and similarly, the second information may also be referred to as the first information. Depending on the context, the word "if" as used herein may be interpreted as "at the time of" or "when" or "in response to determining".

[0030] A high-definition image needs to be synthesized from N raw image frames (raw images), and the theoretical frame collection time = (N / frame rate). When taking photos, users cannot continuously capture high-definition images at a high frequency. Specifically, there is a long pause interval between two high-definition shots, and the pause interval is limited by the frame collection time = (N / frame rate).

[0031] See also Figure 1 , Figure 1The figure shows the frame collection time under the traditional shooting mode. N is the number of original image frames required to synthesize a high-definition image, fps is the frame rate, t is the time point of the r-th shooting, and t+N / fps is the time point when the r+1-th shooting is allowed. N / fps is the frame collection time of the r-th shooting, that is, the shooting time interval between two high-definition shootings.

[0032] Currently, the solution to reduce the shooting time interval is to reduce the number of original image frames for synthesizing high-definition images and convert multiple frames into a single frame (ie, N=1). However, the image quality generated by a single original image frame is poor and cannot meet the requirements of high-definition images.

[0033] In order to solve the above-mentioned problem, an embodiment of this specification proposes an image generation method. Before introducing the image generation method, in order to make the description of the method easier to understand, some concepts involved in the description of the embodiment of this specification are first explained:

[0034] 1) Frame rate

[0035] The frame rate is the number of refreshes per second. Assuming that the frame rate is represented by f1, for example, the sensor used to capture and generate images can generate f1 raw images per second. Common frame rates are 30fps, 60fps, etc. For example, if the frame rate is 30fps, it means that the sensor generates 30 raw images per second.

[0036] 2) Photo frame rate

[0037] The photo frame rate is the number of photos taken per second. Assuming that the photo frame rate is represented by f2, for example, the device used for taking photos can take f2 images per second. Common photo frame rates are 4fps, 2fps, etc. For example, if the photo frame rate is 4fps, it means that the device can take four images per second.

[0038] The shooting frequency can be determined based on the frame rate and the number of original image frames required to synthesize a high-definition image. For example, if six original image frames are required to synthesize a high-definition image and the frame rate is 30fps, then the shooting frame rate can be 30 / 6=5, that is, five images can be taken in one second.

[0039] 3) Sensor

[0040] A sensor is an image acquisition device in mobile phones, tablets, cameras and other devices. It can also be called a sensor. It can convert optical signals into digital signals or other required forms for easy processing and storage.

[0041] 4) Image framework

[0042] The imaging framework is used to process the photo-taking process and can be applied to devices with photo-taking functions. The photo-taking process can be divided into two stages: frame acquisition and post-processing. In the frame acquisition stage, the imaging framework can obtain all the original image frames required for this photo-taking from the sensor. In the post-processing stage, the imaging framework can process the acquired original image frames using a multi-frame synthesis algorithm to generate the image corresponding to this photo-taking.

[0043] To further illustrate this specification, the following examples are provided:

[0044] See also Figure 2 , Figure 2 This is a flowchart of an image generation method according to an exemplary embodiment of the present specification, which can be applied to devices such as mobile phones and cameras with a camera function, and may include the following steps:

[0045] Step 202: Obtain N original image frames that are continuous in time, where N is the number of original image frames required to synthesize a high-definition image; at least some of the N original image frames are multiplexed image frames, and the multiplexed image frames are original image frames used in the previous high-definition image generation adjacent to the current high-definition image generation.

[0046] In this step, the image framework can obtain N raw image frames that are continuous in time. The raw image frame is the original data that the sensor converts the captured light signal into a digital signal, which can also be called a raw image, and is used to synthesize the final high-definition image. N is the number of raw image frames required to synthesize the high-definition image, which is a natural number greater than 1 and is set by different multi-frame synthesis algorithms and devices.

[0047] It should be noted that the number of raw images required for synthesizing high-definition images in the embodiments of this specification does not decrease compared to the number of raw images used for synthesizing high-definition images in the prior art. For example, in the related technologies mentioned above, some manufacturers reduce the number of raw images used for synthesizing images in order to reduce the time interval between photographing, thereby reducing the frame receiving time of the sensor. However, the image generation method of the embodiments of this specification does not reduce the number of raw images required for synthesizing high-definition images, but adopts other methods to reduce the frame receiving time of the sensor. It is precisely because the image generation method of this embodiment ensures the number of raw images required for synthesizing high-definition images, therefore, the quality of the high-definition images synthesized by this method will not be affected.

[0048] At least some of the N original image frames are multiplexed image frames. The N original image frames may include multiplexed image frames and new original image frames. The sum of the number of multiplexed image frames and new original image frames in the N original image frames obtained is N. For example, the N original image frames may include: at least one multiplexed image frame, and at most N-1 new original image frames. The multiplexed image frames are the original image frames used in the previous high-definition image generation adjacent to the current high-definition image generation, and the new original image frames are the original image frames newly generated by the sensor in the current high-definition image generation. The time it takes for the sensor to generate all the new original image frames required for this image generation is the frame receiving time of this photo shooting.

[0049] To further explain the original image frame and the multiplexed image frame, Figure 3 A schematic diagram of obtaining N original image frames is shown. Figure 3 , assuming that the number N of original image frames required for synthesizing a high-definition image is 6, when taking the n+1th photograph, the image framework obtains six original image frames that are continuous in time. For example, the 7th and 8th new original image frames can be obtained from the sensor, and the 3rd to 6th multiplexed image frames that are continuous in time with the 7th and 8th frames can be obtained. The 3rd to 6th multiplexed image frames are the original image frames used when the image was generated in the last nth photograph adjacent to the n+1th photograph. When taking the n+2th photograph, the image framework can obtain the 9th and 10th new original image frames from the sensor, and obtain the 5th to 8th multiplexed image frames used when the image was generated in the n+1th photograph.

[0050] exist Figure 3 In the method of obtaining N original image frames shown, the time point of the n+1th photo is t, and the frame is completed at the time point t+2 / fps. The frame collection time of the n+1th photo = (the number of new original image frames / frame rate). Two new original image frames are newly generated. By obtaining four multiplexed image frames, there is no need to generate four new original image frames, and the frame collection time can be reduced from 6 / fps to 2 / fps. Assuming the frame rate is 30fps, the frame collection time of the n+1th photo is reduced from (6 / 30) = 0.2s to (2 / 30) ≈ 0.067 seconds. Photo frame rate = (1s / frame collection time), which increases the photo frame rate from a maximum of 5 consecutive photos per second to a maximum of 15 consecutive photos per second.

[0051] In another embodiment, the N original image frames may include: one new original image frame, and N-1 multiplexed image frames. Figure 4 Another schematic diagram of obtaining N original image frames is shown, see Figure 4, assuming that N=6, when taking the k+1th photo, the image framework can obtain six original image frames that are continuous in time, can obtain the 7th new original image frame from the sensor, and obtain the 2nd to 6th multiplexed image frames that are continuous in time with the 7th frame. The 2nd to 6th multiplexed image frames are the original image frames used when the previous kth photo was generated adjacent to the k+1th photo. When taking the k+2th photo, the image framework can obtain the 8th new original image frame from the sensor, and obtain the 3rd to 7th multiplexed image frames used when the k+1th photo was generated.

[0052] exist Figure 4 In the method of obtaining N original image frames shown, the time point of the k+1th photo is t, and the frame is completed at the time point t+1 / fps. The frame collection time of the k+1th photo = (the number of new original image frames / frame rate). A new original image frame is generated by obtaining five multiplexed image frames, and there is no need to generate five new original image frames. The frame collection time can be reduced from 6 / fps to 1 / fps. Assuming the frame rate is 30fps, the frame collection time of the k+1th photo is reduced from (6 / 30) = 0.2s to (1 / 30) ≈ 0.033 seconds. The photo frame rate = (1s / frame collection time), and the maximum number of consecutive photos that can be taken in one second is increased from 5 to 30.

[0053] As described above, by including a part of the multiplexed image frames in the N original image frames required for synthesizing the high-definition image, the number of new original image frames generated by the sensor is less than N, thereby shortening the frame collection time of the sensor and reducing the time interval between two shots. Moreover, as described above, the number of raw images required for synthesizing the high-definition image is not reduced, thereby reducing the time interval between shots without affecting the quality of the generated high-definition image. Furthermore, when the number of multiplexed image frames is N-1, the number of new original image frames generated by the sensor required to be used is 1, so that the sensor can start the next shot as soon as it produces one frame, greatly shortening the frame collection time for shooting and reducing the time interval between two shots.

[0054] Step 204: synthesizing the N original image frames to generate a final high-definition image.

[0055] In this step, the image framework can synthesize the N temporally continuous original image frames acquired in step 202 through a multi-frame synthesis algorithm in the image framework to generate a final high-definition image.

[0056] The multi-frame synthesis algorithm can perform alignment, motion estimation and fusion strategies on multiple original image frames to obtain a higher quality output image. It can be a multi-frame noise reduction algorithm, a multi-frame fusion algorithm and other algorithms, and the embodiments of this specification do not limit this.

[0057] As follows Figure 5 An example of a method of synthesizing N original image frames in this step is given. Figure 5 A flow chart of a method for synthesizing N original image frames is shown. The method may include the following steps:

[0058] Step 502: determining a second reference image frame from the N original image frames, where the second reference image frame is an original image frame used as a reference in the current high-definition image generation process.

[0059] In this step, the image framework can select an original image frame used as a reference in the current high-definition image generation process from the N original image frames obtained, as the second reference image frame. The original image frame used as the reference can be an original image frame used as the reference in the previous high-definition image generation in the N original image frames that is later in time sequence. Exemplarily, the original image frame with the highest definition can be selected as the reference. It is understandable that the original image frame that meets other requirements can also be selected as the reference, and this embodiment does not limit it.

[0060] Step 504: Merge other original image frames among the N original image frames into the second benchmark image frame; wherein the acquisition time of the second benchmark image frame is later than the acquisition time of the first benchmark image frame, and the first benchmark image frame is the original image frame used as a benchmark when the last high-definition image was generated.

[0061] In this step, the multi-frame synthesis algorithm in the image framework can merge other original image frames in the N original image frames into the second reference image frame. The acquisition time of the second reference image frame needs to be later than the acquisition time of the first reference image frame in the time sequence. The first reference image frame is the original image frame used as a reference when the last high-definition image was generated.

[0062] Please continue reading Figure 3 , when the high-definition image is generated by the n+1th photograph, the N original image frames used are the 3rd to 8th original image frames. The first reference image frame is the original image frame used as the reference when the high-definition image corresponding to the nth photograph is generated. For example, when the high-definition image corresponding to the nth photograph is generated, the original image frame used as the reference is the 5th original image frame, then the 5th frame is the key frame, which can also be called the anchor frame, and the first reference image frame is the 5th original image frame.

[0063] When determining the second reference image frame among the original image frames from the 3rd to the 8th frames, since the acquisition time of the second reference image frame needs to be later than the acquisition time of the first reference image frame in the time sequence, and the acquisition time of the second reference image frame needs to be later than the acquisition time of the 5th original image frame, it is necessary to select the second reference image frame among the original image frames from the 6th to the 8th frames, and the original image frame with the highest definition among the 6th to the 8th frames can be selected as the second reference image frame. If the original image frame with the highest definition among the 6th to the 8th frames is the 8th original image frame, the 8th frame is the key frame, and the second reference image frame is the 8th original image frame. The original image frames from the 3rd to the 7th frames among the 3rd to the 8th frames can be merged into the 8th original image frame to generate the final high-definition image.

[0064] As described above, by determining a second reference image frame whose acquisition time is later than that of the first reference image frame, and merging the other original image frames in the N original image frames into the second reference image frame, the accuracy of the time sequence of the last high-definition image generated based on the first reference image frame and the high-definition image generated this time based on the second reference image frame can be ensured, thereby avoiding the phenomenon of inverted content in the two generated high-definition images.

[0065] It is understandable that the image generation method in this embodiment can be applied to high-frequency continuous photography scenarios. High-frequency continuous photography is a photography method that uses a very fast speed to capture images, and several to dozens of images can be captured in one second. In high-frequency continuous photography scenarios, using one new original image frame and N-1 multiplexed image frames can also generate a high-definition image as well as using N new original image frames.

[0066] The image generation method of the present embodiment generates a high-definition image by acquiring N original image frames that are continuous in time. The number of original image frames required for synthesizing the high-definition image is not reduced, and the image quality requirements of the generated high-definition image can be guaranteed. At the same time, since the original image frames used in the previous high-definition image generation are reused, the generation time of the original image frames can be reduced, and the frame collection time of each photo can be shortened, thereby reducing the interval between photo shooting times without affecting the image quality of the generated high-definition image.

[0067] In the above embodiment, the acquisition method of the new original image frame is described, that is, the image framework can acquire the new original image frame from the sensor. In the following embodiment, how to manage the new original image frame and the multiplexed image frame will be described in more detail, and can be applied to any of the above embodiments.

[0068] The image framework can obtain the new original image frame newly generated by the sensor in this high-definition image generation, and store the new original image frame in the original image frame set in the storage module. The storage module is used to store the original image frame, which can be a cache, memory and other devices. The original image frame set is a set of original image frames stored in the storage module.

[0069] The image framework can obtain the original image frames required for synthesizing the high-definition image in the next high-definition image generation from the original image frame set of the storage module during the next high-definition image generation process adjacent to the current high-definition image generation.

[0070] Please continue reading Figure 3 , taking the n+1th photo as an example, the storage module of the device takes the cache as an example, and the original image frame set in the cache includes: the 3rd to 6th original image frames. In the high-definition image generation process corresponding to the n+1th photo, the new original image frames generated by the sensor are the 7th and 8th new original image frames. The image framework can store the acquired 7th and 8th new original image frames in the original image frame set.

[0071] The next photo taken adjacent to the n+1th photo taken is the n+2th photo taken. The image framework can obtain the 5th to 8th original image frames from the cached original image frame set during the high-definition image generation process corresponding to the n+2th photo taken, and obtain the 9th and 10th original image frames newly generated by the sensor, and synthesize the obtained 5th to 10th original image frames to generate the final high-definition image.

[0072] As described above, by storing the newly generated new original image frame into the original image frame set of the storage module, the original image frame required for synthesizing the high-definition image is obtained from the original image frame set, so that the new original image frame and the multiplexed image frame can be uniformly stored and managed, thereby shortening the search time of the original image frame and speeding up the generation speed of the high-definition image.

[0073] Furthermore, the image framework can also delete invalid original image frames. The image framework can check whether the original image frames in the original image frame set meet the image frame deletion condition and delete the invalid image frames in the original image frame set. The image frame deletion condition is used to determine whether the original image frames in the original image frame set are invalid image frames. Invalid image frames are original image frames that will no longer be used in the high-definition image generation process. The original image frames are used at most N times and will no longer be reused to generate high-definition pictures.

[0074] The image frame deletion condition may be, for example, to determine whether the number of references of the original image frame reaches a preset number threshold, where the number of references is the number of times the original image frame is used for synthesis processing; or, it may be to determine whether the storage space occupancy rate of the original image frame set in the storage module reaches the standard occupancy rate. Exemplarily, the standard occupancy rate of the original image frame set is set to 80%. If the storage space occupancy rate of the original image frame set in the storage module is greater than 80%, the original image frame stored earliest in the storage module is determined to be an invalid image frame, and the invalid image frame is deleted until the storage space occupancy rate exceeds 80%.

[0075] As described above, by judging whether the original image frame meets the image frame deletion condition and deleting invalid image frames that will no longer be used, invalid resources can be prevented from occupying storage space and the storage pressure of the storage module can be reduced.

[0076] In the above-mentioned embodiment, it is introduced that the image frame deletion condition can be to determine whether the number of references of the original image frame reaches a preset number threshold. In the following embodiment, it will be further explained how to determine whether the image frame deletion condition is met based on the number of references of the original image frame.

[0077] In the case where N original image frames include one new original image frame and N-1 multiplexed image frames, the image framework can accumulate the number of references to the original image frames, that is, when an original image frame is used for synthesis processing, the accumulated number of references to the original image frame is increased by one, or decreased by one. The image frame deletion condition is whether the number of references to the original image frame reaches a preset number threshold. If the number of references to the original image frame reaches the preset number threshold, the original image frame is determined to be an invalid image frame and the invalid image frame is deleted.

[0078] Exemplarily, the number threshold may be preset to N, the initial reference number of the original image frame is set to 0, and each time the original image frame is used, the reference number of the original image frame is increased by 1. When the reference number of the original image frame is equal to N, the original image frame has been synthesized N times, and the original image frame will no longer be used in the subsequent image generation process, and the original image frame may be determined as an invalid image frame and deleted.

[0079] The number threshold can also be preset to 0, the initial reference number of the original image frame is set to N, and each time the original image frame is used, the reference number of the original image frame is reduced by 1. When the reference number of the original image frame is equal to 0, the original image frame has been synthesized N times, and the original image frame will no longer be used in the subsequent image generation process. The original image frame can be determined as an invalid image frame and deleted.

[0080] In a scenario where N original image frames include one new original image frame and N-1 multiplexed image frames, the relationship between the original image frames used for taking a photo and generating a high-definition image from the photo can be shown in Table 1:

[0081] Table 1 Relationship between the original image frames used for taking photos and generating high-definition images

[0082]

[0083]

[0084] Among them, Table 1 is based on Figure 4 The schematic diagram of obtaining N original image frames shown in the figure exemplarily shows the relationship between the original image frames used for partial photography and photography to generate high-definition images.

[0085] For example, in the high-definition image generation process corresponding to the k-2th photo in Table 1, it is assumed that the number of original image frames N required to generate the high-definition image is 6, and the 6 original image frames obtained include: original image frames a, b, c, d, e and f. Among them, original image frame f is a new original image frame, and original image frames a, b, c, d and e are multiplexed image frames. Taking the preset number threshold of 6 as an example, the initial reference count of original image frame f is 0. In the high-definition image generation process corresponding to the k-2th photo, original image frame f is used once, and the reference count of original image frame f is increased by one. At this time, the reference count of original image frame f is 1.

[0086] In the high-definition image generation process corresponding to the k-1th shot in Table 1, the 6 original image frames obtained include: original image frames b, c, d, e, f and g. In the high-definition image generation process corresponding to the k-1th shot, the original image frame f is used once, and the number of references of the original image frame f is increased by one again. At this time, the number of references of the original image frame f is 2.

[0087] By analogy, in the high-definition image generation process corresponding to the k-th shot, the k+1-th shot, the k+2-th shot, and the k+3-th shot in Table 1, the original image frame f has been used 4 times. At this time, the original image frame f has been used 6 times, and the number of references of the original image frame f has accumulated to 6 times. The number of references of the original image frame f has reached the preset number threshold, and the image frame deletion condition is met.

[0088] In the high-definition image generation process corresponding to the k+4th photo in Table 1, the 6 original image frames obtained include: original image frames g, h, i, j, k and l. The subsequent high-definition image generation process no longer uses the original image frame f for synthesis processing. The original image frame f can be determined as an invalid image frame and deleted.

[0089] As described above, by accumulating the number of references to the original image frame, determining whether the original image frame is an invalid image frame based on the number of references, and deleting the invalid image frame, the original image frames that are no longer used in the subsequent high-definition image generation process can be cleared in time, so that only the original image frames that will be reused are stored in the original image frame set, further avoiding invalid resources from occupying storage space and reducing the space occupancy rate of the original image frame set in the storage module.

[0090] In another embodiment, the image framework may also control the sensor to start the image capture function in response to determining that the new original image frame required for the current high-definition image generation has been acquired, so as to capture the new original image frame required for the next high-definition image generation.

[0091] Please continue reading Figure 3 , taking the high-definition image generation process corresponding to the n+1th photo as an example, the new original image frames required for this high-definition image generation are the 7th and 8th original image frames. The image framework can obtain the 3rd to 5th multiplexed image frames before obtaining the newly generated 7th and 8th original image frames, or while obtaining the newly generated 7th and 8th original image frames. When the image framework determines that the 7th and 8th original image frames have been obtained, the sensor is controlled to start the image capture function, allowing the capture of the new original image frames required for the high-definition image generation corresponding to the n+2th photo, that is, allowing the capture of the 9th and 10th original image frames.

[0092] As described above, by starting the image capture function after determining that the new original image frame required for the generation of this high-definition image has been acquired, the frame collection time of each photo can be kept consistent, avoiding the problem of a large difference in the shooting time and exposure time of multiple photos; based on the new original image frame, it is determined whether to allow the next high-definition image to be taken, and there is no need to wait until the multiplexed image frame is acquired to start the image capture function, which can further shorten the interval between photos.

[0093] Furthermore, when any of the following conditions is met, the image framework can determine that the new original image frame required for the current high-definition image generation has been acquired:

[0094] The number of new raw image frames acquired by the image frame from the sensor reaches a preset image threshold; the image threshold is set according to the number of new raw image frames required for this high-definition image generation.

[0095] Alternatively, the timing starts from receiving the first new original image frame of the current high-definition image generation process, and the timing duration reaches the preset frame receiving time; the preset frame receiving time is determined according to the number of new original image frames and the frame rate required for the current high-definition image generation. Exemplarily, the preset frame receiving time can be calculated by preset frame receiving time = (number of new original image frames / frame rate).

[0096] Please continue reading Figure 3 , taking the high-definition image generation process corresponding to the n+1th shooting as an example, the new original image frames required for this high-definition image generation are the 7th and 8th original image frames, and the number of new original image frames obtained is 2, so the preset image threshold can be set to 2. When the number of new original image frames obtained from the sensor is greater than or equal to 2, it can be determined that the new original image frames required for this high-definition image generation have been obtained.

[0097] Correspondingly, in the high-definition image generation process corresponding to the n+1th photo, the preset frame collection time can be determined according to the number of new original image frames and the frame rate. The number of new original image frames generated by the n+1th photo is 2. Assuming the frame rate is 30fps, the preset frame collection time = (number of new original image frames / frame rate), that is, the preset frame collection time = (2 / 30), and the preset frame collection time is 0.067 seconds. The timing starts from receiving the 7th original image frame of this high-definition image generation process. When the timing reaches the preset frame collection time, it can be determined that the new original image frames required for this high-definition image generation have been acquired.

[0098] When the number of new original image frames reaches two times, or the timing duration reaches any of the preset frame receiving time, it can be determined that the new original image frames required for this high-definition image generation have been acquired, and the image shooting function is started, allowing the new original image frames required for the high-definition image generation corresponding to the n+2th photo to be shot.

[0099] As described above, by using the number of new original image frames acquired as a condition for whether the new original image frames have been acquired, the influence of the delay error of the timing duration can be avoided; and by using the timing duration for receiving the new original image frames as another condition for whether the new original image frames have been acquired, the consistency of the frame receiving time for each photo shooting is further ensured.

[0100] Figure 6 1 is a schematic diagram of an electronic device according to an exemplary embodiment of the present specification. The electronic device may be, for example, a mobile phone, a computer, a digital broadcast terminal, a message transceiver device, a game console, a tablet device, a medical device, a fitness device, a personal digital assistant, etc. Figure 6At the hardware level, the electronic device includes a processor 602, an internal bus 604, a network interface 606, a memory 608, and a non-volatile memory 610, and may also include hardware required for other services. The processor 602 reads the corresponding computer program from the non-volatile memory 610 into the memory 608 and then runs it, forming an image generation device at the logical level. Of course, in addition to the software implementation, this application does not exclude other implementation methods, such as logic devices or a combination of software and hardware, etc., that is to say, the execution subject of the following processing flow is not limited to each logic unit, but can also be hardware or logic devices.

[0101] Figure 7 is a block diagram of an image generating device according to an exemplary embodiment of the present specification. Figure 7 The device comprises: an original image frame acquisition module 702 and a high-definition image generation module 704, wherein:

[0102] The original image frame acquisition module 702 is used to acquire N original image frames that are continuous in time, where N is the number of original image frames required to synthesize a high-definition image; at least some of the N original image frames are multiplexed image frames, and the multiplexed image frames are the original image frames used in the previous high-definition image generation adjacent to the current high-definition image generation.

[0103] The high-definition image generation module 704 is used to perform synthesis processing on the N original image frames to generate a final high-definition image.

[0104] In one example, the N original image frames include: a new original image frame, which is an original image frame newly generated by the sensor in this high-definition image generation; and N-1 multiplexed image frames.

[0105] In one example, the N original image frames include: a new original image frame newly generated by the sensor in the current high-definition image generation; the original image frame acquisition module 702 is also used to store the new original image frame in the original image frame set in the storage module; in the next high-definition image generation process adjacent to the current high-definition image generation, the original image frame required for synthesizing the high-definition image in the next high-definition image generation is obtained from the original image frame set in the storage module.

[0106] In one example, the original image frame acquisition module 702 is further used to delete invalid image frames in the original image frame set in response to satisfying an image frame deletion condition; the invalid image frames are original image frames that are no longer used for high-definition image generation.

[0107] In one example, when the N original image frames include one new original image frame and N-1 multiplexed image frames, the original image frame acquisition module 702, when used to delete invalid image frames in the original image frame set in response to satisfying an image frame deletion condition, includes: accumulating the number of references of the original image frame, where the number of references is the number of times the original image frame is used for synthesis processing; if the number of references of the original image frame reaches a preset number threshold, determining the original image frame as an invalid image frame, and deleting the invalid image frame.

[0108] In one example, the high-definition image generation module 704, when used for the synthesis processing of the N original image frames, includes: determining a second reference image frame from the N original image frames, the second reference image frame is the original image frame used as a reference in the current high-definition image generation process; merging other original image frames in the N original image frames into the second reference image frame; wherein the acquisition time of the second reference image frame is later than the acquisition time of the first reference image frame, and the first reference image frame is the original image frame used as a reference during the last high-definition image generation.

[0109] In one example, the N original image frames include: the multiplexed image frames, and the new original image frames newly generated by the sensor in this high-definition image generation; the original image frame acquisition module 702 is also used to control the sensor to start the image capture function in response to determining that the new original image frames required for this high-definition image generation have been acquired, so as to capture the new original image frames required for the next high-definition image generation.

[0110] In one example, the original image frame acquisition module 702, when used in response to determining that the new original image frame required for the current high-definition image generation has been acquired, includes: if any of the following conditions is met, it is determined that the new original image frame required for the current high-definition image generation has been acquired: the number of new original image frames acquired from the sensor reaches a preset image threshold; or, the timing starts from receiving the first new original image frame of the current high-definition image generation process, and the timing duration reaches a preset frame receiving time; the preset frame receiving time is determined according to the number of new original image frames required for the current high-definition image generation and the frame rate.

[0111] The implementation process of the functions and effects of each unit in the above-mentioned device is specifically described in the implementation process of the corresponding steps in the above-mentioned method, and will not be repeated here.

[0112] For the device embodiment, since it basically corresponds to the method embodiment, the relevant parts can refer to the partial description of the method embodiment. The device embodiment described above is only schematic, wherein the modules described as separate components may or may not be physically separated, and the components displayed as modules may or may not be physical modules, that is, they may be located in one place, or they may be distributed on multiple network modules. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this specification. A person of ordinary skill in the art can understand and implement it without paying creative labor.

[0113] In an exemplary embodiment, a non-transitory computer-readable storage medium including instructions is also provided, such as a memory including instructions, and the above instructions can be executed by a processor of a resource access device to implement any of the methods described in the above embodiments.

[0114] The non-temporary computer-readable storage medium may be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, etc., and the present application does not limit this.

[0115] The above is a description of a specific embodiment of the specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in an order different from that in the embodiments and still achieve the desired results. In addition, the processes depicted in the drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0116] Those skilled in the art will readily appreciate other embodiments of the specification after considering the specification and practicing the invention claimed herein. The specification is intended to cover any variations, uses or adaptations of the specification that follow the general principles of the specification and include common knowledge or customary techniques in the art that are not claimed in the specification. The specification and examples are to be considered exemplary only, and the true scope and spirit of the specification are indicated by the following claims.

[0117] It should be understood that the present description is not limited to the precise structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present description is limited only by the appended claims.

[0118] The above description is only a preferred embodiment of this specification and is not intended to limit this specification. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of this specification should be included in the scope of protection of this specification.

Claims

1. An image generation method, characterized in that: The method comprises: Acquire N original image frames that are continuous in time, where N is the number of original image frames required to synthesize a high-definition image; at least some of the N original image frames are multiplexed image frames, and the multiplexed image frames are original image frames used in the previous high-definition image generation adjacent to the current high-definition image generation; The N original image frames are synthesized to generate a final high-definition image.

2. The method according to claim 1, characterized in that The N original image frames include: A new original image frame, wherein the new original image frame is an original image frame newly generated by the sensor in this high-definition image generation; And, N-1 of the multiplexed image frames.

3. The method according to claim 1, characterized in that The N original image frames include: new original image frames newly generated by the sensor in the current high-definition image generation; The method further comprises: Storing the new original image frame in the original image frame set in the storage module; In the next high-definition image generation process adjacent to the current high-definition image generation, the original image frames required for synthesizing the high-definition image in the next high-definition image generation are obtained from the original image frame set in the storage module.

4. The method according to claim 3, characterized in that The method further comprises: In response to satisfying an image frame deletion condition, deleting invalid image frames in the original image frame set; The invalid image frames are original image frames that are no longer used for high-definition image generation.

5. The method according to claim 4, characterized in that In a case where the N original image frames include one new original image frame and N-1 multiplexed image frames, in response to satisfying an image frame deletion condition, deleting invalid image frames in the original image frame set includes: Accumulating the number of times the original image frame is referenced, where the number of times the original image frame is used for synthesis processing; If the number of references of the original image frame reaches a preset number threshold, the original image frame is determined as an invalid image frame and the invalid image frame is deleted.

6. The method according to claim 1, characterized in that The synthesizing process of the N original image frames includes: Determining a second reference image frame from the N original image frames, where the second reference image frame is an original image frame used as a reference in the current high-definition image generation process; Merging other original image frames among the N original image frames into the second reference image frame; The acquisition time of the second reference image frame is later than the acquisition time of the first reference image frame, and the first reference image frame is the original image frame used as a reference when the last high-definition image was generated.

7. The method according to claim 1, characterized in that The N original image frames include: the multiplexed image frame, and a new original image frame newly generated by the sensor in this high-definition image generation; The method further comprises: In response to determining that the new original image frame required for the current high-definition image generation has been acquired, the sensor is controlled to start an image capture function to capture a new original image frame required for the next high-definition image generation.

8. The method according to claim 7, characterized in that In response to determining that the new original image frame required for generating the high-definition image has been acquired, the method includes: If any of the following conditions is met, it is determined that the new original image frame required for the current high-definition image generation has been acquired: The number of new raw image frames acquired from the sensor reaches a preset image threshold; Alternatively, timing starts from receiving the first new original image frame of this high-definition image generation process, and the timing duration reaches a preset frame receiving time; the preset frame receiving time is determined according to the number and frame rate of new original image frames required for this high-definition image generation.

9. An image generating device, characterized in that: The device comprises: An original image frame acquisition module is used to acquire N original image frames that are continuous in time, where N is the number of original image frames required to synthesize a high-definition image; at least some of the N original image frames are multiplexed image frames, and the multiplexed image frames are original image frames used in the previous high-definition image generation adjacent to the current high-definition image generation; The high-definition image generation module is used to perform synthesis processing on the N original image frames to generate a final high-definition image.

10. An electronic device, characterized in that: include: processor; a memory for storing processor-executable instructions; The processor implements the method according to any one of claims 1 to 8 by running the executable instructions.

11. A computer-readable storage medium having computer instructions stored thereon, characterized in that: When the instruction is executed by a processor, the steps of the method according to any one of claims 1 to 8 are implemented.

Images