Video generation method, device, electronic device and storage medium

Abstract

The present disclosure relates to a video generation method, device, electronic device and storage medium, and belongs to the field of computer technology. The method includes: determining multiple offset field groups and representative offset fields corresponding to the multiple offset field groups; offsetting the pixel points in the first image according to the representative offset fields corresponding to the multiple offset field groups to obtain the second images corresponding to the multiple offset field groups; determining the target offset field group from the multiple offset field groups based on the second images corresponding to the multiple offset field groups; offsetting the pixel points in the first image according to at least one offset field in the target offset field group to obtain the target image, and generating a video containing at least one target image. The embodiment of the present disclosure provides a method for generating a dynamic video using an image, generating at least one image using the offset field group that is most suitable for the current image, and generating a video containing the at least one image, so as to ensure the quality of the image used to generate the video, thereby ensuring the quality of the video.

Description

Technical Field

[0001] The present disclosure relates to the field of computer technology, and in particular to a video generation method, device, electronic device and storage medium. Background Art

[0002] With the popularization of multimedia technology, generating videos based on images has become a common way of video generation and is popular among people.

[0003] In the related art, a fixed offset field is used to move each pixel in the image to generate a new image as a video frame, and then a video containing the video frame is generated. However, the fixed offset field is not suitable for all images, which will cause a large number of pseudo textures to appear in the generated video, resulting in poor video quality. Summary of the invention

[0004] The present disclosure provides a video generation method, device, electronic device and storage medium, which improve the video quality.

[0005] According to one aspect of an embodiment of the present disclosure, a video generation method is provided, the method comprising:

[0006] Determine a plurality of offset field groups and representative offset fields corresponding to the plurality of offset field groups, wherein the offset field group includes at least one offset field, and the representative offset field is used to indicate the offset magnitude contained in the offset field in the offset field group;

[0007] According to the representative offset fields corresponding to the plurality of offset field groups, pixel points in the first image are offset respectively to obtain second images corresponding to the plurality of offset field groups;

[0008] determining a target offset field group from the plurality of offset field groups based on the second images corresponding to the plurality of offset field groups;

[0009] According to at least one offset field in the target offset field group, pixel points in the first image are offset respectively to obtain a target image, and a video containing at least one of the target images is generated.

[0010] The disclosed embodiments provide a method for generating dynamic videos using images, wherein each offset group represents a motion trajectory, and representative offset fields of a plurality of offset field groups are used to select an offset field group for generating a video, so as to ensure that the offset field group most suitable for the current image can be selected, that is, a motion trajectory suitable for generating a video from the current image is screened out from a plurality of motion trajectories, and at least one image is generated using the selected offset field group, and then a video containing the at least one image is generated, so that the video can simulate the effect of the content in the image being offset along the screened motion trajectory, so as to ensure the quality of the image used to generate the video, avoid the situation where a large number of pseudo-textures appear in the image in the video due to an inappropriate selection of the offset field group, and then ensure the quality of the video.

[0011] In some embodiments, determining a target offset field group from a plurality of offset field groups based on the second images corresponding to the plurality of offset field groups comprises:

[0012] The target offset field group is determined from the plurality of offset field groups based on black pixel ratios of the second image corresponding to the plurality of offset field groups.

[0013] In the embodiment of the present disclosure, black pixels are generated in the process of offsetting pixel points in the first image according to the representative offset field. The black pixel ratio can reflect the quality of the second image obtained by the offset. Based on the black pixel ratio of the second image corresponding to multiple offset field groups, a target offset field group suitable for the first image is determined from the multiple offset fields, thereby ensuring the quality of the image subsequently generated according to the target offset field group.

[0014] In some embodiments, determining the target offset field group from the plurality of offset field groups based on the black pixel ratios of the second image corresponding to the plurality of offset field groups comprises:

[0015] The offset field group corresponding to the second image with the smallest black pixel ratio among the plurality of second images is determined as the target offset field group.

[0016] Among the multiple second images, the second image with the smallest black pixel ratio has the highest quality, which means that the quality of the image subsequently generated according to the offset field group corresponding to the second image is high. The offset field group corresponding to the second image with the smallest black pixel ratio among the multiple offset field groups is most suitable for the first image. Therefore, the offset field group is determined as the target offset field group to ensure the quality of the subsequently generated video.

[0017] In some embodiments, determining the target offset field group from the plurality of offset field groups based on the black pixel ratios of the second image corresponding to the plurality of offset field groups comprises:

[0018] When the number of the second images whose black pixel ratios are less than the preset ratio is 1, determining the offset field group corresponding to the second images whose black pixel ratios are less than the preset ratio as the target offset field group;

[0019] When there are multiple second images whose black pixel ratio is less than the preset ratio, the offset field group corresponding to the second image whose black pixel ratio is less than the preset ratio and represents the largest offset field is determined as the target offset field group.

[0020] In the embodiment of the present disclosure, a preset ratio is set based on the quality requirement for the image. Since among the multiple second images, the number of the second images whose black pixel ratio is less than the preset ratio may be one or more, when the number of the second images whose black pixel ratio is less than the preset ratio is 1, it means that only one second image meets the quality requirement, and the offset field group corresponding to the second image whose black pixel ratio is less than the preset ratio is determined as the target offset field group; and when the number of the second images whose black pixel ratio is less than the preset ratio is multiple, the offset field group corresponding to the second image representing the largest offset field is preferentially selected as the target offset field group, so as to ensure that the degree of content offset in the multiple images subsequently generated based on the target offset field is as large as possible, thereby ensuring the motion effect of the subsequently generated video.

[0021] In some embodiments, the offset field includes a lateral offset and a longitudinal offset; determining a representative offset field corresponding to a plurality of the offset field groups includes:

[0022] The maximum lateral offset and the maximum longitudinal offset included in the offset field group constitute a representative offset field corresponding to the offset field group; or,

[0023] Determine the offset field including the maximum lateral offset in the offset field group as the representative offset field corresponding to the offset field group; or,

[0024] Determine the offset field including the maximum longitudinal offset in the offset field group as the representative offset field corresponding to the offset field group; or,

[0025] The sum of the lateral offset and the longitudinal offset included in each offset field in the offset field group is determined, and the offset field corresponding to the maximum sum in the offset field group is determined as the representative offset field corresponding to the offset field group.

[0026] The embodiments of the present disclosure provide a variety of ways to determine a representative offset corresponding to an offset field group. Since the maximum lateral offset and the maximum longitudinal offset respectively represent the maximum offset amplitudes in the two offset directions, the sum of the lateral offset and the longitudinal offset in the offset field can also reflect the offset amplitude of the offset field. Therefore, it is preferred to select an offset field containing the maximum lateral offset or the maximum longitudinal offset from the offset field group, or to select an offset field corresponding to the maximum sum value in the offset field group, and determine it as the representative offset field corresponding to the offset field group, so as to ensure that the determined representative offset field can best represent the corresponding offset field group, thereby ensuring the accuracy of the determined representative offset field.

[0027] In some embodiments, after generating a video including at least one of the target images, the method further includes:

[0028] A motion blur process is performed on the image contained in the video.

[0029] In the disclosed embodiment, after the video is generated, the images contained in the video can also be subjected to motion blur processing to enhance the motion effect of the video and further improve the quality of the video.

[0030] In some embodiments, the video includes T images, where T is an integer greater than 1; and performing motion blur processing on the images included in the video includes:

[0031] Obtain S motion blur kernels corresponding to the target offset field group, the sizes of the S motion blur kernels decrease in sequence, and S is a positive integer less than T;

[0032] Based on the S motion blur kernels, motion blur processing is performed on the first S images in the video.

[0033] In the embodiment of the present disclosure, each motion blur kernel is used to perform motion blur processing on an image. Then, based on S motion blur kernels, motion blur processing is performed on the first S images in the video. Except for the first S images in the video, other images remain unchanged, and the sizes of the S motion blur kernels decrease successively. Then, when the video after motion blur processing is played, it can reflect the effect that the content in the image gradually becomes clear due to movement, simulating the situation in which the content in the video is blurred due to the movement of the camera during real shooting, thereby improving the motion effect of the video.

[0034] In some embodiments, the target offset field group includes T offset fields, and before performing motion blur processing on the first S images in the video based on the S motion blur kernels, the method further includes:

[0035] Based on the first S+1 offset fields in the target offset field group, S angles are determined, where the S angles are angles between vectors corresponding to every two adjacent offset fields in the first S+1 offset fields;

[0036] Based on the S angles, the S motion blur kernels are updated.

[0037] In the solution provided by the embodiment of the present disclosure, the first S offset fields in the offset field group are used to determine S angles to express the degree of difference between every two adjacent offset fields in the first S offset fields, and the S motion blur kernels are updated using the S angles, so that the updated motion blur kernels can reflect the differences between adjacent images, simulate the motion blur effect caused by motion during real shooting, ensure the subsequent motion blur processing effect, and further ensure the motion effect of the video after motion blur processing.

[0038] In some embodiments, before updating the S motion blur kernels based on the S angles, the method further includes:

[0039] Performing weighted fusion on the zero angle and the first angle to obtain the updated first angle; performing weighted fusion on the updated j-1th angle and the jth angle to obtain the updated jth angle, where j is an integer greater than 1 and not greater than S; or,

[0040] The first angle and the second angle are weightedly fused to obtain the updated second angle; the updated k-1th angle and the kth angle are weightedly fused to obtain the updated kth angle, where k is an integer greater than 2 and not greater than S.

[0041] After determining the S angles, the S angles are updated using a time-series smoothing method to avoid excessive differences between the S angles, so that the updated S angles tend to be smooth, to ensure that the motion blur kernel after subsequent updates tends to be smooth, and further to ensure that the image in the video after motion blur processing tends to be smooth, thereby reducing the frequency of image flickering in the video and improving the motion effect of the video.

[0042] In some embodiments, the performing motion blur processing on the first S images in the video based on the S motion blur kernels includes:

[0043] For the i-th image, based on the i-th motion blur kernel, the pixel value of the pixel in the image and the pixel value of the neighborhood pixel of the pixel, the pixel in the image is updated to obtain an updated image, where i is a positive integer not greater than S.

[0044] In the embodiment of the present disclosure, for any pixel in the ith image, the pixel value of the pixel is updated based on the ith motion blur kernel, the pixel value of the pixel and the neighborhood pixels of the pixel, so that the pixel values ​​of the neighborhood pixels are integrated into the pixel, so that the pixel value of the updated pixel is close to the pixel value of the neighborhood pixels. In the above manner, after the pixel value of each pixel in the ith image is updated, each pixel is close to the pixel value of the neighborhood pixels, thereby achieving the effect of image blur.

[0045] In some embodiments, the step of offsetting the pixels in the first image according to the representative offset fields corresponding to the plurality of offset field groups to obtain the second images corresponding to the plurality of offset field groups includes:

[0046] offsetting the point cloud data corresponding to the first image respectively according to the representative offset fields corresponding to the plurality of offset field groups, and generating the second images corresponding to the plurality of offset field groups based on the point cloud data obtained after the offset;

[0047] The step of offsetting pixel points in the first image according to at least one offset field in the target offset field group to obtain a target image includes:

[0048] According to at least one offset field in the target offset field group, the point cloud data corresponding to the first image are offset respectively, and the target image is generated based on the point cloud data obtained after the offset.

[0049] First, the pixels of the first image are converted into point cloud data, and then the point cloud data is offset, rather than directly offsetting the pixels at the pixel level. In this way, the second image can be generated based on the offset point cloud data, avoiding a large number of black pixels in the second image, improving the quality of the second image, and thus ensuring the quality of the subsequently generated video.

[0050] According to another aspect of the embodiments of the present disclosure, a video generating device is provided, the device comprising:

[0051] a determining unit configured to determine a plurality of offset field groups and representative offset fields corresponding to the plurality of offset field groups, wherein the offset field group includes at least one offset field, and the representative offset field is used to refer to the offset magnitude included in the offset field in the offset field group;

[0052] An offset unit is configured to execute, according to representative offset fields corresponding to the plurality of offset field groups, offset pixels in the first image respectively to obtain second images corresponding to the plurality of offset field groups;

[0053] The determining unit is further configured to determine a target offset field group from the plurality of offset field groups based on the second images corresponding to the plurality of offset field groups;

[0054] The generating unit is configured to execute, according to at least one offset field in the target offset field group, offset the pixel points in the first image respectively to obtain a target image, and generate a video containing at least one of the target images.

[0055] In some embodiments, the determining unit is configured to determine the target offset field group from the plurality of offset field groups based on black pixel ratios of the second image corresponding to the plurality of offset field groups.

[0056] In some embodiments, the determining unit is configured to determine the offset field group corresponding to the second image with the smallest black pixel ratio among the plurality of second images as the target offset field group.

[0057] In some embodiments, the determination unit is configured to, when the number of second images whose black pixel ratio is less than a preset ratio is 1, determine the offset field group corresponding to the second images whose black pixel ratio is less than the preset ratio as the target offset field group; and when the number of second images whose black pixel ratio is less than the preset ratio is multiple, determine the offset field group corresponding to the second image whose black pixel ratio is less than the preset ratio and represents the largest offset field as the target offset field group.

[0058] In some embodiments, the offset field includes a lateral offset and a longitudinal offset; the determination unit is configured to execute the execution of making the maximum lateral offset and the maximum longitudinal offset included in the offset field group constitute a representative offset field corresponding to the offset field group; or, determine the offset field including the maximum lateral offset in the offset field group as the representative offset field corresponding to the offset field group; or, determine the offset field including the maximum longitudinal offset in the offset field group as the representative offset field corresponding to the offset field group; or, determine the sum of the lateral offset and the longitudinal offset included in each offset field in the offset field group, and determine the offset field corresponding to the maximum sum in the offset field group as the representative offset field corresponding to the offset field group.

[0059] In some embodiments, the apparatus further comprises:

[0060] The blur processing unit is configured to perform motion blur processing on the image contained in the video.

[0061] In some embodiments, the video includes T images, where T is an integer greater than 1; the blur processing unit is configured to execute acquisition of S motion blur kernels corresponding to the target offset field group, the sizes of the S motion blur kernels decrease successively, and S is a positive integer less than T; based on the S motion blur kernels, motion blur processing is performed on the first S images in the video.

[0062] In some embodiments, the target offset field group includes T offset fields, and the apparatus further includes:

[0063] The determining unit is further configured to determine S angles based on the first S+1 offset fields in the target offset field group, where the S angles are angles between vectors corresponding to every two adjacent offset fields in the first S+1 offset fields;

[0064] An updating unit is configured to update the S motion blur kernels based on the S angles.

[0065] In some embodiments, the updating unit is further configured to perform weighted fusion of the zero angle and the first angle to obtain the updated first angle; perform weighted fusion of the updated j-1th angle and the jth angle to obtain the updated jth angle, where j is an integer greater than 1 and not greater than S; or,

[0066] The first angle and the second angle are weightedly fused to obtain the updated second angle; the updated k-1th angle and the kth angle are weightedly fused to obtain the updated kth angle, where k is an integer greater than 2 and not greater than S.

[0067] In some embodiments, the blur processing unit is configured to perform, for the i-th image, an update of the pixel points in the image based on the i-th motion blur kernel, the pixel value of the pixel point in the image, and the pixel value of the neighboring pixel points of the pixel point to obtain an updated image, where i is a positive integer not greater than S.

[0068] In some embodiments, the offset unit is configured to perform offsetting on the point cloud data corresponding to the first image according to the representative offset fields corresponding to the plurality of offset field groups, respectively, and generate the second images corresponding to the plurality of offset field groups based on the point cloud data obtained after the offsetting;

[0069] The generating unit is configured to perform offsetting on the point cloud data corresponding to the first image according to at least one offset field in the target offset field group, and generate the target image based on the point cloud data obtained after the offset.

[0070] According to another aspect of the embodiments of the present disclosure, an electronic device is provided, the electronic device comprising:

[0071] processor;

[0072] a memory for storing instructions executable by the processor;

[0073] The processor is configured to execute the instructions to implement the video generation method described above.

[0074] According to another aspect of the embodiments of the present disclosure, a non-transitory computer-readable storage medium is provided. When instructions in the storage medium are executed by a processor of an electronic device, the electronic device can execute the video generating method described in the above aspects.

[0075] According to yet another aspect of an embodiment of the present disclosure, a computer program product is provided. When instructions in the computer program product are executed by a processor of an electronic device, the electronic device is enabled to execute the video generating method described in the above aspects.

[0076] The disclosed embodiments provide a method for generating dynamic videos using images, wherein each offset group represents a motion trajectory, and representative offset fields of a plurality of offset field groups are used to select an offset field group for generating a video, so as to ensure that the offset field group most suitable for the current image can be selected, that is, a motion trajectory suitable for generating a video from the current image is screened out from a plurality of motion trajectories, and at least one image is generated using the selected offset field group, and then a video containing the at least one image is generated, so that the video can simulate the effect of the content in the image being offset along the screened motion trajectory, so as to ensure the quality of the image used to generate the video, avoid the situation where a large number of pseudo-textures appear in the image in the video due to an inappropriate selection of the offset field group, and then ensure the quality of the video.

[0077] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0078] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the principles of the present disclosure.

[0079] Figure 1 It is a schematic diagram showing an implementation environment according to an exemplary embodiment.

[0080] Figure 2 The figure is a flowchart of a video generating method according to an exemplary embodiment.

[0081] Figure 3The figure is a flowchart of another video generating method according to an exemplary embodiment.

[0082] Figure 4 is a schematic diagram of a second image according to an exemplary embodiment.

[0083] Figure 5 The invention is a flowchart showing a method for screening a target offset field group according to an exemplary embodiment.

[0084] Figure 6 The figure is a flowchart of another video generating method according to an exemplary embodiment.

[0085] Figure 7 is a schematic diagram of a motion blurred image according to an exemplary embodiment.

[0086] Figure 8 The figure is a flowchart of another video generating method according to an exemplary embodiment.

[0087] Fig. 9 The figure is a flowchart of another video generating method according to an exemplary embodiment.

[0088] Fig.10 It is a block diagram of a video generating device according to an exemplary embodiment.

[0089] Fig.11 is a block diagram of another video generating device according to an exemplary embodiment.

[0090] Fig.12 It is a block diagram of a terminal according to an exemplary embodiment.

[0091] Fig.13 It is a block diagram of a server according to an exemplary embodiment. DETAILED DESCRIPTION

[0092] In order to enable ordinary persons in the art to better understand the technical solutions of the present disclosure, the technical solutions in the embodiments of the present disclosure will be clearly and completely described below in conjunction with the accompanying drawings.

[0093] It should be noted that the terms "first", "second", etc. in the specification and claims of the present disclosure and the above-mentioned figure descriptions are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present disclosure described herein can be implemented in an order other than those illustrated or described herein. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present disclosure. Instead, they are merely examples of devices and methods consistent with some aspects of the present disclosure as detailed in the appended claims.

[0094] The terms "at least one", "a plurality", "each", and "any" used in the present disclosure include one, two, or more than two, a plurality includes two or more than two, each refers to each of the corresponding plurality, and any refers to any one of the plurality. For example, the plurality of offset fields include three offset fields, and each refers to each of the three offset fields, and any refers to any one of the three offset fields, which can be the first offset field, the second offset field, or the third offset field.

[0095] It should be noted that the information involved in this disclosure (including but not limited to images, point cloud data, videos, etc.) is all authorized by the user or fully authorized by all parties.

[0096] The method provided in the embodiments of the present disclosure is performed by an electronic device. In some embodiments, the electronic device is a terminal or a server. In some embodiments, the terminal is a terminal of various types such as a mobile phone, a tablet computer, a computer, etc. In some embodiments, the server is a server, or a server cluster composed of several servers, or a cloud computing service center.

[0097] In some embodiments, the electronic device is provided as a server. Figure 1 It is a schematic diagram of an implementation environment provided according to an exemplary embodiment. The implementation environment includes: a terminal 101 and a server 102. The terminal 101 is connected to the server 102 through a network and can interact with the server 102 through the network connection.

[0098] The server 102 is used to provide a video generation service for the terminal 101. The terminal 101 sends an image of a video to be generated to the server 102, and the server 102 receives the image, generates a video using the method provided in the embodiment of the present disclosure, and sends the video to the terminal 101, which receives the video and plays it, so that the user can view the video generated from the image.

[0099] In some embodiments, the terminal 101 installs a target application provided by the server 102, and the terminal 101 implements functions such as video generation and video playback through the target application. For example, the target application is a content sharing application, which has the functions of content sharing and video generation. Currently, the content sharing application can also have other functions, such as live broadcast function and shopping function. The terminal 101 sends the image of the video to be generated to the server 102 through the target application, and the server 102 receives the image, generates a video based on the image, and sends the video to the terminal 101. The terminal 101 receives the video through the target application and plays it.

[0100] Figure 2 is a flowchart of a video generation method according to an exemplary embodiment. Figure 2 The method is performed by an electronic device and comprises the following steps:

[0101] In step 201, a plurality of offset field groups and representative offset fields corresponding to the plurality of offset field groups are determined, the offset field group includes at least one offset field, and the representative offset field is used to indicate the offset magnitude contained in the offset field in the offset field group.

[0102] In the embodiment of the present disclosure, each offset field includes an offset direction and an offset amount, and is used to offset the pixel points in the image according to the offset direction and the offset amount, so that the content in the image is offset, thereby generating a new image. If the offset fields included in the same offset field group are different, then the multiple images generated by the offset fields in the same offset field group are all different. Each offset field group is equivalent to a motion trajectory, and each offset field in the offset field group is used to offset the pixel points in the same image to different degrees. For any original image, after the pixel points in the original image are offset to different degrees by the offset fields in the offset field group, the trajectory formed by the position obtained by the offset of the same pixel point in the original image is the motion trajectory. After the image is offset by the offset field group, when the offset images are combined into a video, the effect of the content in the image gradually offset can be simulated, reflecting the effect that the content in the image gradually offsets along the motion trajectory. In the multiple offset field groups, the offset fields included in different offset field groups are not completely the same, for example, the number of offset fields included in any two offset field groups is different, or the number of offset fields included is the same but the offset fields are not completely the same.

[0103] The representative offset field is used to refer to the offset size contained in the offset field in the corresponding offset field group, reflecting the offset amplitude of the offset field in the corresponding offset field group. Since the offset fields contained in different offset field groups are not exactly the same, the representative offset fields corresponding to different offset field groups may be different.

[0104] In step 202, pixel points in the first image are offset according to representative offset fields corresponding to the multiple offset field groups to obtain second images corresponding to the multiple offset field groups.

[0105] The first image is an image of any type, for example, the first image is a landscape image, or an image containing animals, etc. In the disclosed embodiment, for any representative offset field, the pixels in the first image are offset according to the representative offset field, so that the position of each pixel in the first image changes, so that the content of the first image is offset, for example, the position of the object contained in the image is offset in the image, and a new image is generated, namely, the second image. In the above manner, multiple second images can be obtained by offsetting the pixels in the first image through the representative offset fields corresponding to the multiple offset field groups.

[0106] In step 203, based on the second images corresponding to the multiple offset field groups, a target offset field group is determined from the multiple offset field groups.

[0107] In the embodiment of the present disclosure, each representative offset field is used to refer to the size of the offset field included in the corresponding offset field group, and the multiple second images are respectively obtained using the representative offset field groups corresponding to the multiple offset field groups. The second image is equivalent to the offset effect image corresponding to the corresponding offset field group, and can reflect the image effect when the pixel points in the image are offset using the offset field in the offset field group corresponding to the offset field group. By comparing the image effects of the second images corresponding to the multiple offset field groups, the offset field group most suitable for the first image is determined from the multiple offset field groups, that is, the target offset field group is determined to ensure the quality of the image subsequently obtained by offsetting using the target offset field group, and then ensure the quality of the subsequently generated video.

[0108] In step 204, pixel points in the first image are offset according to at least one offset field in the target offset field group to obtain target images, thereby generating a video including at least one target image.

[0109] After determining the target offset field group, the pixel points in the first image are offset according to any offset field in the target offset field group to obtain a target image. According to the above method, the first image is offset by using at least one offset field in the target offset field group to obtain the target image corresponding to each offset field in the target offset field group, that is, at least one target image is obtained. Since the pixel points in each target image are offset relative to the first image, and the offset degrees of the pixel points in different target images are different, the generated video containing multiple target images reflects the effect of the content in the target image being offset when playing, simulating the gradual offset of the content in the captured video due to the movement of the camera during real shooting, and realizing a method of generating dynamic video using images.

[0110] The disclosed embodiments provide a method for generating dynamic videos using images, wherein each offset group represents a motion trajectory, and representative offset fields of a plurality of offset field groups are used to select an offset field group for generating a video, so as to ensure that the offset field group most suitable for the current image can be selected, that is, a motion trajectory suitable for generating a video from the current image is screened out from a plurality of motion trajectories, and at least one image is generated using the selected offset field group, and then a video containing the at least one image is generated, so that the video can simulate the effect of the content in the image being offset along the screened motion trajectory, so as to ensure the quality of the image used to generate the video, avoid the situation where a large number of pseudo-textures appear in the image in the video due to an inappropriate selection of the offset field group, and then ensure the quality of the video.

[0111] In the above Figure 2 On the basis of the illustrated embodiment, taking the example that each offset field group includes multiple offset fields, the offset field includes a horizontal offset and a vertical offset, and the representative offset field includes the maximum horizontal offset and the maximum vertical offset in the corresponding offset field group, in the process of generating a video, the black pixel ratio of the image is compared to screen out the target offset field group for generating the video to ensure the quality of the video. The specific process is described in detail in the following embodiment.

[0112] Figure 3 is a flowchart of a video generation method according to an exemplary embodiment. Figure 3 The method is performed by an electronic device and comprises the following steps:

[0113] In step 301, a plurality of offset field groups and representative offset fields corresponding to the plurality of offset field groups are determined, the offset field group includes a plurality of offset fields, and the representative offset field is used to refer to the offset magnitudes included in the plurality of offset fields in the offset field group.

[0114] In the embodiment of the present disclosure, each offset field in the plurality of offset field groups includes a lateral offset and a longitudinal offset, wherein the lateral offset is used to laterally offset the pixel points in the image, and the longitudinal offset is used to longitudinally offset the pixel points in the image. Each offset field group includes a plurality of offset fields, and each offset field group includes a plurality of lateral offsets and a plurality of longitudinal offsets, and the representative offset field corresponding to each offset field group includes the maximum lateral offset and the maximum longitudinal offset in the corresponding offset field group, thereby indicating the offset size included in the plurality of offset fields in the corresponding offset field group, and reflecting the maximum offset amplitude of the plurality of offset fields in the corresponding offset field group.

[0115] In some embodiments, the process of determining a representative offset field corresponding to an offset field group includes: for any offset field group, when the maximum lateral offset and the maximum longitudinal offset in the offset field group belong to the same offset field in the offset field group, determining the offset field as the representative offset field corresponding to the offset field group; when the maximum lateral offset and the maximum longitudinal offset in the offset field group do not belong to the same offset field in the offset field group, the maximum lateral offset and the maximum longitudinal offset included in the offset field group constitute the representative offset field corresponding to the offset field group.

[0116] In some embodiments, the multiple offset fields in each offset field group are arranged in order from small to large or from large to small, so that after subsequently generating images according to the multiple offset fields, the degree of offset of the content contained in the image in the generated multiple images gradually increases or decreases, simulating the gradual offset of the content in the captured video due to the movement of the camera during real shooting, thereby reflecting the motion effect of the gradual offset of the content, ensuring the authenticity of the motion effect, and thus ensuring the motion effect of the subsequently generated video.

[0117] In a possible implementation manner of the above embodiment, multiple offset fields in any offset field group gradually increase or decrease in the order of arrangement, or any offset field group includes m offset fields, and the m offset fields are arranged in the order, with the first n offset fields gradually increasing or decreasing, and the last mn offset fields are equal in size, wherein n is a positive integer less than m, and m is an integer greater than 1.

[0118] In the embodiment of the present disclosure, when the multiple offset fields in the offset field group gradually increase or decrease in the order of arrangement, in the images generated according to the multiple offset fields, the offset degree of the content contained in the image in the generated multiple images gradually increases or decreases, thereby reflecting the motion effect of the content gradually shifting. When the first n offset fields in the offset field group gradually increase or decrease, and the size of the last mn offset fields is equal, in the images generated according to the multiple offset fields, the offset degree of the content contained in the image in the first n generated images gradually increases or decreases, and the position of the content in the last mn generated images remains unchanged, thereby reflecting the motion effect of the object gradually moving and then standing still.

[0119] It should be noted that the embodiment of the present disclosure is described by taking the representative offset field including the maximum transverse offset and the maximum longitudinal offset in the corresponding offset field group as an example, while in another embodiment, when the offset field includes the transverse offset and the longitudinal offset, the representative offset field corresponding to the offset field group includes other transverse offsets and longitudinal offsets. In some embodiments, the process of determining the representative offset field corresponding to the offset field group includes: determining the offset field including the maximum transverse offset in the offset field group as the representative offset field corresponding to the offset field group; or determining the offset field including the maximum longitudinal offset in the offset field group as the representative offset field corresponding to the offset field group; or determining the sum of the transverse offset and the longitudinal offset included in each offset field in the offset field group, and determining the offset field corresponding to the maximum sum in the offset field group as the representative offset field corresponding to the offset field group.

[0120] In the embodiment of the present disclosure, for an offset field group, the offset field group includes multiple lateral offsets and multiple longitudinal offsets, wherein the maximum lateral offset and the maximum longitudinal offset respectively represent the maximum offset amplitudes in the two offset directions, therefore, the offset field including the maximum lateral offset or the maximum longitudinal offset in the offset field group is used as the representative offset field corresponding to the offset field group, so that the representative offset field reflects the size of the offset field in the corresponding offset field group. Furthermore, since each offset field is used to offset the pixel points in the image in the lateral and longitudinal directions, the sum of the lateral offset and the longitudinal offset in an offset field can also reflect the offset amplitude of the offset field, therefore, the offset field corresponding to the maximum sum in the offset field group is selected and determined as the representative offset field corresponding to the offset field group, so that the representative offset field reflects the size of the offsets included in the multiple offset fields in the corresponding offset field group.

[0121] The embodiments of the present disclosure provide a variety of ways to determine a representative offset corresponding to an offset field group. Since the maximum lateral offset and the maximum longitudinal offset respectively represent the maximum offset amplitudes in the two offset directions, the sum of the lateral offset and the longitudinal offset in the offset field can also reflect the offset amplitude of the offset field. Therefore, it is preferred to select an offset field containing the maximum lateral offset or the maximum longitudinal offset from the offset field group, or to select an offset field corresponding to the maximum sum value in the offset field group, and determine it as the representative offset field corresponding to the offset field group, so as to ensure that the determined representative offset field can best represent the corresponding offset field group, thereby ensuring the accuracy of the determined representative offset field.

[0122] In step 302, pixel points in the first image are offset according to representative offset fields corresponding to a plurality of offset field groups to obtain a second image.

[0123] In the embodiment of the present disclosure, for any representative offset field, the pixels in the image are horizontally offset and vertically offset according to the horizontal offset and vertical offset included in the representative offset field, thereby obtaining a new image, i.e., the second image. In the above manner, the first image is offset by the representative offset fields corresponding to the plurality of offset field groups, thereby obtaining a plurality of second images.

[0124] In some embodiments, the first image is an image stored in the electronic device, or is an image received from other devices, which is not limited in the present disclosure.

[0125] In some embodiments, step 302 includes: offsetting point cloud data corresponding to the first image according to representative offset fields corresponding to the plurality of offset field groups, respectively, and generating a second image based on the point cloud data obtained after the offset.

[0126] The point cloud data corresponding to the first image indicates the spatial position of the pixel in the first image. In some embodiments, the three-dimensional coordinates of each point in the point cloud data are composed of the two-dimensional coordinates of the corresponding pixel in the first image and the depth of the pixel. For example, the three-dimensional coordinates of a point in the point cloud data are (X, Y, Z), where X is the horizontal coordinate of the point, and Y is the vertical coordinate of the point. X and Y are based on the horizontal coordinate and vertical coordinate of the pixel corresponding to the point in the first image, and Z is the depth of the pixel corresponding to the point.

[0127] In the disclosed embodiment, when the pixel points in the first image are offset by using the representative offset field to obtain the second image, it is necessary to first obtain the point cloud data of the first image, and then offset the points in the point cloud data according to the offset amount and offset direction contained in the representative offset field, and remap the point cloud data obtained after the offset to the two-dimensional space to render the second image. First, the pixel points of the first image are converted into point cloud data, and then the point cloud data is offset, rather than directly offsetting the pixel points at the pixel level, so that the second image can be generated based on the offset point cloud data, avoiding a large number of black pixels in the second image, improving the quality of the second image, and thus ensuring the quality of the subsequently generated video.

[0128] In addition, each pixel in the first image has a pixel value, and the pixel value of the pixel can represent the color presented by the pixel. The process of generating a second image based on the point cloud data obtained after the offset includes: mapping the point cloud data obtained after the offset to a two-dimensional space to obtain the offset position of each pixel in the first image, and generating the second image based on the pixel value of each pixel in the first image and the offset position of each pixel.

[0129] In a possible implementation of the above embodiment, the process of acquiring point cloud data includes: acquiring a depth map of a first image, the depth map indicating the depth of pixels in the first image; and generating point cloud data corresponding to the first image based on the depth map.

[0130] The depth map is an image that uses the depth of the pixel in the first image as the pixel value. In the depth map, the larger the pixel value of the pixel, the closer the distance between the pixel and the camera imaging plane, and the smaller the pixel value of the pixel, the farther the distance between the pixel and the camera imaging plane.

[0131] After determining the depth map of the first image, which indicates the depth of each pixel in the first image, each pixel can be converted into a three-dimensional space based on the position of each pixel in the first image and the depth of each pixel, to obtain multiple pixel points in the three-dimensional space. The multiple points in the three-dimensional space constitute the point cloud data corresponding to the first image.

[0132] In a possible implementation manner of the above embodiment, the process of acquiring the depth map includes: inputting the first image into a depth estimation model, and acquiring the depth map output by the depth estimation model.

[0133] Among them, the depth estimation model is used to obtain the depth of the pixel points in the image. Any image is input into the depth estimation model, and the depth estimation model can output the depth map corresponding to the image. The depth estimation model is an arbitrary network model. For example, the depth estimation model is the depth estimation model proposed in Deeplens (shallow depth of field effect rendering algorithm). In the embodiment of the present disclosure, the first image is input into the depth estimation model, the depth estimation model obtains the depth of each pixel point in the first image, and uses the depth of the pixel point as the pixel value of the pixel point in the depth map to generate a depth map corresponding to the first image.

[0134] In step 303, a target offset field group is determined from the multiple offset field groups based on the black pixel ratios of the second image corresponding to the multiple offset field groups.

[0135] The black pixel ratio is used to indicate the proportion of black pixels in the image. In the embodiment of the present disclosure, the black pixels are generated in the process of offsetting the pixels in the first image according to the representative offset field. The black pixel ratio can reflect the quality of the second image obtained by the offset. Figure 4 As shown in the figure, many black pixels appear on the head and neck of the duck. Based on the black pixel ratio of the second image corresponding to the multiple offset field groups, a target offset field group suitable for the first image is determined from the multiple offset fields, thereby ensuring the quality of the image subsequently generated according to the target offset field group.

[0136] In some embodiments, the method of determining the black pixel ratio of the second image includes: determining a first number whose pixel values ​​of pixels in the second image are a target value and the total number of pixels in the second image, and determining the ratio between the first number and the total number of pixels as the black pixel ratio of the second pixel. The target value is an arbitrary value, for example, the target value is 0, and in the second image, the pixel value of a pixel is 0, indicating that the pixel is a black pixel, and the pixel value of a pixel is not 0, indicating that the pixel is not a black pixel, and the ratio of the number of black pixels to the total number of pixels in the second image is determined as the black pixel ratio of the second image.

[0137] In some embodiments, step 303 includes the following two methods:

[0138] The first way is to determine the offset field group corresponding to the second image with the smallest black pixel ratio among the multiple second images as the target offset field group.

[0139] In the embodiment of the present disclosure, the higher the black pixel ratio is, the more black pixels there are in the second image, and the lower the image quality is; the lower the black pixel ratio is, the lower the black pixels are in the second image, and the higher the image quality is. Therefore, among the multiple second images, the second image with the smallest black pixel ratio has the highest quality, which means that the quality of the image subsequently generated according to the offset field group corresponding to the second image is high. The offset field group corresponding to the second image with the smallest black pixel ratio among the multiple offset field groups is most suitable for the first image. Therefore, the offset field group is determined as the target offset field group to ensure the quality of the subsequently generated video.

[0140] The second method: when the number of second images whose black pixel ratio is less than the preset ratio is 1, the offset field group corresponding to the second image whose black pixel ratio is less than the preset ratio is determined as the target offset field group; when the number of second images whose black pixel ratio is less than the preset ratio is multiple, the offset field group corresponding to the second image whose black pixel ratio is less than the preset ratio and which represents the largest offset field is determined as the target offset field group.

[0141] In the embodiment of the present disclosure, a preset ratio is set based on the quality requirements of the image. If the black pixel ratio in the second image is less than the preset ratio, it means that there are fewer black pixels in the second image, the quality of the second image is higher, and the quality requirements of the image required for generating the video are met. If the black pixel ratio in the second image is not less than the preset ratio, it means that there are more black pixels in the second image, the quality of the second image is lower, and the quality requirements of the image required for generating the video are met. Therefore, the target offset field group is selected from the offset field group corresponding to the second image that meets the quality requirements. Since there may be one or more second images with a black pixel ratio less than the preset ratio among the multiple second images, when the number of the second images with a black pixel ratio less than the preset ratio is 1, it means that only one second image meets the quality requirements, and the offset field group corresponding to the second image with a black pixel ratio less than the preset ratio is determined as the target offset field group; and when there are multiple second images with a black pixel ratio less than the preset ratio, the offset field group corresponding to the second image with the largest offset field is preferentially selected as the target offset field group, so as to ensure that the content of the multiple images generated based on the target offset field is offset as much as possible, thereby ensuring the motion effect of the video generated subsequently.

[0142] In some embodiments, the process of determining the maximum representative offset field includes: determining the sum of the lateral offset and the longitudinal offset in each representative offset field, and determining the representative offset field corresponding to the maximum sum as the maximum representative offset field among multiple representative offset fields.

[0143] It should be noted that the embodiment of the present disclosure uses the black pixel ratio of the second image to determine the target offset field group, while in another embodiment, there is no need to perform the above step 303, but other methods are adopted to determine the target offset field group from multiple offset field groups based on the second images corresponding to the multiple offset field groups.

[0144] In step 304, pixel points in the first image are offset according to the multiple offset fields in the target offset field group to obtain target images, thereby generating a video including the multiple target images.

[0145] In the embodiment of the present disclosure, the target offset field group includes multiple offset fields, and the pixel points in the first image are offset according to each offset field to obtain a target image, that is, multiple target images are obtained, each target image is an image obtained after the first image is offset to a certain extent. After a video containing the multiple target images is generated, the video can present an effect of gradually shifting the content in the first image, reflecting that the content of the first image moves according to the offset trajectory represented by the target offset field group, thereby improving the dynamic effect of the video.

[0146] In some embodiments, the process of generating the target image includes: offsetting the point cloud data corresponding to the first image according to the multiple offset fields in the target offset field group, and generating the target image based on the point cloud data obtained after the offset. The process of generating multiple target images according to the multiple offset fields in the target offset field group is the same as the above step 302, and will not be repeated here.

[0147] In the above Figure 3 Based on the embodiment shown, a process for screening the target offset field group is also provided, such as Figure 5 As shown, for a first image of a video to be generated, point cloud data corresponding to the first image is obtained; based on representative offset fields corresponding to multiple offset field groups, the point cloud data corresponding to the first image are offset respectively to obtain multiple point cloud data, the multiple point cloud data are mapped into a two-dimensional space, and based on the pixel values ​​of the pixels in the first image, second images corresponding to the multiple offset field groups are rendered; the black pixel ratios of the multiple second images are determined, and based on the black pixel ratios of the multiple second images, a target offset field group is determined.

[0148] The disclosed embodiments provide a method for generating dynamic videos using images, wherein each offset group represents a motion trajectory, and representative offset fields of a plurality of offset field groups are used to select an offset field group for generating a video, so as to ensure that the offset field group most suitable for the current image can be selected, that is, a motion trajectory suitable for generating a video from the current image is screened out from a plurality of motion trajectories, and at least one image is generated using the selected offset field group, and then a video containing the at least one image is generated, so that the video can simulate the effect of the content in the image being offset along the screened motion trajectory, so as to ensure the quality of the image used to generate the video, avoid the situation where a large number of pseudo-textures appear in the image in the video due to an inappropriate selection of the offset field group, and then ensure the quality of the video.

[0149] In the embodiment of the present disclosure, each offset field group is equivalent to an offset trajectory. The first image is first offset based on the representative offset field corresponding to each offset trajectory to obtain multiple offset effect graphs, that is, multiple second images. The qualities of the multiple offset effect graphs are compared to select an offset trajectory that is most suitable for the first image from the multiple offset trajectories to ensure the quality of the target image obtained by the offset, and further ensure the quality of the generated multiple target images.

[0150] The disclosed embodiment also provides a plurality of methods for screening target offset field groups. When screening a target offset field group from a plurality of offset field groups, the black pixel ratios of the second image corresponding to the plurality of offset field groups can be used to screen out the target offset field group that is most suitable for the first image in any of a plurality of methods, thereby ensuring the accuracy of the screened target offset field group and the quality of the target image obtained by the offset, thereby ensuring the quality of the generated image containing a plurality of target images.

[0151] When screening the target offset field group based on the black pixel ratio and the preset ratio of the second images corresponding to multiple offset field groups, when there are multiple second images with a black pixel ratio less than the preset ratio, the offset field group corresponding to the second image representing the largest offset field is preferentially selected as the target offset field group to ensure that the degree of content offset in the multiple images subsequently generated based on the target offset field is as large as possible, thereby ensuring the motion effect of the subsequently generated video.

[0152] It should be noted that the above Figure 3 The embodiment shown is described by taking the target offset field group including multiple offset fields as an example, while in another embodiment, the target offset field group includes one offset field, and a target image can be obtained according to the above step 304, and a video containing the target image can be generated. In some embodiments, the video also includes a first image, and when the video is played, it can be reflected that the content in the first image is offset from the position in the first image to the position in the target image, so that the video can reflect the effect of the content in the first image being offset along the selected motion trajectory.

[0153] In addition, in the above Figure 3 Based on the embodiment shown, any offset field group contains only one offset field, and the offset field contained in the offset field group is the representative offset field corresponding to the offset field group. In the case where each offset field group contains one offset field, it is not necessary to perform the above steps 303-304, but based on the black pixel ratio of the second images corresponding to the multiple offset field groups, the target image is determined from the multiple second images, and a video containing the target image is generated. The process of determining the target image from the multiple second images based on the black pixel ratio is the same as the above step 303, and will not be repeated here.

[0154] In the above Figure 3 On the basis of the embodiment shown, after the video is generated, the image contained in the video can also be motion blurred to enhance the motion effect of the video and further improve the quality of the video. The specific motion blur processing process is detailed in the following embodiment.

[0155] Figure 6 is a flowchart of a video generation method according to an exemplary embodiment. Figure 6The method is performed by an electronic device and comprises the following steps:

[0156] In step 601, S motion blur kernels corresponding to the target offset field group are obtained, the sizes of the S motion blur kernels decrease in sequence, and S is a positive integer less than T.

[0157] In the embodiment of the present disclosure, the target offset field group includes T offset fields, and the number of target images generated using the target offset field group is also T. The generated video includes T images, where T is an integer greater than 1.

[0158] In the embodiment of the present disclosure, each offset field group corresponds to multiple motion blur kernels, and the multiple motion blur kernels are used to perform motion blur processing on the video generated by using the offset field group. Each motion blur kernel is used to perform motion blur processing on an image. The sizes of the multiple motion blur kernels decrease successively, so that the content in the multiple images after motion blur processing gradually becomes clear from blurry, so that the images in the video can reflect the process from motion to clarity.

[0159] In some embodiments, among the S motion blur kernels, the size of each motion blur kernel is the ratio of the size of the first motion blur kernel to the sequence number of the motion blur kernel. For example, the size of the first motion blur kernel is E, and the size of each motion blur kernel is E / L, where L is the sequence number of each motion blur kernel, and L is a positive integer not greater than S. In addition, since E / L may be a decimal, the size of each motion blur kernel is determined by rounding. For example, S is 3, the size E of the first motion blur kernel is 52×52, the size E / 2 of the second motion blur kernel is 26×26, and the size E / 3 of the third motion blur kernel is 17×17.

[0160] In some embodiments, the current device stores S motion blur kernels corresponding to the target offset field group, and the S motion blur kernels are obtained from the stored location. In the embodiment of the present disclosure, multiple motion blur kernels corresponding to each offset field group are generated in advance and stored in the current device. When it is necessary to obtain multiple motion blur kernels corresponding to any offset field group, they can be directly obtained from the stored location.

[0161] In some embodiments, the process of determining the number S of motion blur kernels corresponding to the target offset field group includes: obtaining a motion blur ratio A, and determining the number S of motion blur kernels corresponding to the target offset field group based on the motion blur ratio A and the number T of offset fields included in the target offset field group, where S is a positive integer not greater than T*A and having a minimum difference with T*A. For example, if T is 25 and A is 0.3, then T*A is 7.5, and then S is 7; or, if T is 20 and A is 0.3, then T*A is 6, and then S is 6.

[0162] In the embodiment of the present disclosure, the motion blur ratio A is used to represent the ratio of the number of images to be motion blurred in the video, and the value range of the motion blur ratio A is greater than 0 and less than 1. Since different offset field groups may contain different numbers of offset fields, the number of motion blur kernels corresponding to different offset field groups may be different. After determining any offset field group, the number of motion blur kernels corresponding to the offset field group can be determined in the above manner.

[0163] In some embodiments, each motion blur kernel includes multiple weights, and the process of determining multiple motion blur kernels includes: after determining the number of motion blur kernels corresponding to any offset field group, determining the size of each motion blur kernel, and using a Gaussian function, a set blur radius, and a variance to determine the weight value contained in each motion blur kernel.

[0164] In the above embodiment, the motion blur kernel is represented in the form of a matrix. For example, any motion blur kernel is a 3×3 matrix, and the motion blur kernel includes 9 weights.

[0165] In step 602, motion blur processing is performed on the first S images in the video based on S motion blur kernels.

[0166] In the disclosed embodiment, each motion blur kernel is used to perform motion blur processing on an image. Then, based on S motion blur kernels, motion blur processing is performed on the first S images in the video. The images other than the first S images in the video remain unchanged, and the sizes of the S motion blur kernels decrease in sequence. When the video after motion blur processing is played, the effect of the content in the image gradually changing from blurry to clear due to movement can be reflected, simulating the situation in which the content in the video captured is blurred due to the movement of the camera during real shooting, thereby improving the motion effect of the video. Figure 7 As shown, Figure 7 The left image in the figure is the image before motion blur processing. Figure 7 The right picture in the figure is the image after motion blur processing. By comparison, it can be seen that the image after motion blur processing can simulate the situation in which the content in the captured video is blurred due to the movement of the camera during real shooting.

[0167] In some embodiments, step 602 includes: for the i-th image, based on the i-th motion blur kernel, the pixel value of the pixel in the image, and the pixel value of the pixel's neighborhood pixels, updating the pixel points in the i-th image to obtain an updated i-th image, where i is a positive integer not greater than S.

[0168] In the embodiment of the present disclosure, for any pixel in the ith image, the pixel value of the pixel is updated based on the ith motion blur kernel, the pixel value of the pixel and the neighborhood pixels of the pixel, so that the pixel values ​​of the neighborhood pixels are integrated into the pixel, so that the pixel value of the updated pixel is close to the pixel value of the neighborhood pixels. In the above manner, after the pixel value of each pixel in the ith image is updated, each pixel is close to the pixel value of the neighborhood pixels, thereby achieving the effect of image blur.

[0169] In a possible implementation of the above embodiment, the motion blur kernel is represented in the form of a matrix, and the motion blur kernel includes multiple weights. The process of updating the pixel points in the i-th image includes: for each pixel point in the i-th image, determine an area centered on the pixel point and having the same size as the i-th motion blur kernel, and all pixels in the area except the pixel point at the center are neighborhood pixels of the pixel point at the center. According to the position of each pixel point in the area, determine the product of the pixel value of each pixel point in the area and the weight of the same position in the i-th motion blur kernel, and determine the sum of the multiple products as the updated pixel value of the pixel point at the center of the area.

[0170] For example, taking the size of the i-th motion blur kernel as 3×3, for any pixel in the i-th image, a 3×3 area centered on the pixel is determined, and all pixels in the area except the pixel at the center are neighborhood pixels of the pixel at the center. The sum of the products of each pixel in the area and the weight of the corresponding position in the i-th motion blur kernel is determined as the updated pixel value of the pixel at the center.

[0171] It should be noted that since the i-th image includes pixels located at the edge, in the process of updating the pixel values ​​of the pixels in the above manner, when determining the area centered on the pixel located at the edge and with the same size as the i-th motion blur kernel, the positions outside the i-th image in the determined area are supplemented by adding 0, and then the updated pixel values ​​of the pixels located at the edge are determined in the above manner.

[0172] The disclosed embodiments provide a method for performing motion blur processing on a video, using multiple motion blur kernels whose number is less than the number of images contained in the video and whose sizes decay successively, to perform motion blur processing on the first multiple images in the video, so that the multiple images after the motion blur processing produce an effect of blurred images due to motion, so that the images in the video after the motion blur processing can reflect the effect from motion blur to clarity, simulating the situation in which the content in the captured video is blurred due to the movement of the camera during real shooting, thereby improving the motion effect of the video after the motion blur processing.

[0173] In the above Figure 6 On the basis of the embodiment shown, before the image in the video is subjected to motion blur processing, the motion blur kernel can also be updated to ensure the motion effect of the subsequently generated video. The specific process is detailed in the following embodiment.

[0174] Figure 8 is a flowchart of a video generation method according to an exemplary embodiment. Figure 8 The method is performed by an electronic device and comprises the following steps:

[0175] In step 801, based on the first S+1 offset fields in the target offset field group, S angles are determined, where the S angles are the angles between vectors corresponding to every two adjacent offset fields in the first S+1 offset fields.

[0176] In the embodiment of the present disclosure, each offset field in the target offset field group corresponds to a vector, which is a vector pointing from the origin to the point corresponding to the offset field. For example, if the offset field is represented in the form of (X, Y), the vector corresponding to the offset field is a vector pointing from the origin to the point with the offset field as the coordinate. For the first S+1 offset fields, the first angle is determined based on the first offset field and the second offset field, and the second angle is determined based on the second offset field and the third offset field, until the Sth angle is obtained.

[0177] In step 802, S motion blur kernels are updated based on the S angles.

[0178] In the embodiment of the present disclosure, each angle is used to represent the degree of difference between the corresponding two offset fields. Therefore, the motion blur kernel is updated using the angle so that the updated motion blur kernel can reflect the difference between adjacent images, simulating the motion blur effect caused by motion during real shooting, thereby ensuring the subsequent motion blur processing effect, and further ensuring the motion effect of the video after motion blur processing.

[0179] In some embodiments, the S angles correspond to the S motion blur kernels in a one-to-one order, and based on each angle, the motion blur kernel corresponding to the angle is updated to obtain the updated S motion blur kernels. For example, taking S as 3 as an example, the first motion blur kernel is updated based on the first angle, the second motion blur kernel is updated based on the second angle, and the third motion blur kernel is updated based on the third angle.

[0180] In some embodiments, the motion blur kernel is represented in the form of a matrix, and based on the S angles, the S motion blur kernels are transformed to obtain S transformed matrices, that is, S updated motion blur kernels.

[0181] In some embodiments, before step 802, the S angles can be updated by using a time series smoothing method. The process of updating the S angles includes the following two methods:

[0182] The first method: perform weighted fusion on the zero angle and the first angle to obtain the updated first angle; perform weighted fusion on the updated j-1th angle and the jth angle to obtain the updated jth angle, where j is an integer greater than 1 and not greater than S.

[0183] In the embodiment of the present disclosure, when updating each angle, it is necessary to use an angle before the angle. For the first angle, a zero angle is assigned to the first angle to update the first angle. The angle of the zero angle is 0. According to the arrangement order of the S angles, the S angles are time-series smoothed to avoid excessive differences between the S angles, so that the updated S angles tend to be smooth, so as to ensure that the motion blur kernel after subsequent updates tends to be smooth, and then ensure that the image in the video after motion blur processing tends to be smooth, which reduces the problem of image flickering in the video and improves the motion effect of the video.

[0184] In some embodiments, the process of updating the S angles in the first manner satisfies the following relationship:

[0185] T1=T0*B+T1*(1-B)

[0186] T k =T k-1 *B+T k *(1-B)

[0187] Among them, T1, T k-1 、T k They are the 1st angle, k-1th angle and kth angle among the S angles, k is an integer greater than 2 and not greater than S; T0 is the zero angle; B is used to represent the weight, and B is a constant greater than 0 and less than 1.

[0188] The second method: perform weighted fusion on the first angle and the second angle to obtain the updated second angle; perform weighted fusion on the updated k-1th angle and the kth angle to obtain the updated kth angle, where k is an integer greater than 2 and not greater than S.

[0189] In the embodiment of the present disclosure, when updating each angle, it is necessary to use an angle before the angle. Since there is no angle before the first angle, the first angle remains unchanged, and each angle starting from the second is merged with the previous angle to avoid excessive differences between the S angles, so that the updated S angles tend to be smooth, to ensure that the motion blur kernel after subsequent updates tends to be smooth, and then to ensure that the image in the video after motion blur processing tends to be smooth, thereby reducing the problem of image flickering in the video and improving the motion effect of the video.

[0190] In some embodiments, after the motion blur kernel is updated, each motion blur kernel needs to be normalized, and then the image in the video is motion blurred based on the normalized motion blur kernel to avoid color changes in the image after motion blur processing and ensure the quality of the image after subsequent motion blur processing.

[0191] In the solution provided by the embodiment of the present disclosure, the first S offset fields in the offset field group are used to determine S angles to express the degree of difference between every two adjacent offset fields in the first S offset fields, and the S motion blur kernels are updated using the S angles, so that the updated motion blur kernels can reflect the differences between adjacent images, simulate the motion blur effect caused by motion during real shooting, ensure the subsequent motion blur processing effect, and further ensure the motion effect of the video after motion blur processing.

[0192] After determining the S angles, the S angles are updated using a time-series smoothing method to avoid excessive differences between the S angles, so that the updated S angles tend to be smooth, to ensure that the motion blur kernel after subsequent updates tends to be smooth, and further to ensure that the image in the video after motion blur processing tends to be smooth, thereby reducing the frequency of image flickering in the video and improving the motion effect of the video.

[0193] In the above Figures 3 to 8 Based on the embodiment shown in FIG. 1 , the present disclosure also provides a video generation method, see Fig. 9 The method is performed by an electronic device and comprises the following steps:

[0194] In step 901, the motion blur ratio A is obtained.

[0195] In step 902, based on the motion blur ratio A and the number T of offset fields included in the target offset field group, the number S of motion blur kernels corresponding to the target offset field group is determined, where S is a positive integer not greater than T*A and having a minimum difference with T*A.

[0196] In step 903, after determining the number of motion blur kernels corresponding to any offset field group, the size of each motion blur kernel is determined, and the sizes of the S motion blur kernels decrease in sequence. The size of each motion blur kernel is the ratio of the size of the first motion blur kernel to the serial number of the motion blur kernel; using the Gaussian function, the set blur radius and the variance, the weight value contained in each motion blur kernel is determined, that is, S motion blur kernels are obtained.

[0197] In step 904, each motion blur kernel is normalized.

[0198] In some embodiments, the motion blur kernel includes multiple weights, and the process of normalizing the motion blur kernel includes: determining the sum of the multiple weights, determining the ratio between each weight and the sum, and forming the normalized motion blur kernel with the multiple ratios.

[0199] In step 905, based on the first S+1 offset fields in the target offset field group, S angles are determined, where the S angles are the angles between vectors corresponding to every two adjacent offset fields in the first S+1 offset fields.

[0200] In step 906, time series smoothing is performed on the S angles.

[0201] The process of performing time series smoothing on the S angles is the same as the process of performing time series smoothing on the S angles in the above step 802, and will not be described in detail here.

[0202] In step 907, based on the S smoothed angles, the S normalized motion blur kernels are updated.

[0203] In step 908, each updated motion blur kernel is normalized.

[0204] In step 909, motion blur processing is performed on the first S images in the video based on the normalized S motion blur kernels.

[0205] It should be noted that the embodiment of the present disclosure is described by taking the example of performing motion blur processing on a video after normalizing each updated motion blur kernel. In another embodiment, firstly, according to the above steps 901-908, a plurality of motion blur kernels corresponding to each offset field group are determined and stored. Subsequently, after a video is generated using any offset field group, the plurality of motion blur kernels corresponding to the offset field group are obtained from the stored location, and the video is motion blurred according to the above step 909.

[0206] In the above Figure 3On the basis of the embodiment shown, after the video is generated, a variable speed trajectory synthesis algorithm can also be adopted to adjust the frame rate of multiple time periods of the video to enhance the motion effect of the video playback. In some embodiments, multiple time periods are determined according to the duration of the video, the multiple time periods are equal and the sum of the multiple time periods is the duration of the video, and the number of images in each time period is equal; the number of images in each time period is adjusted so that the frame rates of the adjusted video in multiple time periods are not exactly the same, so that the video can reflect the speed-changing effect of the motion being fast and slow when it is played, thereby enhancing the dynamic special effects of the video.

[0207] For example, if 10 images are set in the first time period of the video and 5 images are set in the remaining time periods, the generated video can reflect the special effect of first fast motion and then slow motion when playing. For another example, if the first time period of the video is adjusted to a fast frame rate and the remaining time periods are adjusted to a slow frame rate, the content in the image can reflect the effect of first fast motion and then slow motion when playing the video.

[0208] It should be noted that the above embodiments can be combined arbitrarily. Figure 3 The embodiment shown generates a video, and then according to the above Figures 6 to 9 The embodiment shown performs motion blur processing on the generated video. After that, according to the above embodiment, the frame rates of multiple time periods of the video after motion blur processing are adjusted so that the frame rates of the adjusted video in multiple time periods are not completely the same, so that the video can reflect the speed change effect of the motion being fast and slow when playing, thereby improving the dynamic special effects of the video. Figure 3 After the video is generated in the embodiment shown in the figure, the frame rates of the video in multiple time periods may be adjusted first, so that the frame rates of the adjusted video in multiple time periods are not completely the same, and then, according to the above Figures 6 to 9 The illustrated embodiment performs motion blur on the generated video.

[0209] Fig.10 is a block diagram of a video generation device according to an exemplary embodiment. Fig.10 , the device comprises:

[0210] A determining unit 1001 is configured to determine a plurality of offset field groups and representative offset fields corresponding to the plurality of offset field groups, wherein the offset field group includes at least one offset field, and the representative offset field is used to refer to the offset magnitude included in the offset field in the offset field group;

[0211] The offset unit 1002 is configured to execute, according to the representative offset fields corresponding to the multiple offset field groups, respectively offset the pixel points in the first image to obtain second images corresponding to the multiple offset field groups;

[0212] The determining unit 1001 is further configured to determine a target offset field group from the multiple offset field groups based on the second images corresponding to the multiple offset field groups;

[0213] The generating unit 1003 is configured to execute, according to at least one offset field in the target offset field group, offset the pixel points in the first image respectively to obtain the target image, and generate a video including the at least one target image.

[0214] In some embodiments, the determining unit 1001 is configured to determine a target offset field group from a plurality of offset field groups based on black pixel ratios of the second image corresponding to the plurality of offset field groups.

[0215] In some embodiments, the determination unit 1001 is configured to determine the offset field group corresponding to the second image with the smallest black pixel ratio among the plurality of second images as the target offset field group.

[0216] In some embodiments, the determination unit 1001 is configured to, when the number of second images whose black pixel ratio is less than a preset ratio is 1, determine the offset field group corresponding to the second images whose black pixel ratio is less than the preset ratio as the target offset field group; and when the number of second images whose black pixel ratio is less than the preset ratio is multiple, determine the offset field group corresponding to the second image whose black pixel ratio is less than the preset ratio and represents the largest offset field as the target offset field group.

[0217] In some embodiments, the offset field includes a lateral offset and a longitudinal offset; the determination unit 1001 is configured to execute the execution of forming a representative offset field corresponding to the offset field group with the maximum lateral offset and the maximum longitudinal offset included in the offset field group; or, determining the offset field including the maximum lateral offset in the offset field group as the representative offset field corresponding to the offset field group; or, determining the offset field including the maximum longitudinal offset in the offset field group as the representative offset field corresponding to the offset field group; or, determining the sum of the lateral offset and the longitudinal offset included in each offset field in the offset field group, and determining the offset field corresponding to the maximum sum in the offset field group as the representative offset field corresponding to the offset field group.

[0218] In some embodiments, Fig.11 As shown, the device also includes:

[0219] The blur processing unit 1004 is configured to perform motion blur processing on the image contained in the video.

[0220] In some embodiments, the video includes T images, where T is an integer greater than 1; the blur processing unit 1004 is configured to execute S motion blur kernels corresponding to the target offset field group, the sizes of the S motion blur kernels decrease successively, and S is a positive integer less than T; based on the S motion blur kernels, motion blur processing is performed on the first S images in the video.

[0221] In some embodiments, the target offset field set includes T offset fields, such as Fig.11 As shown, the device also includes:

[0222] The determining unit 1001 is further configured to determine S angles based on the first S+1 offset fields in the target offset field group, where the S angles are angles between vectors corresponding to every two adjacent offset fields in the first S+1 offset fields;

[0223] The updating unit 1005 is configured to update the S motion blur kernels based on the S angles.

[0224] In some embodiments, the updating unit 1005 is further configured to perform weighted fusion of the zero angle and the first angle to obtain the updated first angle; perform weighted fusion of the updated j-1th angle and the jth angle to obtain the updated jth angle, where j is an integer greater than 1 and not greater than S; or,

[0225] The first angle and the second angle are weightedly fused to obtain the updated second angle; the updated k-1th angle and the kth angle are weightedly fused to obtain the updated kth angle, where k is an integer greater than 2 and not greater than S.

[0226] In some embodiments, the blur processing unit 1004 is configured to perform, for the i-th image, an update of the pixels in the image based on the i-th motion blur kernel, the pixel values ​​of the pixels in the image, and the pixel values ​​of the pixel's neighborhood pixels to obtain an updated image, where i is a positive integer not greater than S.

[0227] In some embodiments, the offset unit 1002 is configured to execute offsetting of the point cloud data corresponding to the first image according to the representative offset fields corresponding to the plurality of offset field groups, respectively, and generate second images corresponding to the plurality of offset field groups based on the point cloud data obtained after the offsetting;

[0228] The generating unit 1003 is configured to execute offsetting of the point cloud data corresponding to the first image according to at least one offset field in the target offset field group, and generate a target image based on the point cloud data obtained after the offset.

[0229] In some embodiments, Fig.11 As shown, the device also includes:

[0230] An acquiring unit 1006 is configured to acquire a depth map of the first image, where the depth map indicates the depth of pixels in the first image;

[0231] The generating unit 1003 is further configured to generate point cloud data corresponding to the first image based on the depth map.

[0232] In some embodiments, the acquisition unit 1006 is configured to input the first image into the depth estimation model to acquire the depth map output by the depth estimation model.

[0233] Regarding the device in the above embodiment, the specific manner in which each unit performs the operation has been described in detail in the embodiment of the method, and will not be elaborated here.

[0234] In an exemplary embodiment, an electronic device is also provided, characterized in that the electronic device includes:

[0235] one or more processors;

[0236] volatile or non-volatile memory for storing one or more processor executable instructions;

[0237] Among them, one or more processors are configured to execute the steps performed by the electronic device in the above-mentioned video generation method.

[0238] In some embodiments, the electronic device is provided as a terminal. Fig.12 1 is a block diagram of a terminal 1200 according to an exemplary embodiment. The terminal 1200 may be a portable mobile terminal, such as a smart phone, a tablet computer, an MP3 player (Moving Picture Experts Group Audio Layer III), an MP4 player (Moving Picture Experts Group Audio Layer IV), a laptop computer or a desktop computer. The terminal 1200 may also be referred to as a user device, a portable terminal, a laptop terminal, a desktop terminal or other names.

[0239] The terminal 1200 includes: a processor 1201 and a memory 1202 .

[0240] The processor 1201 may include one or more processing cores, such as a 4-core processor, an 8-core processor, etc. The processor 1201 may be implemented in at least one hardware form of DSP (Digital Signal Processing), FPGA (Field-Programmable Gate Array), and PLA (Programmable Logic Array). The processor 1201 may also include a main processor and a coprocessor. The main processor is a processor for processing data in the awake state, also known as a CPU (Central Processing Unit); the coprocessor is a low-power processor for processing data in the standby state. In some embodiments, the processor 1201 may be integrated with a GPU (Graphics Processing Unit), which is responsible for rendering and drawing the content to be displayed on the display screen. In some embodiments, the processor 1201 may also include an AI (Artificial Intelligence) processor, which is used to process computing operations related to machine learning.

[0241] The memory 1202 may include one or more computer-readable storage media, which may be non-transitory. The memory 1202 may also include a high-speed random access memory, and a non-volatile memory, such as one or more disk storage devices, flash memory storage devices. In some embodiments, the non-transitory computer-readable storage medium in the memory 1202 is used to store at least one program code, which is used to be executed by the processor 1201 to implement the video generation method provided by the method embodiment of the present disclosure.

[0242] In some embodiments, the terminal 1200 may also optionally include: a peripheral device interface 1203 and at least one peripheral device. The processor 1201, the memory 1202 and the peripheral device interface 1203 may be connected via a bus or a signal line. Each peripheral device may be connected to the peripheral device interface 1203 via a bus, a signal line or a circuit board. Specifically, the peripheral device includes: at least one of a radio frequency circuit 1204, a display screen 1205, a camera assembly 1206, an audio circuit 1207, a positioning assembly 1208 and a power supply 1209.

[0243] The peripheral device interface 1203 may be used to connect at least one peripheral device related to I / O (Input / Output) to the processor 1201 and the memory 1202. In some embodiments, the processor 1201, the memory 1202, and the peripheral device interface 1203 are integrated on the same chip or circuit board; in some other embodiments, any one or two of the processor 1201, the memory 1202, and the peripheral device interface 1203 may be implemented on a separate chip or circuit board, which is not limited in this embodiment.

[0244] The radio frequency circuit 1204 is used to receive and transmit RF (Radio Frequency) signals, also known as electromagnetic signals. The radio frequency circuit 1204 communicates with the communication network and other communication devices through electromagnetic signals. The radio frequency circuit 1204 converts electrical signals into electromagnetic signals for transmission, or converts received electromagnetic signals into electrical signals. Optionally, the radio frequency circuit 1204 includes: an antenna system, an RF transceiver, one or more amplifiers, a tuner, an oscillator, a digital signal processor, a codec chipset, a user identity module card, and the like. The radio frequency circuit 1204 can communicate with other terminals through at least one wireless communication protocol. The wireless communication protocol includes, but is not limited to: the World Wide Web, a metropolitan area network, an intranet, various generations of mobile communication networks (2G, 3G, 4G and 5G), a wireless local area network and / or a WiFi (Wireless Fidelity) network. In some embodiments, the radio frequency circuit 1204 may also include circuits related to NFC (Near Field Communication), which is not limited in the present disclosure.

[0245] The display screen 1205 is used to display a UI (User Interface). The UI may include graphics, text, icons, videos, and any combination thereof. When the display screen 1205 is a touch display screen, the display screen 1205 also has the ability to collect touch signals on the surface or above the surface of the display screen 1205. The touch signal can be input to the processor 1201 as a control signal for processing. At this time, the display screen 1205 can also be used to provide virtual buttons and / or virtual keyboards, also known as soft buttons and / or soft keyboards. In some embodiments, the display screen 1205 can be one, set on the front panel of the terminal 1200; in other embodiments, the display screen 1205 can be at least two, respectively set on different surfaces of the terminal 1200 or in a folding design; in other embodiments, the display screen 1205 can be a flexible display screen, set on a curved surface or a folding surface of the terminal 1200. Even, the display screen 1205 can also be set to a non-rectangular irregular shape, that is, a special-shaped screen. The display screen 1205 can be made of materials such as LCD (Liquid Crystal Display) and OLED (Organic Light-Emitting Diode).

[0246] The camera assembly 1206 is used to capture images or videos. Optionally, the camera assembly 1206 includes a front camera and a rear camera. The front camera is arranged on the front panel of the terminal, and the rear camera is arranged on the back of the terminal. In some embodiments, there are at least two rear cameras, which are any one of a main camera, a depth of field camera, a wide-angle camera, and a telephoto camera, so as to realize the fusion of the main camera and the depth of field camera to realize the background blur function, the fusion of the main camera and the wide-angle camera to realize the panoramic shooting and VR (Virtual Reality) shooting function or other fusion shooting functions. In some embodiments, the camera assembly 1206 may also include a flash. The flash can be a monochrome temperature flash or a dual-color temperature flash. A dual-color temperature flash refers to a combination of a warm light flash and a cold light flash, which can be used for light compensation at different color temperatures.

[0247] The audio circuit 1207 may include a microphone and a speaker. The microphone is used to collect sound waves from the user and the environment, and convert the sound waves into electrical signals and input them into the processor 1201 for processing, or input them into the radio frequency circuit 1204 to achieve voice communication. For the purpose of stereo acquisition or noise reduction, there may be multiple microphones, which are respectively arranged at different parts of the terminal 1200. The microphone may also be an array microphone or an omnidirectional acquisition microphone. The speaker is used to convert the electrical signal from the processor 1201 or the radio frequency circuit 1204 into sound waves. The speaker may be a traditional film speaker or a piezoelectric ceramic speaker. When the speaker is a piezoelectric ceramic speaker, it can not only convert the electrical signal into sound waves audible to humans, but also convert the electrical signal into sound waves inaudible to humans for purposes such as ranging. In some embodiments, the audio circuit 1207 may also include a headphone jack.

[0248] The positioning component 1208 is used to locate the current geographical location of the terminal 1200 to implement navigation or LBS (Location Based Service). The positioning component 1208 can be a positioning component based on the US GPS (Global Positioning System), China's Beidou system or Russia's Galileo system.

[0249] The power supply 1209 is used to power various components in the terminal 1200. The power supply 1209 can be an alternating current, a direct current, a disposable battery, or a rechargeable battery. When the power supply 1209 includes a rechargeable battery, the rechargeable battery can be a wired rechargeable battery or a wireless rechargeable battery. A wired rechargeable battery is a battery charged through a wired line, and a wireless rechargeable battery is a battery charged through a wireless coil. The rechargeable battery can also be used to support fast charging technology.

[0250] In some embodiments, the terminal 1200 further includes one or more sensors 1210 , including but not limited to: an acceleration sensor 1211 , a gyroscope sensor 1212 , a pressure sensor 1213 , a fingerprint sensor 1214 , an optical sensor 1215 , and a proximity sensor 1216 .

[0251] The acceleration sensor 1211 can detect the magnitude of acceleration on the three coordinate axes of the coordinate system established by the terminal 1200. For example, the acceleration sensor 1211 can be used to detect the components of gravity acceleration on the three coordinate axes. The processor 1201 can control the display screen 1205 to display the user interface in a horizontal view or a vertical view according to the gravity acceleration signal collected by the acceleration sensor 1211. The acceleration sensor 1211 can also be used to collect motion data of games or users.

[0252] The gyro sensor 1212 can detect the body direction and rotation angle of the terminal 1200, and the gyro sensor 1212 can cooperate with the acceleration sensor 1211 to collect the user's 3D actions on the terminal 1200. The processor 1201 can implement the following functions based on the data collected by the gyro sensor 1212: motion sensing (such as changing the UI according to the user's tilt operation), image stabilization during shooting, game control, and inertial navigation.

[0253] The pressure sensor 1213 can be set on the side frame of the terminal 1200 and / or the lower layer of the display screen 1205. When the pressure sensor 1213 is set on the side frame of the terminal 1200, the user's holding signal of the terminal 1200 can be detected, and the processor 1201 performs left and right hand recognition or shortcut operations according to the holding signal collected by the pressure sensor 1213. When the pressure sensor 1213 is set on the lower layer of the display screen 1205, the processor 1201 controls the operability controls on the UI interface according to the user's pressure operation on the display screen 1205. The operability controls include at least one of a button control, a scroll bar control, an icon control, and a menu control.

[0254] The fingerprint sensor 1214 is used to collect the user's fingerprint, and the processor 1201 identifies the user's identity based on the fingerprint collected by the fingerprint sensor 1214, or the fingerprint sensor 1214 identifies the user's identity based on the collected fingerprint. When the user's identity is identified as a trusted identity, the processor 1201 authorizes the user to perform relevant sensitive operations, including unlocking the screen, viewing encrypted information, downloading software, paying, and changing settings. The fingerprint sensor 1214 can be set on the front, back, or side of the terminal 1200. When a physical button or a manufacturer logo is set on the terminal 1200, the fingerprint sensor 1214 can be integrated with the physical button or the manufacturer logo.

[0255] The optical sensor 1215 is used to collect the ambient light intensity. In one embodiment, the processor 1201 can control the display brightness of the display screen 1205 according to the ambient light intensity collected by the optical sensor 1215. Specifically, when the ambient light intensity is high, the display brightness of the display screen 1205 is increased; when the ambient light intensity is low, the display brightness of the display screen 1205 is reduced. In another embodiment, the processor 1201 can also dynamically adjust the shooting parameters of the camera assembly 1206 according to the ambient light intensity collected by the optical sensor 1215.

[0256] The proximity sensor 1216, also called a distance sensor, is disposed on the front panel of the terminal 1200. The proximity sensor 1216 is used to collect the distance between the user and the front of the terminal 1200. In one embodiment, when the proximity sensor 1216 detects that the distance between the user and the front of the terminal 1200 is gradually decreasing, the processor 1201 controls the display screen 1205 to switch from the screen-on state to the screen-off state; when the proximity sensor 1216 detects that the distance between the user and the front of the terminal 1200 is gradually increasing, the processor 1201 controls the display screen 1205 to switch from the screen-off state to the screen-on state.

[0257] Those skilled in the art will understand that Fig.12 The structure shown in the figure does not constitute a limitation on the terminal 1200, and the terminal 1200 may include more or fewer components than those shown in the figure, or combine some components, or adopt a different component arrangement.

[0258] In some embodiments, the electronic device is provided as a server. Fig.13 This is a schematic diagram of the structure of a server according to an exemplary embodiment. The server 1300 may have relatively large differences due to different configurations or performances, and may include one or more processors (Central Processing Units, CPU) 1301 and one or more memories 1302, wherein the memory 1302 stores at least one program code, and the at least one program code is loaded and executed by the processor 1301 to implement the methods provided by the above-mentioned various method embodiments. Of course, the server may also have components such as a wired or wireless network interface, a keyboard, and an input and output interface for input and output, and the server may also include other components for implementing device functions, which will not be described in detail here.

[0259] In an exemplary embodiment, a non-transitory computer-readable storage medium is also provided, and when the instructions in the storage medium are executed by the processor of the electronic device, the electronic device can perform the steps performed by the electronic device in the above-mentioned video generation method. Optionally, the storage medium can be a non-transitory computer-readable storage medium, for example, the non-transitory computer-readable storage medium can be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, etc.

[0260] In an exemplary embodiment, a computer program product is further provided. When instructions in the computer program product are executed by a processor of an electronic device, the electronic device is enabled to execute the steps executed by the electronic device in the above-mentioned video generation method.

[0261] Those skilled in the art will readily appreciate other embodiments of the present disclosure after considering the specification and practicing the disclosure herein. The present disclosure is intended to cover any variations, uses or adaptations of the present disclosure that follow the general principles of the present disclosure and include common knowledge or customary techniques in the art that are not disclosed in the present disclosure. The description and examples are to be considered exemplary only, and the true scope and spirit of the present disclosure are indicated by the following claims.

[0262] It should be understood that the present disclosure is not limited to the exact structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.

Claims

1. A video generation method, characterized in that: The method comprises: Determine a plurality of offset field groups and a plurality of representative offset fields corresponding to the offset field groups, wherein the offset field group includes at least one offset field, and the representative offset field is used to indicate the offset magnitude included in the offset field in the offset field group; According to the representative offset fields corresponding to the plurality of offset field groups, pixel points in the first image are offset respectively to obtain second images corresponding to the plurality of offset field groups; Determining a target offset field group from the plurality of offset field groups based on black pixel ratios of the second image corresponding to the plurality of offset field groups; the higher the black pixel ratio, the lower the image quality, and the lower the black pixel ratio, the higher the image quality; According to at least one offset field in the target offset field group, pixel points in the first image are offset respectively to obtain a target image, and a video containing at least one of the target images is generated.

2. The method according to claim 1, characterized in that The step of determining a target offset field group from the plurality of offset field groups based on the black pixel ratios of the second image corresponding to the plurality of offset field groups comprises: The offset field group corresponding to the second image with the smallest black pixel ratio among the plurality of second images is determined as the target offset field group.

3. The method according to claim 1, characterized in that The step of determining a target offset field group from the plurality of offset field groups based on the black pixel ratios of the second image corresponding to the plurality of offset field groups comprises: When the number of the second images whose black pixel ratio is less than the preset ratio is 1, determining the offset field group corresponding to the second images whose black pixel ratio is less than the preset ratio as the target offset field group; When there are multiple second images whose black pixel ratio is less than the preset ratio, the offset field group corresponding to the second image whose black pixel ratio is less than the preset ratio and represents the largest offset field is determined as the target offset field group.

4. The method according to claim 1, characterized in that: The offset field includes a lateral offset and a longitudinal offset; determining a representative offset field corresponding to a plurality of the offset field groups includes: The maximum lateral offset and the maximum longitudinal offset included in the offset field group constitute a representative offset field corresponding to the offset field group; or, Determine the offset field including the maximum lateral offset in the offset field group as the representative offset field corresponding to the offset field group; or, Determine the offset field including the maximum longitudinal offset in the offset field group as the representative offset field corresponding to the offset field group; or, The sum of the lateral offset and the longitudinal offset included in each offset field in the offset field group is determined, and the offset field corresponding to the maximum sum in the offset field group is determined as the representative offset field corresponding to the offset field group.

5. The method according to claim 1, characterized in that: After generating a video including at least one of the target images, the method further includes: A motion blur process is performed on the image contained in the video.

6. The method according to claim 5, characterized in that The video includes T images, where T is an integer greater than 1; and the step of performing motion blur processing on the images included in the video includes: Obtain S motion blur kernels corresponding to the target offset field group, the sizes of the S motion blur kernels decrease in sequence, and S is a positive integer less than T; Based on the S motion blur kernels, motion blur processing is performed on the first S images in the video.

7. The method according to claim 6, characterized in that The target offset field group includes T offset fields. Before performing motion blur processing on the first S images in the video based on the S motion blur kernels, the method further includes: Based on the first S+1 offset fields in the target offset field group, S angles are determined, where the S angles are angles between vectors corresponding to every two adjacent offset fields in the first S+1 offset fields; Based on the S angles, the S motion blur kernels are updated.

8. The method according to claim 7, characterized in that Before updating the S motion blur kernels based on the S angles, the method further includes: Performing weighted fusion on the zero angle and the first angle to obtain the updated first angle; performing weighted fusion on the updated j-1th angle and the jth angle to obtain the updated jth angle, where j is an integer greater than 1 and not greater than S; or, The first angle and the second angle are weightedly fused to obtain the updated second angle; the updated k-1th angle and the kth angle are weightedly fused to obtain the updated kth angle, where k is an integer greater than 2 and not greater than S.

9. The method according to claim 6, characterized in that The performing motion blur processing on the first S images in the video based on the S motion blur kernels includes: For the i-th image, based on the i-th motion blur kernel, the pixel value of the pixel in the image and the pixel value of the neighborhood pixel of the pixel, the pixel in the image is updated to obtain an updated image, where i is a positive integer not greater than S.

10. The method according to any one of claims 1 to 9, characterized in that: The step of offsetting the pixel points in the first image according to the representative offset fields corresponding to the plurality of offset field groups to obtain the second images corresponding to the plurality of offset field groups comprises: offsetting the point cloud data corresponding to the first image respectively according to the representative offset fields corresponding to the plurality of offset field groups, and generating the second images corresponding to the plurality of offset field groups based on the point cloud data obtained after the offset; The step of offsetting pixel points in the first image according to at least one offset field in the target offset field group to obtain a target image includes: According to at least one offset field in the target offset field group, the point cloud data corresponding to the first image are offset respectively, and the target image is generated based on the point cloud data obtained after the offset.

11. A video generating device, characterized in that: The device comprises: a determining unit configured to determine a plurality of offset field groups and representative offset fields corresponding to the plurality of offset field groups, wherein the offset field group includes at least one offset field, and the representative offset field is used to refer to the offset magnitude included in the offset field in the offset field group; An offset unit is configured to execute, according to representative offset fields corresponding to the plurality of offset field groups, offset pixels in the first image respectively to obtain second images corresponding to the plurality of offset field groups; The determining unit is further configured to determine a target offset field group from the plurality of offset field groups based on black pixel ratios of the second image corresponding to the plurality of offset field groups; the higher the black pixel ratio, the lower the image quality, and the lower the black pixel ratio, the higher the image quality; The generating unit is configured to execute offsetting of pixel points in the first image according to at least one offset field in the target offset field group to obtain a target image, and generate a video containing at least one of the target images.

12. An electronic device, characterized in that: The electronic device comprises: processor; a memory for storing instructions executable by the processor; The processor is configured to execute the instructions to implement the video generating method according to any one of claims 1 to 10.

13. A non-transitory computer-readable storage medium, characterized in that: When the instructions in the storage medium are executed by a processor of an electronic device, the electronic device is enabled to execute the video generating method according to any one of claims 1 to 10.

Images