Video generation method, device, electronic device and computer readable storage medium

By calculating the ripple height information of video frame images and processing their pixel coordinate information, a target image with ripples is generated, which solves the problem of the single nature of existing stylization processing methods and achieves rich video stylization effects and improved user experience.

CN117676201BActive Publication Date: 2024-11-05SHUXING TECH (BEIJING) CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202311734057.X
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2023-12-15
Publication Date
2024-11-05
Estimated Expiration
2043-12-15

AI Technical Summary

Technical Problem

Existing image and video stylization methods are too simplistic and cannot meet the diverse needs of users.

Method used

By acquiring the ripple motion information, time information, and pixel coordinate information of video frame images, the ripple height information is calculated, and this information is used to process the pixel coordinate information of the video frame images to generate a target image with ripples, and finally, the target video is generated.

Benefits of technology

It enables stylized processing of video frame images, transforming them into stylized images with distorted content, thus enriching the stylized processing methods for videos and improving the user experience.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN117676201B_ABST
    Figure CN117676201B_ABST
Patent Text Reader

Abstract

The embodiment of the present application discloses a video generation method, device, electronic device and computer-readable storage medium, the method comprising: obtaining a video to be processed and ripple motion information of a video frame image in the video to be processed, wherein the video to be processed comprises at least two video frame images; determining the ripple height information of the video frame image according to the ripple motion information, time information and pixel coordinate information of the video frame image; processing the pixel coordinate information in the video frame image according to the ripple height information and pixel coordinate information of the video frame image to obtain a target image with ripples; generating a target video according to each target image. The ripple motion information and time information of the video frame image are used to generate a target image corresponding to the video frame image, thereby realizing the generation of a stylized image after the content is twisted, and then obtaining a stylized video.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The embodiments of the present application relate to the field of computers, and specifically to a video generation method, device, electronic device, and computer-readable storage medium. Background Art

[0002] At present, the stylization processing of images or videos mainly includes processing images or videos in styles such as "ancient style" and "animation", so that ordinary images or videos are converted into stylized images or videos to enrich the content and gameplay of images and videos.

[0003] However, the current stylization processing methods are still relatively simple. With the increasing user demands, new stylization processing methods are needed to meet user demands. Summary of the invention

[0004] The embodiments of the present application provide a video generation method, device, electronic device and computer-readable storage medium, which can realize stylized processing of videos, enrich the stylized processing methods of videos, and improve user experience.

[0005] The present application provides a video generation method, the method comprising:

[0006] Acquire a video to be processed and ripple motion information of a video frame image in the video to be processed, wherein the video to be processed includes at least two video frame images;

[0007] Determining ripple height information of the video frame image according to the ripple motion information, time information and pixel coordinate information of the video frame image;

[0008] According to the ripple height information and pixel coordinate information of the video frame image, the pixel coordinate information in the video frame image is processed to obtain a target image with ripples;

[0009] A target video is generated according to each of the target images.

[0010] Accordingly, an embodiment of the present application further provides a video generating device, the device comprising:

[0011] An acquisition module, used to acquire a video to be processed and ripple motion information of a video frame image in the video to be processed, wherein the video to be processed includes at least two video frame images;

[0012] A determination module, used to determine the ripple height information of the video frame image according to the ripple motion information, time information and pixel coordinate information of the video frame image;

[0013] A processing module, used for processing the pixel coordinate information in the video frame image according to the ripple height information and the pixel coordinate information of the video frame image to obtain a target image with ripples;

[0014] A generating module is used to generate a target video according to each of the target images.

[0015] Optionally, in some embodiments of the present application, the determination module includes:

[0016] A first determining unit, configured to determine at least one reference texture point from a texture image corresponding to the video frame image;

[0017] A first acquisition unit is used to obtain, for a texture pixel in a texture image corresponding to the video frame image, a reference point height value of each reference texture point according to texture coordinates of the texture pixel, ripple motion information of the video frame image, time information, and texture coordinates of each reference texture point;

[0018] A second acquisition unit, configured to obtain a periodic noise value of the texture pixel using a periodic function according to the texture coordinates of the texture pixel and the ripple motion information and time information of the video frame image;

[0019] A first generating unit, configured to generate a simulated height value of the texture pixel according to the periodic noise value and a reference point height value of each of the reference texture points;

[0020] The second generating unit is used to generate the ripple height information of the video frame image according to the simulated height value of each texture pixel in the texture image.

[0021] In some embodiments of the present application, the first generating unit includes:

[0022] A first calculation subunit, configured to calculate a ripple height noise value of the texture pixel according to the periodic noise value and a reference point height value of each reference texture point;

[0023] A second calculation subunit, configured to generate a transition coordinate corresponding to the texture pixel according to the ripple height noise value and the texture coordinate of the texture pixel;

[0024] The third calculation subunit is used to generate a simulated height value corresponding to the texture coordinate using a periodic function according to the transition coordinate, the ripple motion information of the video frame image, time information and the texture coordinates of each reference texture point.

[0025] In some embodiments of the present application, the first acquisition unit includes:

[0026] A first acquisition subunit is used to obtain, for a texture pixel in a texture image corresponding to a video frame image, a position offset value between the texture pixel and each reference texture point according to a texture coordinate of the texture pixel and a texture coordinate of each reference texture point;

[0027] The second acquisition subunit is used to calculate the reference point height value of each reference texture point by using a periodic function according to the ripple motion information, time information and position offset value of each reference texture point of the video frame image.

[0028] In some embodiments of the present application, the processing module includes:

[0029] A second determining unit, configured to determine a texture resolution of a texture image corresponding to the video frame image and texture coordinates of texture pixels in the texture image according to pixel coordinate information of the video frame image;

[0030] An extraction unit, used for extracting a simulated height value of the texture pixel from the ripple height information;

[0031] a third determining unit, configured to determine, for each texture pixel in the texture image, a target texture coordinate of the texture pixel after twisting according to a simulated height value and a texture coordinate of the texture pixel and a texture resolution of the texture image;

[0032] The sampling unit is used to sample the video frame image according to the target texture coordinates corresponding to each texture pixel in the texture image to obtain a target image with ripples.

[0033] In some embodiments of the present application, the third determining unit includes:

[0034] A coordinate calculation subunit, configured to calculate, for each texture pixel in the texture image, a first direction offset coordinate and a second direction offset coordinate of the texture pixel according to a simulated height value of the texture pixel and a texture resolution of the texture image;

[0035] an offset height calculation subunit, configured to generate a first direction offset height value corresponding to the first direction offset coordinate according to the ripple motion information and time information of the video frame image, and to generate a second direction offset height value corresponding to the second direction offset coordinate according to the ripple motion information and time information of the video frame image;

[0036] a vector adjustment subunit, configured to adjust an initial normal vector of the texture pixel according to the first direction offset coordinate and the second direction offset coordinate to obtain a target normal vector, wherein the initial normal vector is a vector perpendicular to the two-dimensional space where the texture pixel is located;

[0037] A vector generating subunit, used for generating a reflection vector corresponding to the target normal vector according to the texture coordinates and the simulated height value;

[0038] The coordinate generation subunit is used to generate the target texture coordinates corresponding to the texture pixel according to the reflection vector.

[0039] In some embodiments of the present application, the vector adjustment subunit is specifically used for:

[0040] Obtaining a first direction height increment value according to the first direction offset height value and the simulated height value, and obtaining a second direction height increment value according to the second direction offset height value and the simulated height value;

[0041] Calculate a first-direction height gradient value corresponding to the first-direction height increment value according to an offset step length between two texture pixels corresponding to a texture resolution of the texture image, and calculate a second-direction height gradient value corresponding to the second-direction height increment value according to the offset step length;

[0042] The initial normal vector of the texture pixel is adjusted according to the first direction height gradient value and the second direction height gradient value to obtain a target normal vector.

[0043] In some embodiments of the present application, the coordinate generation subunit is specifically used for:

[0044] Setting the abscissa of the reflection vector to the target abscissa;

[0045] Setting the ordinate of the reflection vector as the target ordinate;

[0046] The texture coordinates corresponding to the target abscissa and the target ordinate are set as the target texture coordinates of the texture pixel.

[0047] In a third aspect, an embodiment of the present application further provides an electronic device, which includes a memory, a processor, and a computer program stored in the memory and executable on the processor, and which implements the steps in the above-mentioned video generation method when the computer program is executed by the processor.

[0048] In a fourth aspect, an embodiment of the present application further provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the steps in the above-mentioned video generation method are implemented.

[0049] In a fifth aspect, the embodiments of the present application further provide a computer program product or a computer program, which includes a computer instruction stored in a computer-readable storage medium. The processor of the computer device reads the computer instruction from the computer-readable storage medium, and the processor executes the computer instruction, so that the computer device executes the method provided in various optional implementations described in the embodiments of the present application.

[0050] In summary, the embodiments of the present application calculate the ripple height information of the video frame image through the ripple motion information, time information and pixel coordinate information corresponding to the video frame image, and use the ripple height information and the pixel coordinate information of the video frame image to process the pixel coordinate information of the video frame image to obtain a target image with ripples, and then generate a target video through the target images corresponding to each video frame image.

[0051] By processing pixel coordinate information in a video frame image, a target image with ripples is obtained, and stylized processing of the video frame image is achieved, so that the video frame image is transformed into a stylized image with twisted content.

[0052] Among them, each video frame image is stylized according to its corresponding ripple motion information and time information, so that the target video presents a ripple flow visual effect based on each stylized image, enriching the stylized processing method of the video and improving the user experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] In order to more clearly illustrate the technical solutions in the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative work.

[0054] Figure 1 is a scene schematic diagram of the video generation method provided in an embodiment of the present application;

[0055] Figure 2 It is a flowchart of a video generation method provided in an embodiment of the present application;

[0056] Figure 3 is a video frame image provided by an embodiment of the present application;

[0057] Figure 4 The embodiment of the present application provides Figure 3 The target image after the content of the video frame image is twisted;

[0058] Figure 5 is an architecture diagram of a video generation system provided in an embodiment of the present application;

[0059] Figure 6 is a schematic diagram of the structure of a video generating device provided in an embodiment of the present application;

[0060] Figure 7 It is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0061] The following will be combined with the drawings in this application to clearly and completely describe the technical solutions in this application. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of them. Based on the embodiments in the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present invention.

[0062] The embodiments of the present application provide a video generation method, device, electronic device and computer-readable storage medium. Specifically, the embodiments of the present application provide a video generation device suitable for an electronic device, which includes a terminal device or a server. Among them, the terminal may include devices such as a mobile phone, a desktop computer, a tablet computer or a laptop computer. The server may be an independent physical server, or a server cluster or distributed system composed of multiple physical servers. It can also be a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content distribution networks (CDN, Content Delivery Network), and big data and artificial intelligence platforms. The server can be directly or indirectly connected via wired or wireless communication.

[0063] In the embodiment of the present application, the video generation method can be executed by the terminal device alone, or by the server alone, or by the terminal device and the server together. For example, see Figure 1 , Figure 1 : is a schematic diagram of a scenario in which a terminal device executes the video generation method provided in an embodiment of the present application, and the video generation method executed by a server or by a terminal device and a server together can be understood by reference, wherein the specific execution process of the video generation method executed by the terminal device is as follows:

[0064] The terminal device 10 obtains a video to be processed and ripple motion information of a video frame image in the video to be processed, wherein the video to be processed includes at least two video frame images, determines the ripple height information of the video frame image according to the ripple motion information, time information and pixel coordinate information corresponding to the video frame image, processes the pixel coordinate information in the video frame image according to the ripple height information and pixel coordinate information of the video frame image, obtains a target image with ripples, and generates a target video corresponding to the video to be processed according to each target image.

[0065] In the embodiment of the present application, the ripple motion information is a parameter used to control the degree of distortion of the image content, and the parameter can be dynamically changed based on the requirement of the degree of distortion. The time information is the timestamp information corresponding to the video frame image.

[0066] Among them, the embodiment of the present application calculates the ripple height information of the video frame image through the ripple motion information, time information and pixel coordinate information corresponding to the video frame image, and uses the ripple height information and the pixel coordinate information of the video frame image to process the pixel coordinate information of the video frame image to obtain a target image with ripples, and then generates a target video through the target images corresponding to each video frame image.

[0067] By processing pixel coordinate information in a video frame image, a target image with ripples is obtained, and stylized processing of the video frame image is achieved, so that the video frame image is transformed into a stylized image with twisted content.

[0068] Among them, each video frame image is stylized according to its corresponding ripple motion information and time information, so that the target video presents a ripple flow visual effect based on each stylized image, enriching the stylized processing method of the video and improving the user experience.

[0069] It should be noted that the order of description of the following embodiments is not intended to limit the priority order of the embodiments.

[0070] See also Figure 2 , Figure 2 The following is a flow chart of a video generation method provided in an embodiment of the present application. The specific flow of the video generation method is as follows:

[0071] 101. Obtain a video to be processed and ripple motion information of a video frame image in the video to be processed, wherein the video to be processed includes at least two video frame images.

[0072] It is understandable that the video to be processed is a video that needs to be processed, and the video can be a video uploaded by a user locally or a video viewed through a video platform.

[0073] The video to be processed may be a video obtained by shooting a real scene, or a video processed based on a style transformation such as "ancient style" or "animation". Accordingly, compared with the style of the target video after transformation in the embodiment of the present application, the content, layout, and style of the video to be processed are usually regular videos that conform to the laws of nature.

[0074] The ripple motion information is a parameter used to control the degree of distortion of the image content, and the parameter can be dynamically changed based on the degree of twisting. For example, in the embodiment of the present application, the ripple motion information includes but is not limited to a fluctuation speed parameter, that is, the degree of fluctuation of the content in the video frame image.

[0075] In this embodiment of the present application, each video frame image corresponds to a ripple motion information.

[0076] 102. Determine ripple height information of the video frame image according to the ripple motion information, time information, and pixel coordinate information of the video frame image.

[0077] In this embodiment of the present application, the time information is the time of the video frame image. For example, the time information includes but is not limited to the timestamp information corresponding to the video frame image.

[0078] It should be noted that the ripple height information is virtual height information. The height value defined for calculation is not the height value of the pixel in the video frame image. It can be understood that the embodiment of the present application defines the pixel height value of the video frame image, so that the pixels of the video frame image are converted from two-dimensional space to three-dimensional space, and the calculation of the pixel coordinate position is realized based on the pixel height value in the three-dimensional space.

[0079] The ripple motion information and time information correspond to the video frame image, that is, each video frame image corresponds to a ripple motion information and time information. Some parameters can also be supplemented or reduced based on the twisting requirements. The ripple motion information includes a fluctuation speed parameter, which is used to control the movement speed of the noise. For example, the higher the value of the fluctuation speed parameter, the faster the noise moves. The time information includes a video frame timestamp parameter, which is the display timestamp corresponding to each video frame image. Through the difference in timestamps, each video frame image is twisted based on the difference in timestamps. When each processed image is combined to obtain a new video, a ripple flow visual effect is formed based on the timestamp.

[0080] Calculating the ripple height information helps to utilize the ripple height information to perform twisting processing on pixel coordinate information in the video frame image, thereby obtaining a target image with ripples.

[0081] 103. Process the pixel coordinate information in the video frame image according to the ripple height information and the pixel coordinate information of the video frame image to obtain a target image with ripples.

[0082] Among them, the pixel coordinate information can be transformed and processed through the ripple height information to obtain a target image with ripples.

[0083] 104. Generate a target video according to each of the target images.

[0084] Among them, the target video corresponding to the video to be processed is obtained by combining and splicing various target images.

[0085] It can be understood that in the embodiments of the present application, the twisting process can be performed on each video frame image in the processed video, or only on part of the video frame images therein. Accordingly, the twisting process can also be performed on only a single video frame image. For example, the twisting transformation can be applied to a single image to obtain a stylized image, etc.

[0086] In summary, the embodiments of the present application calculate the ripple height information of the video frame image through the ripple motion information, time information and pixel coordinate information corresponding to the video frame image, and use the ripple height information and the pixel coordinate information of the video frame image to process the pixel coordinate information of the video frame image to obtain a target image with ripples, and then generate a target video through the target images corresponding to each video frame image.

[0087] By processing pixel coordinate information in a video frame image, a target image with ripples is obtained, and stylized processing of the video frame image is achieved, so that the video frame image is transformed into a stylized image with twisted content.

[0088] Among them, each video frame image is stylized according to its corresponding ripple motion information and time information, so that the target video presents a ripple flow visual effect based on each stylized image, enriching the stylized processing method of the video and improving the user experience.

[0089] Optionally, in an embodiment of the present application, the pixels in the video frame image may be twisted with the help of texture pixels, that is, optionally, in some embodiments of the present application, the step of “determining the ripple height information of the video frame image according to the ripple motion information, time information and pixel coordinate information of the video frame image” includes:

[0090] Determine at least one reference texture point from the texture image corresponding to the video frame image;

[0091] For texture pixels in the texture image corresponding to the video frame image, according to texture coordinates of the texture pixels, ripple motion information of the video frame image, time information and texture coordinates of each reference texture point, obtaining a reference point height value of each reference texture point;

[0092] Obtaining a periodic noise value of the texture pixel using a periodic function according to the texture coordinates of the texture pixel and the ripple motion information and time information of the video frame image;

[0093] Generating a simulated height value of the texture pixel according to the periodic noise value and the reference point height value of each of the reference texture points;

[0094] The ripple height information of the video frame image is generated according to the simulated height value of each texture pixel in the texture image.

[0095] It can be understood that for a two-dimensional video frame image, its corresponding texture image also has two-dimensional texture coordinates. For example, the image coordinates (x, y) of the video frame image correspond to the texture coordinates (u, v) of the texture image. u represents the horizontal coordinate, and the value range is usually [0, 1], where 0 represents the left edge of the texture and 1 represents the right edge of the texture. v represents the vertical coordinate, and the value range is also [0, 1], where 0 represents the bottom edge of the texture and 1 represents the top edge of the texture. Texture pixels are also called image uv, which are used to locate specific pixels in a texture image.

[0096] Among them, the texture image is obtained by mapping and transforming the video frame image. For example, the relationship between the image uv and the image coordinates can be realized through mapping. For example, assuming that the width of the image is W pixels and the height is H pixels, the image coordinates (x, y) can be calculated by the following formula to obtain the corresponding image uv coordinates (u, v): u=x / W v=y / H; conversely, the image uv coordinates (u, v) can be calculated by the following formula to obtain the corresponding image coordinates (x, y): x=round(u*W)y=round(v*H).

[0097] The simulated height value of the texture pixel is not the real height value of the texture pixel. It can be understood that the embodiment of the present application defines the height value of the texture pixel, so that the texture pixel is converted from the two-dimensional space to the three-dimensional space, and the new texture coordinates of the texture pixel are calculated based on the height information of the texture pixel in the three-dimensional space. For example, the simulated height value can be understood as the Z-axis coordinate of the texture pixel when it is mapped to the three-dimensional model.

[0098] Among them, the reference texture point is a texture pixel in the texture image. The reference texture point can be the same as the texture pixel of the simulated height value to be calculated, or it can be different. Accordingly, the number of the reference texture points can also be customized based on demand. For example, in an embodiment of the present application, the texture pixels corresponding to the four corners of the texture image are used as reference texture points, that is, the four texture pixels with texture coordinates of (1.0, 1.0), (1.0, 0.0), (0.0, 1.0) and (0.0, 0.0) in the texture image are used as reference texture points. Correspondingly, the texture pixels at the midpoints of the four edges of the texture image can also be selected as reference texture points.

[0099] The reference point height value is the height value corresponding to the reference texture point, which is also defined based on calculation requirements, and is not the value corresponding to the actual height of the texture pixel. Correspondingly, it can also be understood as the height value of the reference texture point when it is mapped to the three-dimensional model. By calculating the reference point height value of each reference texture point, the reference point height value can be used to assist in calculating the simulated height value of the texture pixel, adding more details to the calculation of the simulated height value and improving the accuracy of the calculation of the simulated height value.

[0100] It can be understood that the periodic noise value is calculated based on the periodic demand of the fluctuation of the texture pixel, and is used to characterize the noise value of the texture pixel in the fluctuation period. Since the ripple motion information and time information corresponding to each texture image are different, the periodic noise value of each texture pixel is related to the texture coordinates of the texture pixel and the ripple motion information and time information of the video frame image corresponding to the texture pixel. For example, the periodic noise value is calculated by combining formulas according to the texture coordinates of the texture pixel and the ripple motion information and time information of the video frame image corresponding to the texture pixel.

[0101] For example, in the embodiment of the present application, the periodic noise value horizontalWaves of the texture pixel can be calculated by the following formula:

[0102] horizontal floatWaves=sin(iTime*(speed+2.0)+sin(uv.x)*53.0);

[0103] Among them, horizontalWaves is the periodic noise value of the texture pixel, iTime is the video frame timestamp parameter corresponding to the video frame image, speed is the fluctuation speed parameter, speed defaults to 6.0, uv represents the texture pixel, which can also be represented by uv0, and uv.x is the coordinate value of the texture pixel on the u axis. 2.0 and 53.0 are empirical parameters, which can be transformed according to actual needs, and sin is a periodic function.

[0104] Among them, the simulated height value of the texture pixel is calculated by using the periodic noise value and the reference point height value of each reference texture point, which adds more influencing factors to the calculation of the simulated height value, adds more details, and increases the randomness of the calculation of the simulated height value.

[0105] Optionally, in an embodiment of the present application, based on the texture coordinates of the texture pixel and the texture coordinates of the reference texture point, the reference point height value of the reference texture point can be calculated by the position offset between the coordinate points, that is, optionally, in some embodiments of the present application, the step of "for the texture pixels in the texture image corresponding to the video frame image, according to the texture coordinates of the texture pixels, the ripple motion information of the video frame image, the time information and the texture coordinates of each of the reference texture points, obtaining the reference point height value of each reference texture point" includes:

[0106] For a texture pixel in a texture image corresponding to a video frame image, obtaining a position offset value between the texture pixel and each reference texture point according to a texture coordinate of the texture pixel and a texture coordinate of each reference texture point;

[0107] According to the ripple motion information and time information of the video frame image and the position offset value of each reference texture point, a reference point height value of each reference texture point is calculated using a periodic function.

[0108] The position offset value between the texture pixel and the reference texture point can be calculated by a distance calculation formula. For example, taking the reference texture point (0.0, 0.0) as an example, the position offset value S between the texture pixel and the reference texture point can be expressed as:

[0109] S=length(uv-vec2(0.0,0.0));

[0110] Among them, uv represents the texture coordinates of the texture pixel, which can also be represented by uv0, vec2(u, v) represents the texture pixel with coordinate value (u, v), the reference texture point corresponding to (0.0, 0.0) represented by the current formula, and length() is a function for calculating the distance between two coordinate points.

[0111] Correspondingly, the reference point height value of the reference texture point can be calculated by the following formula, where the reference texture points are (1.0, 1.0), (1.0, 0.0), (0.0, 1.0) and (0.0, 0.0) for example, where the formula is:

[0112] float topright=sin(iTime*(speed+1.0)-sin(length(uv-vec2(1.0,1.0)))*53.0);

[0113] float topleft=sin(iTime*(speed+1.0)-sin(length(uv-vec2(0.0,1.0)))*53.0);

[0114] Float bottomright=sin(iTime*(speed+1.0)-sin(length(uv-vec2(1.0,0.0)))*53.0);

[0115] float bottomleft=sin(iTime*(speed+1.0)-sin(length(uv-vec2(0.0,0.0)))*53.0);

[0116] Among them, iTime is the video frame timestamp parameter corresponding to the video frame image, speed is the fluctuation speed parameter, uv represents texture pixel, vec2(0.0,1.0) represents the reference texture point, 1.0 and 53.0 are empirical parameters, which can be transformed according to actual needs, sin is a periodic function, topright, topleft, bottomright and bottomleft represent the corner points of the texture image in the upper right corner, upper left corner, lower right corner and lower left corner respectively.

[0117] Optionally, in the embodiment of the present application, in order to make the disturbance effect richer, more details can be added in the process of calculating the simulated height value to further enrich the influencing factors in the process of calculating the simulated height value. For example, the step of "generating the simulated height value of the texture pixel according to the periodic noise value and the reference point height value of each reference texture point" includes:

[0118] Calculating a ripple height noise value of the texture pixel according to the periodic noise value and a reference point height value of each of the reference texture points;

[0119] Generating transition coordinates corresponding to the texture pixel according to the ripple height noise value and the texture coordinates of the texture pixel;

[0120] According to the transition coordinates, the ripple motion information, time information and the texture coordinates of each of the reference texture points, a simulated height value corresponding to the texture coordinates is generated using a periodic function.

[0121] The calculation formula of the simulated height value f can be referred to as follows:

[0122] float f=heightMap(vec2(uv0.x+heightMap(uv0)*0.11,uv0.y));

[0123] Among them, heightMap(uv0) represents the ripple height noise value of texture pixel uv0, and the transition coordinate is represented by (uv0.x+heightMap(uv0)*0.11,uv0.y), wherein uv0.x is the abscissa (u-axis coordinate) of texture pixel uv0, and uv0.y is the ordinate (v-axis coordinate) of texture pixel uv0. The ripple height noise value waveHeight can be calculated by the following formula:

[0124] float temp=horizontalWaves+(bottomleft+bottomright+topleft+topright)*0.4;

[0125] float waveHeight=linearstep(-5.0,5.0,temp)*0.9;

[0126] Among them, the linearstep(min,max,value) function calculates the position of value in the interval [min,max] and returns a linear interpolation function with a range in the interval [0.0,1.0]. temp represents a temporary height value, which may be negative or positive during the calculation process. In order to unify it into a positive number, the linearstep(min,max,value) function is used to transform it. Bottomleft, bottomright, topleft, and topright are the reference point height values ​​of the reference texture points with texture coordinates (1.0,1.0), (1.0,0.0), (0.0,1.0), and (0.0,0.0), respectively. Among them, 0.4 in the temp calculation formula is a scaling factor. The purpose of 0.9 in waveHeight is to obtain a ripple height noise value in the interval [0.0,0.9].

[0127] Among them, by calculating heightMap() twice, more influencing factors are added to the calculation of the simulated height value, thereby improving the accuracy of the calculation of the simulated height value.

[0128] Optionally, in an embodiment of the present application, the target texture coordinates of each texture pixel after twisting can be calculated using the ripple height information, and then the video frame image is sampled based on the target texture coordinates of each texture pixel to obtain a target image with ripples. That is, optionally, in some embodiments of the present application, the step of "processing the pixel coordinate information in the video frame image according to the ripple height information and pixel coordinate information of the video frame image to obtain a target image with ripples" includes:

[0129] Determining a texture resolution of a texture image corresponding to the video frame image and texture coordinates of texture pixels in the texture image according to pixel coordinate information of the video frame image;

[0130] Extracting a simulated height value of the texture pixel from the ripple height information;

[0131] For each texture pixel in the texture image, according to the simulated height value and texture coordinate of the texture pixel and the texture resolution of the texture image, determine the target texture coordinate of the texture pixel after the twisting;

[0132] The video frame image is sampled according to the target texture coordinates corresponding to each texture pixel in the texture image to obtain a target image with ripples.

[0133] It can be understood that texture resolution is the resolution corresponding to the texture image. Texture resolution refers to the number or density of texture pixels in the texture image, usually expressed in horizontal and vertical pixels. Texture resolution is reasonably selected and optimized according to application requirements and device performance.

[0134] By calculating the simulated height value of each texture pixel, the simulated height value and the texture resolution of the texture image can be used to calculate the new texture coordinates of each texture pixel after twisting, that is, the target texture coordinates. By obtaining the new texture coordinates corresponding to the texture pixel, the twisting transformation of the texture pixel is realized.

[0135] The coordinates of the texture pixels in the texture image are obtained by mapping and transforming based on the video frame image, that is, in general, the texture pixels in the texture image correspond to the pixels in the video frame image, and when each texture pixel in the texture image samples the video frame image according to the target texture coordinates, the coordinate position of each pixel in the video frame image changes, achieving the effect of twisting the content of the video frame image. That is, the target image is the image after the content in the video frame image is twisted.

[0136] Optionally, in an embodiment of the present application, since the normal vector of a texture pixel in a three-dimensional model is related to the height value of the texture pixels near the texture pixel, in an embodiment of the present application, the reflection vector corresponding to the current texture pixel can be determined based on the height values ​​of other nearby texture pixels, and the target texture coordinates of the current texture pixel can be obtained based on the reflection vector. That is, optionally, in some embodiments of the present application, the step of "for each texture pixel in the texture image, determining the target texture coordinates of the texture pixel after twisting according to the simulated height value and texture coordinates of the texture pixel and the texture resolution of the texture image" includes:

[0137] For each texture pixel in the texture image, calculating a first direction offset coordinate and a second direction offset coordinate of the texture pixel according to a simulated height value of the texture pixel and a texture resolution of the texture image;

[0138] Generate a first direction offset height value corresponding to the first direction offset coordinate according to the ripple motion information and time information of the video frame image, and generate a second direction offset height value corresponding to the second direction offset coordinate according to the ripple motion information and time information of the video frame image;

[0139] Adjusting the initial normal vector of the texture pixel according to the first direction offset height value and the second direction offset height value to obtain a target normal vector, wherein the initial normal vector is a vector perpendicular to the two-dimensional space where the texture pixel is located;

[0140] Generate a reflection vector corresponding to the target normal vector according to the texture coordinates and the simulated height value;

[0141] A target texture coordinate corresponding to the texture pixel is generated according to the reflection vector.

[0142] Among them, the initial normal vector is the vector corresponding to the texture coordinates of the texture pixel, which is a vector in the two-dimensional space perpendicular to the texture pixel. For example, the initial normal vector of the texture pixel can be expressed as (0.0, 0.0, -1.0) to indicate the direction of the vertical texture image display interface.

[0143] Among them, the offset coordinates in the first direction and the offset coordinates in the second direction can be calculated by the offset step corresponding to the texture image, where the offset step is the normalized uv offset step, which refers to the normalized offset of each pixel in the texture coordinates. In most cases, the range of texture coordinates is from 0 to 1, so the normalized offset step of one pixel is 1 / texture width. If your texture width is w pixels, the normalized offset step of one pixel is 1 / w. Therefore, if you want to offset in pixels, you can multiply the normalized offset step by 2, that is, 2 / w. Therefore, the normalized uv offset step of 2 pixel units is 2 / w.

[0144] Correspondingly, the normalized uv offset step of 2 pixel units can be calculated by the following formula:

[0145] Step = 2. / iResolution.y;

[0146] Among them, iResolution is the texture resolution of the texture image, iResolution.y is the width of the texture image, and accordingly, the offset coordinate eps=vec2(2. / iResolution.y,0.0) can be calculated.

[0147] Correspondingly, the first direction offset coordinate is expressed as:

[0148] vec2 sp_x = uv0 - eps.xy;

[0149] The offset coordinates in the second direction are expressed as:

[0150] vec2 sp_y = uv0 - eps.yx;

[0151] Among them, eps.xy=(2. / iResolution.y,0.0), eps.yx=(0.0,2. / iResolution.y,).

[0152] Among them, the first direction offset height value is the height value corresponding to the first direction offset coordinate, and the second direction offset height value is the height value corresponding to the second direction offset coordinate. Among them, the calculation method of the first direction offset height value and the second direction offset height value is the same as the calculation method of the simulated height value of the texture pixel. Therefore, the calculation process of the simulated height value of the texture pixel can be referred to to understand the calculation of the first direction offset height value and the second direction offset height value. For example, the first direction offset coordinate is the offset coordinate of the texture pixel in the direction of the horizontal coordinate axis, and the second direction offset coordinate is the offset coordinate of the texture pixel in the direction of the vertical coordinate axis. Correspondingly, the first direction offset height value of the first direction offset coordinate can be expressed as:

[0153] float dx=heightMap(vec2(sp_x.x+heightMap(sp_x)*0.11,sp_x.y));

[0154] Correspondingly, the second direction offset height value of the second direction offset coordinate can be expressed as:

[0155] float dy=heightMap(vec2(sp_y.x+heightMap(sp_y)*0.11,sp_y.y));

[0156] It can be understood that the first direction offset coordinate is the first texture coordinate of the texture pixel that is near the texture pixel in the first direction, and the second direction offset coordinate is the second texture coordinate of the texture pixel that is near the texture pixel in the second direction. By determining the first texture coordinate and the second texture coordinate that are near the texture pixel and in two directions, the height values ​​of the first texture coordinate and the second texture coordinate can be used to adjust the initial normal vector of the texture pixel to obtain a target normal vector. Based on the target normal vector, the target normal vector is adjusted in combination with the simulated height value of the texture pixel to obtain a reflection vector. Subsequently, the new texture coordinate of the texture pixel, i.e., the target texture coordinate, can be obtained based on the reflection vector.

[0157] Optionally, in an embodiment of the present application, after obtaining the height values ​​of the first texture coordinate and the second texture coordinate, a height gradient value can be obtained based on the difference between the height values ​​of the first texture coordinate and the second texture coordinate and the simulated height value of the texture pixel, and the initial normal vector of the texture pixel is adjusted based on the height gradient value to obtain a target normal vector, that is, optionally, in some embodiments of the present application, the step of "adjusting the initial normal vector of the texture pixel according to the first direction offset height value and the second direction offset height value to obtain a target normal vector" includes:

[0158] Obtaining a first direction height increment value according to the first direction offset height value and the simulated height value, and obtaining a second direction height increment value according to the second direction offset height value and the simulated height value;

[0159] Calculate a first-direction height gradient value corresponding to the first-direction height increment value according to an offset step length between two texture pixels corresponding to a texture resolution of the texture image, and calculate a second-direction height gradient value corresponding to the second-direction height increment value according to the offset step length;

[0160] The initial normal vector of the texture pixel is adjusted according to the first direction height gradient value and the second direction height gradient value to obtain a target normal vector.

[0161] It can be understood that the first direction height increment value is the increment value of the first direction offset height value compared to the simulated height value of the texture pixel, and the second direction height increment value is the increment value of the second direction offset height value compared to the simulated height value of the texture pixel.

[0162] For example, it is calculated by the following formula:

[0163] The height increment value in the first direction is: dx-f, and the height increment value in the second direction is: dy-f.

[0164] Wherein, dx represents the offset height value in the first direction, and dy represents the offset height value in the second direction.

[0165] Correspondingly, the height gradient values ​​of the texture pixels in the two directions can be calculated based on the height increment values ​​in the two directions and the offset step size.

[0166] For example, it is calculated by the following formula:

[0167] float fx=(dx-f) / eps.x;

[0168] float fy=(dy-f) / eps.x;

[0169] Among them, fx represents the height gradient value in the first direction, fy represents the height gradient value in the second direction, and eps.x is 2. / iResolution.y.

[0170] Correspondingly, after obtaining the first direction height gradient value fx and the second direction height gradient value fy, the initial normal vector of the texture pixel can be adjusted to obtain the true normal vector of the texture pixel, that is, the target normal vector. Correspondingly, the calculation formula of the target normal vector Kn is as follows:

[0171] vec3 Kn=normalize(sn+vec3(fx*0.004,fy*0.004,0.));

[0172] Wherein, sn is the initial normal vector of the texture pixel. In the embodiment of the present application, its value is (0.0, 0.0, -1.0), fx represents the height gradient value in the first direction, fy represents the height gradient value in the second direction, and the normalize function is used to normalize the calculation result.

[0173] Accordingly, in the embodiment of the present application, the reflection vector corresponding to the target normal vector is generated according to the texture coordinates and the simulated height value, which can be achieved by the following calculation formula, specifically:

[0174] vec3 ref=reflect(vec3(uv0,f),Kn);

[0175] Among them, ref represents the reflection vector of the texture pixel, uv0 is the texture pixel, f is the simulated height value of the texture pixel, and Kn is the target normal vector of the texture pixel.

[0176] Among them, the reflect function is a commonly used vector operation function, which is used to calculate the reflection vector of a vector relative to a normal vector. The reflect function accepts two parameters: an incident vector and a normal vector. The incident vector represents the direction of the light or particle on the collision surface, while the normal vector represents the normal direction of the surface. Both vectors need to be unit vectors (that is, the length is 1).

[0177] The calculation formula of the reflect function is:

[0178] reflect(I,N)=I-2*dot(I,N)*N;

[0179] Where I is the incident vector, N is the normal vector, and dot(I,N) represents the dot product of the two vectors.

[0180] The reflection vector represents the direction in which the light or particle is reflected from the collision surface. It is obtained by reflecting the incident vector along the mirror direction of the normal vector. The dot product multiplication and subtraction operations cause the projection of the incident vector on the normal vector to be flipped and remain on the normal vector plane.

[0181] Correspondingly, in the embodiment of the present application, vec3(uv0,f) represents the vector of the reflection vector to be calculated, also called the incident vector, Kn is the true normal vector of the texture pixel, that is, the target normal vector, and through reflect(vec3(uv0,f),Kn), the reflection vector ref of the incident vector vec3(uv0,f) compared to the target normal vector Kn is calculated.

[0182] Optionally, in an embodiment of the present application, after obtaining the reflection vector of the texture pixel, the texture pixel may be sampled based on the reflection vector, that is, optionally, in some embodiments of the present application, the step of “generating a target texture coordinate corresponding to the texture pixel according to the reflection vector” includes:

[0183] Setting the abscissa of the reflection vector to the target abscissa;

[0184] Setting the ordinate of the reflection vector as the target ordinate;

[0185] The texture coordinates corresponding to the target abscissa and the target ordinate are set as the target texture coordinates of the texture pixel.

[0186] That is, the abscissa and ordinate of the reflection vector are used as new texture coordinates of the texture pixel, namely, target texture coordinates, and then the video frame image is sampled based on the target texture coordinates.

[0187] The sampling of the video frame image can refer to the following formula, specifically:

[0188] vec4 outputColor=texture2D(uTexture0,ref.xy);

[0189] Among them, uTexture0 is the video frame image, outputColor is the target image, ref.xy is the x and y axis coordinates of the reflection vector ref, and the texture2D function is the texture sampling function.

[0190] In summary, the embodiments of the present application define a simulated height value of a texture pixel, and use the simulated height value and the offset step size of the texture image to calculate the true normal vector of the texture pixel, and obtain the reflection vector of the texture pixel by calculating the simulated height value and the texture coordinates of the texture pixel, thereby realizing the calculation of new texture coordinates of the texture pixel, sampling the video frame image through the new texture coordinates, obtaining a target image after the image content is distorted, and obtaining a target video by combining the target images corresponding to each video frame image.

[0191] Among them, through the ripple motion information and time information of each video frame image, the processed target image presents a ripple flow effect after rendering, thereby realizing the generation of new stylized videos, enriching the stylized processing method of videos, and improving user experience.

[0192] See also Figure 3 and Figure 4 As shown, Figure 3 is a video frame image provided by an embodiment of the present application, Figure 4 The embodiment of the present application provides Figure 3 The target image is obtained by twisting the content of the video frame image. The stylized image is generated.

[0193] Correspondingly, in an embodiment of the present application, ripple motion information, time information, and related influencing factors and other parameters can be provided to the user, so that the user can generate the desired style image by adjusting the ripple motion information, time information, influencing factors and other parameters, and thus obtain a stylized video.

[0194] Accordingly, in the embodiment of the present application, the video generation method can be integrated into a video generation module, which outputs a stylized target image by receiving an input video frame image. Figure 5 As shown, Figure 5: is an architecture diagram of a video generation system provided in an embodiment of the present application. Specifically, the video generation system includes:

[0195] An input module 201 is used to input an image to be processed into a video generation module;

[0196] For example, the image to be processed may be a video frame image in an embodiment of the present application.

[0197] The video generation module 202 is used to receive the image to be processed input by the input module 201, perform content twisting processing on the image to be processed to obtain a target image, and output the target image to the output module 203;

[0198] The output module 203 is used to output the target image.

[0199] Correspondingly, after the target image is output, a target video can be obtained based on a combination of multiple target images.

[0200] Accordingly, the user only needs to input the video or image to be processed through the input module to obtain the stylized video or image. Accordingly, the input module and the output module can be configured as corresponding visualization interfaces so that the user can preview the input and output through the visualization interface.

[0201] In order to better implement the video generation method of the present application, the present application also provides a video generation device based on the above video generation method. The meanings of the terms are the same as those in the above video generation method, and the specific implementation details can refer to the description in the method embodiment.

[0202] See also Figure 6 , Figure 6 : is a schematic diagram of the structure of a video generating device provided in an embodiment of the present application, wherein the video generating device may be specifically as follows:

[0203] An acquisition module 301 is used to acquire a video to be processed and ripple motion information of a video frame image in the video to be processed, wherein the video to be processed includes at least two video frame images;

[0204] A determination module 302, configured to determine the ripple height information of the video frame image according to the ripple motion information, time information and pixel coordinate information of the video frame image;

[0205] The processing module 303 is used to process the pixel coordinate information in the video frame image according to the ripple height information and the pixel coordinate information of the video frame image to obtain a target image with ripples;

[0206] The generating module 304 is used to generate a target video according to each of the target images.

[0207] Optionally, in some embodiments of the present application, the determining module 302 includes:

[0208] A first determining unit, configured to determine at least one reference texture point from a texture image corresponding to the video frame image;

[0209] A first acquisition unit is used to obtain, for a texture pixel in a texture image corresponding to the video frame image, a reference point height value of each reference texture point according to texture coordinates of the texture pixel, ripple motion information of the video frame image, time information, and texture coordinates of each reference texture point;

[0210] A second acquisition unit, configured to obtain a periodic noise value of the texture pixel using a periodic function according to the texture coordinates of the texture pixel and the ripple motion information and time information of the video frame image;

[0211] A first generating unit, configured to generate a simulated height value of the texture pixel according to the periodic noise value and a reference point height value of each of the reference texture points;

[0212] The second generating unit is used to generate the ripple height information of the video frame image according to the simulated height value of each texture pixel in the texture image.

[0213] In some embodiments of the present application, the first generating unit includes:

[0214] A first calculation subunit, configured to calculate a ripple height noise value of the texture pixel according to the periodic noise value and a reference point height value of each reference texture point;

[0215] A second calculation subunit, configured to generate a transition coordinate corresponding to the texture pixel according to the ripple height noise value and the texture coordinate of the texture pixel;

[0216] The third calculation subunit is used to generate a simulated height value corresponding to the texture coordinate using a periodic function according to the transition coordinate, the ripple motion information of the video frame image, time information and the texture coordinates of each reference texture point.

[0217] In some embodiments of the present application, the first acquisition unit includes:

[0218] A first acquisition subunit is used to obtain, for a texture pixel in a texture image corresponding to a video frame image, a position offset value between the texture pixel and each reference texture point according to a texture coordinate of the texture pixel and a texture coordinate of each reference texture point;

[0219] The second acquisition subunit is used to calculate the reference point height value of each reference texture point by using a periodic function according to the ripple motion information, time information and position offset value of each reference texture point of the video frame image.

[0220] In some embodiments of the present application, the processing module 303 includes:

[0221] A second determining unit, configured to determine a texture resolution of a texture image corresponding to the video frame image and texture coordinates of texture pixels in the texture image according to pixel coordinate information of the video frame image;

[0222] An extraction unit, used for extracting a simulated height value of the texture pixel from the ripple height information;

[0223] a third determining unit, configured to determine, for each texture pixel in the texture image, a target texture coordinate of the texture pixel after twisting according to a simulated height value and a texture coordinate of the texture pixel and a texture resolution of the texture image;

[0224] The sampling unit is used to sample the video frame image according to the target texture coordinates corresponding to each texture pixel in the texture image to obtain a target image with ripples.

[0225] In some embodiments of the present application, the third determining unit includes:

[0226] A coordinate calculation subunit, configured to calculate, for each texture pixel in the texture image, a first direction offset coordinate and a second direction offset coordinate of the texture pixel according to a simulated height value of the texture pixel and a texture resolution of the texture image;

[0227] an offset height calculation subunit, configured to generate a first direction offset height value corresponding to the first direction offset coordinate according to the ripple motion information and time information of the video frame image, and to generate a second direction offset height value corresponding to the second direction offset coordinate according to the ripple motion information and time information of the video frame image;

[0228] a vector adjustment subunit, configured to adjust an initial normal vector of the texture pixel according to the first direction offset coordinate and the second direction offset coordinate to obtain a target normal vector, wherein the initial normal vector is a vector perpendicular to the two-dimensional space where the texture pixel is located;

[0229] A vector generating subunit, used for generating a reflection vector corresponding to the target normal vector according to the texture coordinates and the simulated height value;

[0230] The coordinate generation subunit is used to generate the target texture coordinates corresponding to the texture pixel according to the reflection vector.

[0231] In some embodiments of the present application, the vector adjustment subunit is specifically used for:

[0232] Obtaining a first direction height increment value according to the first direction offset height value and the simulated height value, and obtaining a second direction height increment value according to the second direction offset height value and the simulated height value;

[0233] Calculate a first-direction height gradient value corresponding to the first-direction height increment value according to an offset step length between two texture pixels corresponding to a texture resolution of the texture image, and calculate a second-direction height gradient value corresponding to the second-direction height increment value according to the offset step length;

[0234] The initial normal vector of the texture pixel is adjusted according to the first direction height gradient value and the second direction height gradient value to obtain a target normal vector.

[0235] In some embodiments of the present application, the coordinate generation subunit is specifically used for:

[0236] Setting the abscissa of the reflection vector to the target abscissa;

[0237] Setting the ordinate of the reflection vector as the target ordinate;

[0238] The texture coordinates corresponding to the target abscissa and the target ordinate are set as the target texture coordinates of the texture pixel.

[0239] In the embodiment of the present application, the acquisition module 301 first acquires the video to be processed and the ripple motion information of the video frame image in the video to be processed, wherein the video to be processed includes at least two video frame images. Then, the determination module 302 determines the ripple height information of the video frame image according to the ripple motion information, time information and pixel coordinate information of the video frame image. Subsequently, the processing module 303 processes the pixel coordinate information in the video frame image according to the ripple height information and pixel coordinate information of the video frame image to obtain a target image with ripples. Then, the generation module 304 generates a target video according to each of the target images.

[0240] In summary, the embodiments of the present application calculate the ripple height information of the video frame image through the ripple motion information, time information and pixel coordinate information corresponding to the video frame image, and use the ripple height information and the pixel coordinate information of the video frame image to process the pixel coordinate information of the video frame image to obtain a target image with ripples, and then generate a target video through the target images corresponding to each video frame image.

[0241] By processing pixel coordinate information in a video frame image, a target image with ripples is obtained, and stylized processing of the video frame image is achieved, so that the video frame image is transformed into a stylized image with twisted content.

[0242] Among them, each video frame image is stylized according to its corresponding ripple motion information and time information, so that the target video presents a ripple flow visual effect based on each stylized image, enriching the stylized processing method of the video and improving the user experience.

[0243] In addition, the present application also provides an electronic device, such as Figure 7 As shown, it shows a schematic diagram of the structure of the electronic device involved in this application, specifically:

[0244] The electronic device may include components such as a processor 401 with one or more processing cores, a memory 402 with one or more computer-readable storage media, a power supply 403, and an input unit 404. Those skilled in the art will appreciate that Figure 7 The electronic device structure shown in the figure does not constitute a limitation on the electronic device, and may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently.

[0245] The processor 401 is the control center of the electronic device, and uses various interfaces and lines to connect various parts of the entire electronic device. By running or executing software programs and / or modules stored in the memory 402, and calling data stored in the memory 402, the processor 401 performs various functions of the electronic device and processes data, thereby monitoring the electronic device as a whole. Optionally, the processor 401 may include one or more processing cores; preferably, the processor 401 may integrate an application processor and a modem processor, wherein the application processor mainly processes the operating system, user interface, and application programs, and the modem processor mainly processes wireless communications. It is understandable that the above-mentioned modem processor may not be integrated into the processor 401.

[0246] The memory 402 can be used to store software programs and modules. The processor 401 executes various functional applications and data processing by running the software programs and modules stored in the memory 402. The memory 402 may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc.; the data storage area may store data created according to the use of the electronic device, etc. In addition, the memory 402 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other volatile solid-state storage devices. Accordingly, the memory 402 may also include a memory controller to provide the processor 401 with access to the memory 402.

[0247] The electronic device also includes a power supply 403 for supplying power to each component. Preferably, the power supply 403 can be logically connected to the processor 401 through a power management system, so as to manage charging, discharging, and power consumption through the power management system. The power supply 403 can also include one or more DC or AC power supplies, recharging systems, power supply device debugging circuits, power converters or inverters, power status indicators, and other arbitrary components.

[0248] The electronic device may further include an input unit 404, which may be used to receive input digital or character information and generate keyboard, mouse, joystick, optical or trackball signal input related to user settings and function control.

[0249] Although not shown, the electronic device may further include a display unit, etc., which will not be described in detail herein. Specifically in this embodiment, the processor 401 in the electronic device will load the executable files corresponding to the processes of one or more application programs into the memory 402 according to the following instructions, and the processor 401 will run the application programs stored in the memory 402, thereby implementing the steps in any one of the video generation methods provided in the embodiments of the present application.

[0250] An embodiment of the present application obtains a video to be processed and ripple motion information of a video frame image in the video to be processed, wherein the video to be processed includes at least two video frame images, and determines ripple height information of the video frame image based on the ripple motion information, time information, and pixel coordinate information of the video frame image; based on the ripple height information and pixel coordinate information of the video frame image, processes the pixel coordinate information in the video frame image to obtain a target image with ripples, and generates a target video based on each of the target images.

[0251] In summary, the embodiments of the present application calculate the ripple height information of the video frame image through the ripple motion information, time information and pixel coordinate information corresponding to the video frame image, and use the ripple height information and the pixel coordinate information of the video frame image to process the pixel coordinate information of the video frame image to obtain a target image with ripples, and then generate a target video through the target images corresponding to each video frame image.

[0252] By processing pixel coordinate information in a video frame image, a target image with ripples is obtained, and stylized processing of the video frame image is achieved, so that the video frame image is transformed into a stylized image with twisted content.

[0253] Among them, each video frame image is stylized according to its corresponding ripple motion information and time information, so that the target video presents a ripple flow visual effect based on each stylized image, enriching the stylized processing method of the video and improving the user experience.

[0254] The specific implementation of the above operations can be found in the previous embodiments, which will not be described in detail here.

[0255] A person of ordinary skill in the art will appreciate that all or part of the steps in the various methods of the above embodiments may be completed by instructions, or by controlling related hardware through instructions. The instructions may be stored in a computer-readable storage medium and loaded and executed by a processor.

[0256] To this end, the present application provides a computer-readable storage medium, on which a computer program is stored. The computer program can be loaded by a processor to execute the steps in any video generation method provided in the present application.

[0257] The specific implementation of the above operations can be found in the previous embodiments, which will not be described in detail here.

[0258] The computer-readable storage medium may include: a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, etc.

[0259] Since the instructions stored in the computer-readable storage medium can execute the steps in any video generation method provided in the present application, the beneficial effects that can be achieved by any video generation method provided in the present application can be achieved. Please refer to the previous embodiments for details and will not be repeated here.

[0260] The above is a detailed introduction to a video generation method, device, electronic device and computer-readable storage medium provided by the present application. Specific examples are used herein to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core idea. At the same time, for those skilled in the art, according to the idea of ​​the present invention, there will be changes in the specific implementation method and application scope. In summary, the content of this specification should not be understood as a limitation on the present invention.

[0261] It should be noted that in the specific implementation of the present application, it involves data related to the video, image pixels, texture pixels, coordinates, etc. to be processed. When the above embodiments of the present application are applied to specific products or technologies, user permission or consent is required, and the collection, use and processing of relevant data need to comply with relevant laws, regulations and standards of relevant countries and regions.

Claims

1. A video generation method, characterized in that: The method comprises: Acquire a video to be processed and ripple motion information of a video frame image in the video to be processed, wherein the video to be processed includes at least two video frame images; Determine at least one reference texture point from the texture image corresponding to the video frame image; For texture pixels in the texture image corresponding to the video frame image, obtaining a reference point height value of each reference texture point according to texture coordinates of the texture pixels, ripple motion information of the video frame image, time information, and texture coordinates of each reference texture point; Obtaining a periodic noise value of the texture pixel using a periodic function according to the texture coordinates of the texture pixel and the ripple motion information and time information of the video frame image; Generating a simulated height value of the texture pixel according to the periodic noise value and the reference point height value of each of the reference texture points; Generate ripple height information of the video frame image according to the simulated height value of each texture pixel in the texture image; According to the ripple height information and pixel coordinate information of the video frame image, the pixel coordinate information in the video frame image is processed to obtain a target image with ripples; A target video is generated according to each of the target images.

2. The video generation method according to claim 1, characterized in that: The step of generating the simulated height value of the texture pixel according to the periodic noise value and the reference point height value of each reference texture point comprises: Calculating a ripple height noise value of the texture pixel according to the periodic noise value and a reference point height value of each of the reference texture points; Generating transition coordinates corresponding to the texture pixel according to the ripple height noise value and the texture coordinates of the texture pixel; According to the transition coordinates, the ripple motion information of the video frame image, the time information and the texture coordinates of each of the reference texture points, a simulated height value corresponding to the texture coordinates is generated using a periodic function.

3. The video generation method according to claim 1, characterized in that: The step of obtaining the reference point height value of each reference texture point according to the texture coordinates of the texture pixel in the texture image corresponding to the video frame image, the ripple motion information of the video frame image, the time information and the texture coordinates of each reference texture point comprises: For a texture pixel in a texture image corresponding to a video frame image, obtaining a position offset value between the texture pixel and each reference texture point according to a texture coordinate of the texture pixel and a texture coordinate of each reference texture point; According to the ripple motion information and time information of the video frame image and the position offset value of each reference texture point, a reference point height value of each reference texture point is calculated using a periodic function.

4. The video generation method according to claim 1, characterized in that: The step of processing the pixel coordinate information in the video frame image according to the ripple height information and the pixel coordinate information of the video frame image to obtain a target image with ripples includes: Determining a texture resolution of a texture image corresponding to the video frame image and texture coordinates of texture pixels in the texture image according to pixel coordinate information of the video frame image; Extracting a simulated height value of the texture pixel from the ripple height information; For each texture pixel in the texture image, according to the simulated height value and texture coordinate of the texture pixel and the texture resolution of the texture image, determine the target texture coordinate of the texture pixel after the twisting; The video frame image is sampled according to the target texture coordinates corresponding to each texture pixel in the texture image to obtain a target image with ripples.

5. The video generation method according to claim 4, characterized in that: The step of determining, for each texture pixel in the texture image, a target texture coordinate of the texture pixel after twisting according to a simulated height value and a texture coordinate of the texture pixel and a texture resolution of the texture image, comprises: For each texture pixel in the texture image, calculating a first direction offset coordinate and a second direction offset coordinate of the texture pixel according to a simulated height value of the texture pixel and a texture resolution of the texture image; Generate a first direction offset height value corresponding to the first direction offset coordinate according to the ripple motion information and time information of the video frame image, and generate a second direction offset height value corresponding to the second direction offset coordinate according to the ripple motion information and time information of the video frame image; Adjusting the initial normal vector of the texture pixel according to the first direction offset height value and the second direction offset height value to obtain a target normal vector, wherein the initial normal vector is a vector perpendicular to the two-dimensional space where the texture pixel is located; Generate a reflection vector corresponding to the target normal vector according to the texture coordinates and the simulated height value; A target texture coordinate corresponding to the texture pixel is generated according to the reflection vector.

6. The video generation method according to claim 5, characterized in that: The step of adjusting the initial normal vector of the texture pixel according to the first direction offset height value and the second direction offset height value to obtain a target normal vector includes: Obtaining a first direction height increment value according to the first direction offset height value and the simulated height value, and obtaining a second direction height increment value according to the second direction offset height value and the simulated height value; Calculate a first-direction height gradient value corresponding to the first-direction height increment value according to an offset step length between two texture pixels corresponding to a texture resolution of the texture image, and calculate a second-direction height gradient value corresponding to the second-direction height increment value according to the offset step length; The initial normal vector of the texture pixel is adjusted according to the first direction height gradient value and the second direction height gradient value to obtain a target normal vector.

7. The video generation method according to claim 5, characterized in that: Generating the target texture coordinates corresponding to the texture pixel according to the reflection vector includes: Setting the abscissa of the reflection vector to the target abscissa; Setting the ordinate of the reflection vector as the target ordinate; The texture coordinates corresponding to the target abscissa and the target ordinate are set as the target texture coordinates of the texture pixel.

8. A video generating device, characterized in that: The device comprises: An acquisition module, used to acquire a video to be processed and ripple motion information of a video frame image in the video to be processed, wherein the video to be processed includes at least two video frame images; A determination module, used to determine at least one reference texture point from the texture image corresponding to the video frame image; for texture pixels in the texture image corresponding to the video frame image, obtain a reference point height value of each reference texture point according to the texture coordinates of the texture pixel, the ripple motion information and time information of the video frame image, and the texture coordinates of each reference texture point; obtain a periodic noise value of the texture pixel using a periodic function according to the texture coordinates of the texture pixel and the ripple motion information and time information of the video frame image; generate a simulated height value of the texture pixel according to the periodic noise value and the reference point height value of each reference texture point; generate the ripple height information of the video frame image according to the simulated height value of each texture pixel in the texture image; A processing module, used for processing the pixel coordinate information in the video frame image according to the ripple height information and the pixel coordinate information of the video frame image to obtain a target image with ripples; A generating module is used to generate a target video according to each of the target images.

9. An electronic device, characterized in that: The method comprises a memory, a processor and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps in the video generation method according to any one of claims 1 to 7 when executing the computer program.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps in the video generation method according to any one of claims 1 to 7 are implemented.