Dynamic effect processing method and device of video frame, storage medium and electronic equipment

Abstract

The invention discloses a dynamic effect processing method and device of a video frame, a storage medium and electronic equipment. The method comprises the steps that a video frame to be processed is acquired, and the video frame is associated with a corresponding dynamic effect element; first dynamic effect description information of the video frame is read, the first dynamic effect description information is used for indicating a first display effect of the dynamic effect element, and the first dynamic effect description information comprises first texture information of the video frame; in response to the dynamic editing request, updating the first texture information to obtain second texture information of the dynamic effect element; based on the second texture information, second dynamic effect description information of the video frame is generated, and the second dynamic effect description information is used for indicating a second display effect of the dynamic effect element; and generating a dynamic effect layer corresponding to the dynamic effect element based on the second dynamic effect description information. The technical problem that the dynamic effect display effect is poor is solved.

Description

Technical Field

[0001] The present application relates to the field of computers, and more specifically, to a method and device for processing motion effects of video frames, a storage medium, and an electronic device. Background Art

[0002] Currently, the display effects of motion elements of media resources are limited by predefined resources and cannot be dynamically edited and adjusted according to specific scenarios during playback. They cannot meet the needs of rapidly changing application scenarios with higher interactivity requirements, which reduces the reusability and adaptability of motion effects and leads to poor display effects.

[0003] Therefore, there is a technical problem in the related art that the display effect of the dynamic effect is poor. Summary of the invention

[0004] The embodiments of the present application provide a method and device for processing motion effects of video frames, a storage medium, and an electronic device, so as to at least solve the technical problem of poor display effect of motion effects in related technologies.

[0005] According to one aspect of an embodiment of the present application, a method for processing motion effects of a video frame is provided, comprising: obtaining a video frame to be processed, wherein the video frame is associated with a corresponding motion effect element; reading first motion effect description information of the video frame, wherein the first motion effect description information is used to indicate a first display effect of the motion effect element, and the first motion effect description information includes first texture information of the video frame; in response to a dynamic editing request, updating the first texture information to obtain second texture information of the motion effect element; based on the second texture information, generating second motion effect description information of the video frame, wherein the second motion effect description information is used to indicate a second display effect of the motion effect element; based on the second motion effect description information, generating a motion effect layer corresponding to the motion effect element.

[0006] According to another aspect of the embodiment of the present application, a video frame motion effect processing device is also provided, including:

[0007] An acquisition unit, used to acquire a video frame to be processed, wherein the video frame is associated with a corresponding motion effect element;

[0008] A reading unit is used to read first motion effect description information of a video frame, wherein the first motion effect description information is used to indicate a first display effect of a motion effect element, and the first motion effect description information includes first texture information of the video frame; an updating unit is used to update the first texture information in response to a dynamic editing request to obtain second texture information of the motion effect element; a first generating unit is used to generate second motion effect description information of the video frame based on the second texture information, wherein the second motion effect description information is used to indicate a second display effect of the motion effect element; a second generating unit is used to generate a motion effect layer corresponding to the motion effect element based on the second motion effect description information.

[0009] According to another aspect of the embodiments of the present application, a computer program product is provided, the computer program product including a computer program / instruction, the computer instruction being stored in a computer-readable storage medium. A processor of a computer device reads the computer program / instruction from the computer-readable storage medium, and the processor executes the computer program / instruction, so that the computer device executes the above video frame motion effect processing method.

[0010] According to another aspect of the embodiments of the present application, there is also provided an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the above-mentioned method for processing motion effects of video frames through the computer program.

[0011] In this embodiment, a video frame to be processed is obtained, wherein the video frame is associated with a corresponding motion effect element; the first motion effect description information of the video frame is read, wherein the first motion effect description information is used to indicate a first display effect of the motion effect element, and the first motion effect description information includes first texture information of the video frame; in response to a dynamic editing request, the first texture information is updated to obtain second texture information of the motion effect element; based on the second texture information, second motion effect description information of the video frame is generated, wherein the second motion effect description information is used to indicate a second display effect of the motion effect element; based on the second motion effect description information, a motion effect layer corresponding to the motion effect element is generated. By dynamically updating the texture information and motion effect description information of the motion effect element, a motion effect layer matching the editing request is generated, thereby achieving real-time editing and adjustment of the display effect of the motion effect element, solving the problem in the prior art that the display effect of the motion effect element is fixed and cannot be flexibly adjusted and optimized according to the specific scene, thereby significantly improving the reusability, personalized display capability and scene adaptability of the motion effect, and achieving the technical effect of improving the display effect of the motion effect. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0013] Figure 1 is a schematic diagram of a process of an optional method for processing motion effects of video frames according to an embodiment of the present application;

[0014] Figure 2 is a schematic diagram of an optional method for processing motion effects of video frames according to an embodiment of the present application;

[0015] Figure 3 is a schematic diagram of an optional method for processing motion effects of video frames according to an embodiment of the present application;

[0016] Figure 4 is a schematic diagram of an optional method for processing motion effects of video frames according to an embodiment of the present application;

[0017] Figure 5 is a schematic diagram of an optional method for processing motion effects of video frames according to an embodiment of the present application;

[0018] Figure 6 is a schematic diagram of an optional video frame motion effect processing device according to an embodiment of the present application;

[0019] Figure 7 A schematic structural diagram of an optional electronic device according to an embodiment of the present application. DETAILED DESCRIPTION

[0020] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present application.

[0021] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0022] Optionally, as an optional implementation, as Figure 1 As shown, the method for processing the motion effect of a video frame includes the following specific steps:

[0023] S102, obtaining a video frame to be processed, wherein the video frame is associated with a corresponding motion effect element;

[0024] S104, reading first motion effect description information of the video frame, wherein the first motion effect description information is used to indicate a first display effect of the motion effect element, and the first motion effect description information includes first texture information of the video frame;

[0025] S106, in response to the dynamic editing request, updating the first texture information to obtain second texture information of the dynamic effect element;

[0026] S108, generating second motion effect description information of the video frame based on the second texture information, wherein the second motion effect description information is used to indicate a second display effect of the motion effect element;

[0027] S110: Generate a motion effect layer corresponding to the motion effect element based on the second motion effect description information.

[0028] Optionally, in this embodiment, a video frame is the most basic unit constituting a video, and each frame is a still image. Animation elements in a video frame refer to animation components that can change independently or interact with a user, such as text, icons, graphics, etc., and have their own dynamic properties and behaviors.

[0029] Optionally, in this embodiment, the first motion effect description information refers to the native information describing the initial state and change trajectory of the motion effect element stored in the video frame metadata, including the texture information, position, size, rotation angle, etc. of the motion effect element.

[0030] Optionally, in this embodiment, the first texture information is the surface attribute of the motion effect element, such as color, pattern, image, etc., which is a key factor constituting the appearance of the motion effect element.

[0031] Optionally, in this embodiment, the dynamic editing request is a signal from application logic or user interaction during the playback of the media file, which is used to indicate that the display effect of the motion element needs to be modified in real time.

[0032] Optionally, in this embodiment, the second texture information is new surface attribute information of the dynamic effect element in response to the dynamic editing request, reflecting the appearance and visual effect after editing.

[0033] It can be understood that the second motion effect description information is the updated motion effect element description information, which not only includes the modified texture information, but may also include parameters such as new position, size, rotation angle, etc., which are used to indicate the display effect after editing.

[0034] Optionally, in this embodiment, the display effects and behaviors of the motion effect elements are encapsulated into an independent layer, which can be synthesized with other parts of the video frame to achieve dynamic editing and interactive response.

[0035] Optionally, in this embodiment, first, a video frame is obtained, which contains motion effect elements. These elements may have initial texture information and display effects, and this information is stored in the first motion effect description information.

[0036] Then, when a dynamic editing request is received, that is, the display effect of the motion effect element needs to be changed, the system will update the first texture information and generate new second texture information to reflect the change requirements of the motion effect element, such as adjustment of texture style, color or transparency.

[0037] Next, based on the updated second texture information, second motion effect description information is generated, which not only includes the update of texture information, but may also involve the adjustment of the position, size and rotation angle of the motion effect element to ensure that the display effect of the motion effect element in the video frame is consistent with the editing request.

[0038] Finally, the second motion effect description information is used to generate a motion effect layer, which contains all the visual information and dynamic properties of the edited motion effect element and can be synthesized with other parts of the video frame to achieve real-time updating and display of the editing effect.

[0039] Through the above method, the display effect of the motion effect elements can be adjusted in real time during video playback to meet the needs of dynamic editing, improve the reusability and adaptability of the motion effect, and enhance the interactive experience between users and the motion effect elements. It overcomes the limitations of traditional motion effect processing methods that the motion effect elements are fixed and cannot be adjusted in real time, and provides a more flexible and powerful motion effect processing method for video applications, especially in transparent background processing, 3D effect presentation and performance optimization. It shows significant advantages.

[0040] As an optional solution, based on the second animation description information, generating an animation layer corresponding to the animation element includes:

[0041] Acquire size and position information of the motion effect element in the video frame from the second motion effect description information, and generate a first element layer of the motion effect element according to the size and position information;

[0042] Merge the first element layer with the original element layer of the motion effect element in the video frame to obtain a second element layer of the motion effect element;

[0043] Determine the second element layer as the animation layer.

[0044] Optionally, in this embodiment, the first element layer is a layer of the updated motion effect element generated based on the size and position information in the second motion effect description information, reflecting the impact of the dynamic editing request on the appearance and position of the motion effect element.

[0045] Optionally, in this embodiment, the original element layer is a motion effect element layer generated according to the first motion effect description information of the video frame, reflecting the initial state and display effect of the motion effect element in the video frame.

[0046] Optionally, in this embodiment, the second element layer is a layer obtained by merging the first element layer and the original element layer, contains all original and edited display effect information, and is the final form of the motion effect layer.

[0047] Optionally, in this embodiment, first, the size and position information of the motion effect element is read from the second motion effect description information, which is the effect parameter after the dynamic editing request is applied, and is used to generate the first element layer. The first element layer intuitively shows the appearance and position changes of the motion effect element after editing, such as changing the visual effect of the element by changing the color, size or transparency, and changing its layout in the video frame by adjusting the position and rotation.

[0048] Next, merge the generated first element layer with the original element layer of the motion effect element in the video frame to generate the second element layer. The original element layer contains the initial state of the motion effect element in the video frame, while the first element layer contains the changes after editing. Through the merging operation, the second element layer can fully reflect all display effect information of the motion effect element from the initial to the edited state, ensuring the visibility and integrity of the editing effect during video playback.

[0049] Finally, the second element layer is determined to be the dynamic effect layer, which contains all display effect information of the dynamic effect element after being updated according to the dynamic editing request, such as color, size, position, transparency, rotation, etc. The dynamic effect layer can be synthesized with the background layer of the video frame, other dynamic effect layers, and any static element layer to achieve real-time display and interactive response of the editing effect.

[0050] Through the embodiments provided in the present application, not only can the dynamic editing requests be responded to in real time and the motion effect elements in the video frames be accurately updated, but it can also ensure that the edited motion effect elements can be seamlessly integrated with the video background and other elements, thereby improving the visual effect and user experience.

[0051] As an optional solution, generating second motion effect description information of the video frame based on the second texture information includes:

[0052] Obtaining transformation matrix information, wherein the transformation matrix information is used to indicate position information of the motion effect element in the video frame, and the first motion effect description information includes the transformation matrix information;

[0053] Using the transformation matrix information, a matrix operation is performed on the second texture information to obtain second motion effect description information of the video frame.

[0054] Optionally, in this embodiment, the transformation matrix information is used to describe transformation operations such as position, rotation and translation of the motion effect element in the video frame, and is key data for realizing dynamic editing and position adjustment of the motion effect element.

[0055] Optionally, in this embodiment, first, by reading the first motion effect description information, the transformation matrix information of the motion effect element is obtained, which includes the initial position of the element in the video frame and the transformation parameters required for any editing operation. As a mathematical representation, the transformation matrix can concisely and accurately describe the position, rotation, translation and other operations of the motion effect element, and is the core data for dynamic editing and position adjustment.

[0056] Then, the acquired transformation matrix information is used to perform matrix operations on the second texture information. The second texture information is the surface property of the edited motion effect element, which contains the modified color, pattern and other information. Through matrix operations, the system can calculate the specific position of the edited motion effect element in the video frame, including any displacement, rotation or scaling transformations. This step ensures that the display effect of the motion effect element not only reflects the changes in appearance, but also accurately locates the new position of the element in the video frame.

[0057] Finally, the calculation results are integrated into the second animation description information, which contains the complete edited state of the animation element, including the transformed second texture information and the new position parameters. The generation of the second animation description information not only supports the dynamic editing of the animation elements, but also ensures the accurate correspondence between the editing effect and the position information of the animation elements in the video frame, providing a key basis for the generation of subsequent animation element layers, thereby realizing real-time adjustment and optimization of the animation, and improving the reusability and visual experience of the animation.

[0058] As an optional solution, obtain the transformation matrix information, including:

[0059] Obtain transformation matrix information from metadata of a media file where the video frame is located, wherein the transformation matrix information is embedded in the metadata in a corresponding order according to the distribution order of the video frames in the media file;

[0060] Before obtaining the transformation matrix information, the method further includes:

[0061] Generate transformation matrix information according to the transformation parameters of the video frame;

[0062] Integrate the transformation matrix information into the first motion effect description information of the motion effect element.

[0063] Optionally, in this embodiment, metadata of a media file refers to data that describes a media file. For a media file (such as a video file), metadata generally includes basic information such as the file's format, encoding, resolution, frame rate, and additional descriptive information such as transformation matrix information.

[0064] Optionally, in this embodiment, the transformation parameter is a numerical value indicating changes in position, rotation, scaling, etc. of the motion effect element in the video frame in the motion effect design, and is the basis for generating transformation matrix information.

[0065] Optionally, in this embodiment, first, in the stage of designing the motion effect, the corresponding transformation matrix information is generated according to the transformation parameters (such as initial position, rotation, scaling, etc.) of the motion effect elements in the video frame. This generation process converts the dynamic changes of the motion effect elements into a mathematical matrix form, which is convenient for unified management and application in different frames. Subsequently, the generated transformation matrix information is integrated into the first motion effect description information of the motion effect element. This information contains the initial texture information of the motion effect element, as well as the newly added transformation matrix information, which provides a detailed data basis for subsequent dynamic editing and position adjustment.

[0066] Optionally, in this embodiment, when obtaining the transformation matrix information, this information is read from the metadata of the media file where the video frame is located. Since the transformation matrix information is embedded in the metadata according to the order of the video frames, the system can accurately locate and read the corresponding transformation matrix information in the metadata according to the currently processed video frame. This reading step ensures that the system can obtain the position changes of the motion effect elements in the current frame in real time, providing accurate data support for subsequent dynamic editing and position adjustment operations.

[0067] Through the embodiments provided in this application, not only the generation and integration process of transformation matrix information is clarified, but also how to obtain this information from the metadata of media files is explained in detail. This provides key data support for real-time dynamic editing and position adjustment of motion effect elements, ensuring that the edited motion effect elements can accurately present their modified appearance and position in the video frame, thereby improving the reusability and interactive performance of the motion effect.

[0068] As an optional solution, before updating the first texture information in response to the dynamic editing request to obtain the second texture information of the dynamic effect element, the method further includes:

[0069] Acquire a touch event input by a user during playback of a media file, wherein the touch event is used to indicate dynamic adjustment of a motion effect on a video frame;

[0070] Get the touch area corresponding to the touch event;

[0071] A dynamic editing request is generated according to the touch area, wherein the dynamic editing request is used to instruct to dynamically adjust the display effect of the motion effect element in the interactive area.

[0072] Optionally, in this embodiment, the touch event refers to a user's touch operation on the device screen during the playback of a media file, including actions such as touch, slide, and zoom, which provides direct user input for triggering a dynamic editing request.

[0073] Optionally, in this embodiment, touch area: In a touch event, the specific position or area where the user touches the screen is the key basis for the system to identify and generate dynamic editing requests. Dynamic editing request: Based on the user touch event and touch area, the instruction generated by the system is used to instruct the real-time adjustment of the display effect of the dynamic effect element in the video frame, such as changing the color, size or transparency.

[0074] Optionally, in this embodiment, first, real-time monitoring of user touch events during the playback of the media file is maintained. Once the user touches the screen, the system will immediately analyze the touch event and determine the specific location or area of ​​the touch, i.e., the touch area, which is a leading step in generating a dynamic editing request. The identification of the touch area ensures that the system can accurately locate the user's intention so that the motion effect elements in the video frame can be accurately adjusted dynamically.

[0075] Then, a dynamic editing request is generated based on the determined touch area. This request contains real-time adjustment instructions for the display effect of the dynamic effect element in the touch area, such as changing the color, size, position or transparency. The generation of a dynamic editing request indicates that the user touch event is converted into a specific editing operation by the system, realizing the direct conversion from user input to system response, enhancing the interactivity between the user and the dynamic effect and the system's ability to respond to user needs.

[0076] Through the embodiments provided in the present application, not only can the user's touch events be responded to in real time, but also the touch area can be accurately located and a dynamic editing request can be generated to achieve real-time adjustment of the display effects of the motion elements in the touch area.

[0077] As an optional solution, before obtaining the video frame to be processed, the method further includes:

[0078] Get the metadata of the media file where the video frame is located;

[0079] The storage area of ​​the metadata is doubled to obtain an expanded area;

[0080] Based on the first pixel point information of the extended area and the second pixel point information of the storage area, a mixed process is performed to obtain target pixel point information of the media file, wherein the media file is rendered and displayed with the pixel value indicated by the target pixel point information during playback.

[0081] Optionally, in this embodiment, the storage area is an area in the media file that stores original pixel information, that is, it contains pixel data such as color and brightness of each frame of the video.

[0082] Optionally, in this embodiment, the storage area is doubled to obtain a new area for storing transparency information (alpha value).

[0083] Optionally, in this embodiment, the target pixel point information is pixel point data including color value and transparency information obtained after the storage area and the expansion area of ​​the media file are mixed, and is used to indicate the rendering display effect of the media file during playback.

[0084] Optionally, in this embodiment, first, the metadata of the media file containing the video frame to be processed is obtained, which not only includes the basic parameters of video playback, such as resolution, frame rate, etc., but more importantly, includes additional information for describing the properties and behaviors of motion effect elements, providing data support for subsequent transparency processing.

[0085] Subsequently, the storage area indicated in the metadata is doubled to generate an extended area. The purpose of this expansion operation is to increase the space for storing transparency information (alpha value) in the media file, so that the dynamic effect elements can have a transparent background in the video frame and blend seamlessly with the surrounding environment or native view. The extended area is adjacent to the storage area and has the same size as the storage area, ensuring that the alpha value information can correspond one-to-one with the color information, which is convenient for subsequent mixing processing.

[0086] Finally, the first pixel point information (alpha value) in the expansion area and the second pixel point information (color value, etc.) in the storage area are mixed to generate the target pixel point information. This mixing process actually combines the alpha value information with the color information to generate comprehensive pixel point data containing transparency and color information. Based on the target pixel point information, the media file can be accurately rendered and displayed during playback, achieving seamless integration of the motion effect elements and the background, and improving the visual effects and user experience of the video screen.

[0087] Through the embodiments provided in the present application, the transparency of video frames in media files is processed, more complex visual effects and perfect integration with native views are supported, and the limitations of transparent background processing in the prior art are overcome.

[0088] As an optional solution, the above-mentioned video frame motion processing method is applied to the implementation scenario of motion effects in applications. At present, there are technical solutions such as PNG sequence diagrams, SVGA, and Lottie for implementing motion effects in applications, but these solutions have their own shortcomings: for example, the resource file size of PNG sequence diagrams is large, which increases the application package size; SVGA and Lottie do not support more complex special effects (such as 3D effects, etc.); PNG sequence diagrams and Lottie have poor playback performance, causing the application to be stuck and not smooth; moreover, these motion effect production processes are complicated and cannot change dynamically with different scenes; they cannot interact with specified motion effect sub-elements, and cannot meet the needs of fast-changing application scenarios with higher requirements for interactivity.

[0089] In view of the above problems, this embodiment proposes a new way to implement motion effects. Motion effects can be presented in the form of video, thereby supporting various complex animation effects, such as 3D effects; and video motion effects with transparent channels can also be perfectly integrated with native views to enhance visual effects; at the same time, on this basis, this embodiment also proposes that animation elements can be dynamically edited and changed, and can interact with motion effect sub-elements, so that the same motion effect can be used in different scenes, improving the reusability and interactivity of motion effects, and comprehensively improving user experience.

[0090] The new dynamic effect technical solution proposed in this embodiment mainly includes four links: dynamic effect resource production link, data processing link, client dynamic effect restoration link, and dynamic effect sub-element interactive response link.

[0091] The process diagram of the animation resource production process is as follows: Figure 2 As shown, specifically including:

[0092] S202, production of dynamic effect source materials;

[0093] S204, expanding the area of ​​the source material by one time and storing the alpha value of the material;

[0094] S206, adding dynamically editable sublayers;

[0095] S208, processing and converting the layer information of the dynamic effect and the corresponding transformation matrix information using the designed data structure;

[0096] S210, storing the above information into the MP4 file MetaData to form the final dynamic effect video file resource.

[0097] It should be noted that, during the video motion effect production stage, the designer saves animation effects such as 3D effects as video resource files; and in order to perfectly integrate with the native view, the video needs to be set to a transparent background. Here, this embodiment adopts a method of doubling the area of ​​the source material to store the alpha value of the material; when rendering on the client, the R value of the pixel point in the expanded area is directly used and divided by 255 to obtain an alpha value between 0 and 1; then the source material area is mixed with the alpha value area, so that the pixel value rendered to the screen can be obtained.

[0098] The flow chart of data processing is as follows: Figure 3 As shown, specifically including:

[0099] S302, defining a data structure including layer marking information and location information of sub-elements such as text and pictures;

[0100] S304, calculating changes in information of these sub-elements in each frame during the animation process, such as position changes, color changes, and size changes;

[0101] S306, storing the above data information in the MP4 file MetaData in frame order.

[0102] It should be noted that the essence of animation is to translate, rotate, scale and other related operations on the pixels within the element. These operations can be presented in the mathematical form of transformation matrix. Similar to the implementation principle of transparent MP4, an area is added to store the transformation matrix of the dynamically replaceable dynamic effect elements. For example, this information can be stored in the Metadata data of the MP4 file. When restoring on the client side, the element at the specified position is dynamically replaced and the movement process of the element is restored.

[0103] The process diagram of the client animation restoration process is as follows: Figure 4 As shown, specifically including:

[0104] S402, the client obtains the dynamic effect video resource file;

[0105] S404, the client reads the basic information of the motion effect video and the description information, position information, and transformation matrix information of each dynamic sub-element;

[0106] S406, setting different sub-element texture information according to different scenes;

[0107] S408, combining the texture information of the sub-element and the transformation matrix information during the animation process to obtain the sub-element layer information of each frame;

[0108] S410, synthesizing the sub-element layer information with the original frame information of the motion effect video to obtain final video information of each frame;

[0109] S412, organizing into complete video data according to the frame sequence;

[0110] S414, hard-decode the video data for playback.

[0111] It should be noted that the client extracts the video frame data from the video file, and then the transformation matrix of each frame in the Metadata. By multiplying the original data by the transformation matrix, the position of the layer in each frame can be obtained.

[0112] The flow chart of the interactive response phase of the dynamic sub-element is as follows: Figure 5 As shown, specifically including:

[0113] S502, playing the animation file;

[0114] S504, adding a response to the entire motion effect video;

[0115] S506, the user clicks on the video at a certain frame;

[0116] S508, obtaining the click position of the user on the rendering screen, as well as the layer information and layer position information of the current frame;

[0117] S510, mapping the original region information of the frame layer and the click position obtained after the actual video rendering;

[0118] S512, determine which layer the clicked position belongs to, and distribute the response.

[0119] Finally, it is to realize the interactive response of the animation sub-element. Because the animation sub-element and the entire animation are a whole, the conventional method cannot add a response separately. Here, this application adopts an overall response, and distributes it to the specific elements in the corresponding area of ​​the current frame based on the judgment of the user's click area.

[0120] Through the above processing flow, this embodiment realizes a transparent MP4 animation that supports complex effects and perfectly integrates with the native view, and supports dynamic custom editing of animations during the animation playback process, so that it can be used in different scenarios. The interactive response of sub-elements it supports can further improve the interactivity of the animation, thereby comprehensively improving the user experience.

[0121] It should be noted that due to the high compression rate, high frame rate and high resolution of the video, this embodiment uses this format to present the motion effect. In order to solve the problem that the original video does not support transparent background and cannot be perfectly integrated with the native view, this embodiment processes the video data and adds an area to store the alpha value of the corresponding pixel point. Then, during the client rendering stage, data synthesis is performed so that the pixel points finally rendered to the screen have alpha value attributes, thus realizing the transparent background of the video.

[0122] During the animation design phase, this embodiment defines which sub-elements in the animation can be dynamically edited and labels them; then calculates the information about the changes in size, position, color, etc. of these sub-elements during the animation, assembles them according to a specific data structure, and then organizes this information in frame order and stores it as a whole in the MetaData of the video file.

[0123] After the above two steps, you can get the final dynamic effect video file resources.

[0124] After obtaining the video file resource, the client will restore it according to the following process: First, the client will read the basic information of the video, the description information of the dynamic sub-element, and the transformation matrix of the sub-element in each frame during the dynamic process from the video resource file; then, based on the current application scenario, different sub-element textures are set to replace the sub-element information in the original video; then, multiply by the transformation matrix information to obtain the size and position information of the custom sub-element in each frame to form the actual sub-element layer, which is then merged with the original video layer to obtain the final layer rendered to the screen; finally, the client performs hardware decoding and playback of the video data.

[0125] During the playback of the animation, interact with the sub-elements. Since the video file is a whole, it is not possible to directly add interactive responses to the sub-elements. This embodiment proposes a response distribution algorithm to deal with this problem. The specific principle is as follows: add a response to the video as a whole. When the user clicks on the video, obtain the click position on the rendering screen, as well as the description information and position information of all layers of the current frame. The original area information of the frame layer and the obtained click position on the rendering screen are mapped to the same coordinate system through the algorithm, and then determine which layer the click position belongs to, so as to distribute the response.

[0126] It is understandable that in the specific implementation of this application, related data such as user information is involved. When the above embodiments of this application are applied to specific products or technologies, user permission or consent is required, and the collection, use and processing of relevant data need to comply with relevant laws, regulations and standards of relevant countries and regions.

[0127] It should be noted that, for the aforementioned method embodiments, for the sake of simplicity, they are all expressed as a series of action combinations, but those skilled in the art should be aware that the present application is not limited by the described order of actions, because according to the present application, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the present application.

[0128] According to another aspect of the embodiment of the present application, a video frame motion effect processing device for implementing the above-mentioned video frame motion effect processing method is also provided. Figure 6 As shown, the device comprises:

[0129] An acquisition unit 602 is used to acquire a video frame to be processed, wherein the video frame is associated with a corresponding motion effect element;

[0130] A reading unit 604 is used to read first motion effect description information of a video frame, wherein the first motion effect description information is used to indicate a first display effect of a motion effect element, and the first motion effect description information includes first texture information of the video frame;

[0131] An updating unit 606, configured to update the first texture information in response to the dynamic editing request to obtain second texture information of the dynamic effect element;

[0132] A first generating unit 608 is used to generate second motion effect description information of the video frame based on the second texture information, wherein the second motion effect description information is used to indicate a second display effect of the motion effect element;

[0133] The second generating unit 610 is used to generate a motion effect layer corresponding to the motion effect element based on the second motion effect description information.

[0134] As an optional solution, the second generating unit 610 includes:

[0135] A first acquisition module is used to acquire size and position information of the motion effect element in the video frame from the second motion effect description information, and generate a first element layer of the motion effect element according to the size and position information;

[0136] A merging module, used to merge the first element layer with the original element layer of the motion effect element in the video frame to obtain a second element layer of the motion effect element;

[0137] The determination module is used to determine the second element layer as a motion effect layer.

[0138] As an optional solution, the first generating unit 608 includes:

[0139] A second acquisition module is used to acquire transformation matrix information, wherein the transformation matrix information is used to indicate position information of the motion effect element in the video frame, and the first motion effect description information includes the transformation matrix information;

[0140] The operation module is used to use the transformation matrix information to perform a matrix operation on the second texture information to obtain the second motion effect description information of the video frame.

[0141] As an optional solution, the second acquisition module includes:

[0142] An acquisition submodule is used to acquire transformation matrix information from metadata of a media file where a video frame is located, wherein the transformation matrix information is embedded in the metadata in a corresponding order according to the distribution order of the video frames in the media file;

[0143] The device also includes:

[0144] A first generating module, used for generating transformation matrix information according to the transformation parameters of the video frame before acquiring the transformation matrix information;

[0145] The integration module is used to integrate the transformation matrix information into the first motion effect description information of the motion effect element before obtaining the transformation matrix information.

[0146] As an optional solution, the device also includes:

[0147] A third acquisition module is used to update the first texture information in response to the dynamic editing request, and before obtaining the second texture information of the motion effect element, obtain a touch event input by a user during the playback of the media file, wherein the touch event is used to indicate the dynamic adjustment of the motion effect on the video frame;

[0148] A fourth acquisition module is used to acquire a touch area corresponding to a touch event before updating the first texture information in response to a dynamic editing request to obtain second texture information of the dynamic effect element;

[0149] The second generation module is used to generate a dynamic editing request based on the touch area before updating the first texture information in response to the dynamic editing request to obtain the second texture information of the dynamic effect element, wherein the dynamic editing request is used to indicate the dynamic adjustment of the display effect of the dynamic effect element in the interactive area.

[0150] As an optional solution, the device also includes:

[0151] A fifth acquisition module, before acquiring the video frame to be processed, acquires metadata of the media file where the video frame is located;

[0152] An expansion module, used for doubling the storage area of ​​metadata to obtain an expanded area before obtaining the video frame to be processed;

[0153] The mixing module is used to perform mixing processing based on the first pixel point information of the expansion area and the second pixel point information of the storage area before obtaining the video frame to be processed, so as to obtain the target pixel point information of the media file, wherein the media file is rendered and displayed with the pixel value indicated by the target pixel point information during the playback process.

[0154] According to another aspect of the embodiment of the present application, an electronic device for implementing the above-mentioned video frame motion effect processing method is further provided, Figure 7 As shown, the electronic device includes a memory 702 and a processor 704. The memory 702 stores a computer program, and the processor 704 is configured to execute the steps in any of the above method embodiments through the computer program.

[0155] Optionally, in this embodiment, the electronic device may be located in at least one network device among a plurality of network devices of a computer network.

[0156] Optionally, in this embodiment, the processor may be configured to perform the following steps through a computer program:

[0157] S1, obtaining a video frame to be processed, wherein the video frame is associated with a corresponding motion effect element;

[0158] S2, reading first motion effect description information of the video frame, wherein the first motion effect description information is used to indicate a first display effect of the motion effect element, and the first motion effect description information includes first texture information of the video frame;

[0159] S3, in response to the dynamic editing request, updating the first texture information to obtain second texture information of the dynamic effect element;

[0160] S4, generating second motion effect description information of the video frame based on the second texture information, wherein the second motion effect description information is used to indicate a second display effect of the motion effect element;

[0161] S5: Generate a motion effect layer corresponding to the motion effect element based on the second motion effect description information.

[0162] Alternatively, a person skilled in the art may understand that: Figure 7 The structure shown is for illustration only. Figure 7 The structure of the electronic device is not limited. Figure 7 More or fewer components (such as network interfaces, etc.) as shown in, or with Figure 7 Different configurations are shown.

[0163] Among them, the memory 702 can be used to store software programs and modules, such as the program instructions / modules corresponding to the method and device for processing the dynamic effects of video frames in the embodiments of the present application. The processor 704 executes various functional applications and data processing by running the software programs and modules stored in the memory 702, that is, to implement the above-mentioned method for processing the dynamic effects of video frames. The memory 702 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 702 may further include a memory remotely located relative to the processor 704, and these remote memories may be connected to the electronic device via a network. Examples of the above-mentioned networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof. Among them, the memory 702 may be specifically, but not limited to, used to store information such as the first dynamic effect description information. As an example, such as Figure 7As shown, the memory 702 may include, but is not limited to, the acquisition unit 602, the reading unit 604, the updating unit 606, the first generating unit 608, and the second generating unit 610 in the video frame motion effect processing device. In addition, it may also include, but is not limited to, other module units in the video frame motion effect processing device, which will not be described in detail in this example.

[0164] Optionally, the transmission device 706 is used to receive or send data via a network. Specific examples of the network may include a wired network and a wireless network. In one example, the transmission device 706 includes a network adapter (Network Interface Controller, NIC), which can be connected to other network devices and routers via a network cable so as to communicate with the Internet or a local area network. In one example, the transmission device 706 is a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.

[0165] In addition, the electronic device further includes: a display 708 for displaying information such as the first motion effect description information; and a connection bus 710 for connecting various module components in the electronic device.

[0166] In other embodiments, the client or server may be a node in a distributed system, wherein the distributed system may be a blockchain system, and the blockchain system may be a distributed system formed by connecting the multiple nodes in the form of network communication. A peer-to-peer network may be formed between the nodes, and any form of computing device, such as a server, a client, or other electronic device, may become a node in the blockchain system by joining the peer-to-peer network.

[0167] According to one aspect of the present application, a computer program product is provided, the computer program product comprising a computer program / instruction, the computer program / instruction comprising a program code for executing the method shown in the flow chart. In such an embodiment, the computer program can be downloaded and installed from a network through a communication part, and / or installed from a removable medium. When the computer program is executed by a central processing unit, various functions provided by the embodiments of the present application are executed.

[0168] The serial numbers of the above-mentioned embodiments of the present application are for description only and do not represent the advantages or disadvantages of the embodiments.

[0169] It should be noted that the computer system of the electronic device is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.

[0170] The computer system includes a central processing unit (CPU), which can perform various appropriate actions and processes according to the program stored in the read-only memory (ROM) or the program loaded from the storage part to the random access memory (RAM). In the random access memory, various programs and data required for system operation are also stored. The central processing unit, the read-only memory and the random access memory are connected to each other through a bus. The input / output interface (I / O interface) is also connected to the bus.

[0171] The following components are connected to the input / output interface: an input part including a keyboard, a mouse, etc.; an output part including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker; a storage part including a hard disk, etc.; and a communication part including a network interface card such as a local area network card, a modem, etc. The communication part performs communication processing via a network such as the Internet. A drive is also connected to the input / output interface as needed. Removable media, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., are installed on the drive as needed so that the computer program read therefrom is installed into the storage part as needed.

[0172] In particular, according to an embodiment of the present application, the process described in each method flow chart can be implemented as a computer software program. For example, an embodiment of the present application includes a computer program product, which includes a computer program carried on a computer readable medium, and the computer program contains a program code for executing the method shown in the flow chart. In such an embodiment, the computer program can be downloaded and installed from a network through a communication part, and / or installed from a removable medium. When the computer program is executed by a central processing unit, various functions defined in the system of the present application are executed.

[0173] According to one aspect of the present application, a computer-readable storage medium is provided, and a processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes the methods provided in the above-mentioned various optional implementations.

[0174] Optionally, in this embodiment, the computer-readable storage medium may be configured to store a computer program for performing the following steps:

[0175] S1, obtaining a video frame to be processed, wherein the video frame is associated with a corresponding motion effect element;

[0176] S2, reading first motion effect description information of the video frame, wherein the first motion effect description information is used to indicate a first display effect of the motion effect element, and the first motion effect description information includes first texture information of the video frame;

[0177] S3, in response to the dynamic editing request, updating the first texture information to obtain second texture information of the dynamic effect element;

[0178] S4, generating second motion effect description information of the video frame based on the second texture information, wherein the second motion effect description information is used to indicate a second display effect of the motion effect element;

[0179] S5: Generate a motion effect layer corresponding to the motion effect element based on the second motion effect description information.

[0180] Optionally, in this embodiment, a person of ordinary skill in the art may understand that all or part of the steps in the various methods of the above embodiments may be completed by instructing hardware related to the electronic device through a program, and the program may be stored in a computer-readable storage medium, and the storage medium may include: a flash drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, etc.

[0181] The serial numbers of the above-mentioned embodiments of the present application are for description only and do not represent the advantages or disadvantages of the embodiments.

[0182] If the integrated units in the above embodiments are implemented in the form of software functional units and sold or used as independent products, they can be stored in the above computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for enabling one or more computer devices (which can be personal computers, servers or network devices, etc.) to execute all or part of the steps of the methods of each embodiment of the present application.

[0183] In the above embodiments of the present application, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, please refer to the relevant description of other embodiments.

[0184] In the several embodiments provided in the present application, it should be understood that the recorded client can be implemented in other ways. Among them, the device embodiments described above are only schematic, for example, the division of units is only a logical function division, and there may be other division methods in actual implementation, for example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of units or modules, which can be electrical or other forms.

[0185] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0186] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0187] The above are only preferred implementations of the present application. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present application. These improvements and modifications should also be regarded as the scope of protection of the present application.

Claims

1. A method for processing motion effects of video frames, characterized in that: include: Obtaining a video frame to be processed, wherein the video frame is associated with a corresponding motion effect element; Reading first motion effect description information of the video frame, wherein the first motion effect description information is used to indicate a first display effect of the motion effect element, and the first motion effect description information includes first texture information of the video frame; In response to the dynamic editing request, the first texture information is updated to obtain second texture information of the dynamic effect element; Based on the second texture information, generating second motion effect description information of the video frame, wherein the second motion effect description information is used to indicate a second display effect of the motion effect element; Based on the second motion effect description information, a motion effect layer corresponding to the motion effect element is generated.

2. The method according to claim 1, characterized in that The step of generating a motion effect layer corresponding to the motion effect element based on the second motion effect description information includes: Acquire the size and position information of the motion effect element in the video frame from the second motion effect description information, and generate a first element layer of the motion effect element according to the size and position information; Merging the first element layer with the original element layer of the motion effect element in the video frame to obtain a second element layer of the motion effect element; The second element layer is determined as the motion effect layer.

3. The method according to claim 1, characterized in that: The generating second motion effect description information of the video frame based on the second texture information includes: Acquire transformation matrix information, wherein the transformation matrix information is used to indicate position information of the motion effect element in the video frame, and the first motion effect description information includes the transformation matrix information; Using the transformation matrix information, a matrix operation is performed on the second texture information to obtain second motion effect description information of the video frame.

4. The method according to claim 3, characterized in that The obtaining of transformation matrix information includes: Acquire the transformation matrix information from metadata of a media file where the video frame is located, wherein the transformation matrix information is correspondingly embedded in the metadata according to a distribution order of the video frames in the media file; Before obtaining the transformation matrix information, the method further includes: Generating the transformation matrix information according to the transformation parameters of the video frame; The transformation matrix information is integrated into the first motion effect description information of the motion effect element.

5. The method according to claim 4, characterized in that Before updating the first texture information in response to the dynamic editing request to obtain the second texture information of the dynamic effect element, the method further includes: Acquire a touch event input by a user during playback of the media file, wherein the touch event is used to instruct dynamic adjustment of the motion effect of the video frame; Obtaining the touch area corresponding to the touch event; The dynamic editing request is generated according to the touch area, wherein the dynamic editing request is used to instruct to dynamically adjust the display effect of the motion effect element in the interactive area.

6. The method according to any one of claims 1 to 5, characterized in that: Before obtaining the video frame to be processed, the method further includes: Obtain metadata of the media file where the video frame is located; doubling the storage area of ​​the metadata to obtain an expanded area; Based on the first pixel point information of the extended area and the second pixel point information of the storage area, a mixing process is performed to obtain the target pixel point information of the media file, wherein the media file is rendered and displayed with the pixel value indicated by the target pixel point information during playback.

7. A video frame motion effect processing device, characterized in that: include: An acquisition unit, used for acquiring a video frame to be processed, wherein the video frame is associated with a corresponding motion effect element; A reading unit, configured to read first motion effect description information of the video frame, wherein the first motion effect description information is used to indicate a first display effect of the motion effect element, and the first motion effect description information includes first texture information of the video frame; An updating unit, configured to update the first texture information in response to a dynamic editing request to obtain second texture information of the dynamic effect element; A first generating unit, configured to generate second motion effect description information of the video frame based on the second texture information, wherein the second motion effect description information is used to indicate a second display effect of the motion effect element; The second generating unit is used to generate a motion effect layer corresponding to the motion effect element based on the second motion effect description information.

8. The device according to claim 7, characterized in that The second generating unit comprises: A first acquisition module, configured to acquire size and position information of the motion effect element in the video frame from the second motion effect description information, and generate a first element layer of the motion effect element according to the size and position information; A merging module, used for merging the first element layer with the original element layer of the motion effect element in the video frame to obtain a second element layer of the motion effect element; A determination module is used to determine the second element layer as the motion effect layer.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium includes a stored program, wherein the program executes the method described in any one of claims 1 to 7 when executed by an electronic device.

10. An electronic device comprising a memory and a processor, characterized in that: A computer program is stored in the memory, and the processor is configured to execute the method according to any one of claims 1 to 7 through the computer program.

Images