Method, device, electronic device and storage medium for generating transition frame

Abstract

The present application provides a method, device, electronic device and storage medium for generating a transition frame. The method includes first obtaining the style features and feature matrix of the key frames of the animation to be produced, then inputting the style features and the feature matrix of the key frames into a trained prediction model to obtain the feature matrix of the transition frame of the animation, and obtaining the transition frame of the animation based on the feature matrix of the transition frame, so that the feature matrix of the transition frame generated by the prediction model can be controlled by the obtained style features, so that the final transition frame is more accurate.

Description

Technical Field

[0001] The present application relates to the technical field of transition frames, and in particular to a method, device, electronic device, and storage medium for generating a transition frame. Background Art

[0002] This section is intended to provide a background or context to the embodiments of the application that are recited in the claims. No statement herein is admitted to be prior art by virtue of its inclusion in this section.

[0003] Currently, games or film animations often face the problem of connecting two actions, which requires generating an intermediate transition frame between the two key frames. However, there are two main methods for generating transition frames in the existing technology. One method is manual production, in which animators design a series of actions based on the requirements and process each frame of animation. This method is not only inefficient but also requires high requirements from the animators themselves. Another method is to generate transition frames through linear interpolation. This method can cause problems such as slippage and stiffness in the animation, which can be very noticeable in longer sequences. Summary of the Invention

[0004] In view of this, the purpose of the present application is to provide a method, device, electronic device and storage medium for generating a transition frame.

[0005] Based on the above objectives, the present application provides a method for generating a transition frame, comprising:

[0006] Obtain the style features and keyframe feature matrix of the animation to be produced;

[0007] Inputting the style features and the feature matrix of the key frames into the trained prediction model to obtain a feature matrix of the transition frames of the animation;

[0008] The transition frame of the animation is obtained based on the feature matrix of the transition frame.

[0009] In some embodiments, the method of training the prediction model includes:

[0010] Setting multiple style features of the animation and determining a feature matrix of sample key frames corresponding to each style feature;

[0011] The feature matrix of each style feature and its corresponding sample key frame is input into the benchmark model of the prediction model for model training, so that the feature matrix of the transition frame output by the benchmark model during testing corresponds to the feature matrix of the sample key frame.

[0012] In some embodiments, the baseline model of the prediction model is GAN, and the generator and discriminator of the GAN are both transformers.

[0013] In some embodiments, the style features and the feature matrix of the key frames are input into a trained prediction model to obtain a feature matrix of the transition frames of the animation, which specifically includes:

[0014] Obtain the target number of transition frames required for the animation;

[0015] The style features, the feature matrix of the key frames, and the target quantity are input into the trained prediction model to obtain the feature matrix of the transition frames of the animation.

[0016] In some embodiments, the style feature, the feature matrix of the key frame, and the target number are input into a trained prediction model to obtain a feature matrix of the transition frame of the animation, specifically including:

[0017] Inputting the style features and the feature matrix of the key frames into the trained prediction model to obtain feature matrices of a preset number of candidate transition frames of the animation; wherein each candidate transition frame corresponds to an arrangement sequence number in chronological order;

[0018] Determine a plurality of arrangement numbers corresponding to the target number, and determine a feature matrix of the transition frame of the animation from a preset number of feature matrices of the candidate transition frames based on the plurality of arrangement numbers; wherein the target number is not greater than the preset number.

[0019] In some embodiments, the feature matrix includes a displacement matrix and a skeletal rotation matrix.

[0020] In some embodiments, after obtaining the feature matrix of the transition frame of the animation, the method further includes:

[0021] The characteristic matrix of the transition frame of the animation is adjusted based on a forward dynamics algorithm and an inverse dynamics algorithm.

[0022] In some embodiments, the keyframes include a beginning keyframe and an ending keyframe of the animation.

[0023] Based on the same inventive concept, the exemplary embodiment of the present application further provides a device for generating a transition frame, including:

[0024] An acquisition module obtains the style features and keyframe feature matrix of the animation to be produced;

[0025] A model module, inputting the style features and the feature matrix of the key frame into the trained prediction model to obtain the feature matrix of the transition frame of the animation;

[0026] A generation module generates the transition frame of the animation based on the feature matrix of the transition frame.

[0027] Based on the same inventive concept, an exemplary embodiment of the present application further provides an electronic device, comprising a memory, a processor, and a computer program stored on the memory and executable by the processor, wherein the processor implements the method for generating a transition frame as described above when executing the program.

[0028] Based on the same inventive concept, an exemplary embodiment of the present application further provides a non-transitory computer-readable storage medium, wherein the non-transitory computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a computer to execute the method for generating a transition frame as described above.

[0029] As can be seen from the above, the method, device, electronic device and storage medium for generating transition frames provided by the present application first obtain the style features of the animation to be produced and the feature matrix of the key frames, and then input the style features and the feature matrix of the key frames into the prediction model obtained by training to obtain the feature matrix of the transition frame of the animation, and obtain the transition frame of the animation based on the feature matrix of the transition frame, so that the feature matrix of the transition frame generated by the prediction model can be controlled by the acquired style features, so that the final transition frame is more accurate. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] In order to more clearly illustrate the technical solutions in this application or related technologies, the following briefly introduces the drawings required for use in the embodiments or related technical descriptions. Obviously, the drawings described below are merely embodiments of this application. For ordinary technical members in this field, other drawings can be obtained based on these drawings without creative work.

[0031] Figure 1 This is a schematic diagram of an application scenario of an exemplary embodiment of the present application;

[0032] Figure 2 A schematic flow chart of a method for generating a transition frame according to an embodiment of the present application;

[0033] Figure 3 This is a schematic structural diagram of a device for generating a transition frame according to an embodiment of the present application;

[0034] Figure 4 This is a schematic structural diagram of a specific electronic device according to an embodiment of the present application. DETAILED DESCRIPTION

[0035] The principles and spirit of the present application will be described below with reference to several exemplary embodiments. It should be understood that these embodiments are provided solely to enable those skilled in the art to better understand and implement the present application, and are not intended to limit the scope of the present application in any way. Rather, these embodiments are provided to make the present application more thorough and complete, and to fully convey the scope of the present application to those skilled in the art.

[0036] According to the embodiments of the present application, a method, device, electronic device, and storage medium for generating a transition frame are proposed.

[0037] It should be understood herein that any number of elements in the drawings is for illustration only and not for limitation, and any naming is only for distinction and does not have any limiting meaning.

[0038] The principles and spirit of the present application are explained in detail below with reference to several representative embodiments of the present application.

[0039] There are two main methods for generating transition frames in the existing technology. One is manual creation, where animators design a series of movements based on the requirements and process each frame of animation. This method is not only inefficient but also requires high standards of the animators themselves. Another method is to generate transition frames through linear interpolation, which can lead to problems such as slippage and stiffness in the animation, which are particularly noticeable in longer sequences.

[0040] In addition, the inventors of this application found that if transition frames are generated directly through a deep learning model, there is a problem of uncontrollable generated content. For example, if a user wants to generate a set of transition frames of a person running, directly inputting the key frames into the neural network model may result in a set of transition frames of the person long jumping. This results in the inability to meet user needs by directly generating transition frames through a deep learning model.

[0041] In order to solve the above problems, the present application provides a method for generating a transition frame, which specifically includes:

[0042] The style features and feature matrix of the key frames of the animation to be produced are obtained, and then the style features and the feature matrix of the key frames are input into the prediction model obtained through training to obtain the feature matrix of the transition frames of the animation, and the transition frames of the animation are obtained based on the feature matrix of the transition frames. The feature matrix of the transition frames generated by the prediction model can be controlled by the obtained style features, so that the final transition frames are more accurate and meet the needs of users.

[0043] After introducing the basic principles of the present application, various non-limiting implementation methods of the present application are described in detail below.

[0044] refer to Figure 1, which is a schematic diagram of an application scenario of the document correction method provided by the exemplary embodiment of the present application. The application scenario includes a terminal device 101, a server 102, and a data storage system 103. Among them, the terminal device 101, the server 102 and the data storage system 103 can be connected through a wired or wireless communication network. The terminal device 101 includes but is not limited to a desktop computer, a mobile phone, a mobile computer, a tablet computer, a media player, a smart wearable device, a personal digital assistant (PDA) or other electronic devices that can realize the above functions. The server 102 and the data storage system 103 can be independent physical servers, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms.

[0045] The server 102 is used to provide a transition frame generation service to the user of the terminal device 101. The terminal device 101 is installed with a client that communicates with the server 102, and the user can obtain the animation transition frame through the client. During the document correction process, the client obtains the style features of the animation to be produced and the feature matrix of the key frames by calling the input components of the terminal device 101 (such as mouse, keyboard, touch screen control, camera, etc.); the client sends the style features of the animation to be produced and the feature matrix of the key frames to the server 102. The server 102 obtains the feature matrix of the transition frame of the animation through the prediction model obtained through training, and obtains the transition frame of the animation based on the feature matrix of the transition frame. After obtaining the transition frame of the animation, the acquisition result is sent to the terminal device 101, so as to accurately generate the transition frame of the animation.

[0046] The data storage system 103 stores a large amount of training data, each of which includes a style feature and a feature matrix of sample keyframes corresponding to the style feature. The server 102 can train the prediction model based on this large amount of training data, enabling the prediction model to obtain transition frames of the animation corresponding to the style feature based on the input animation style features and keyframe feature matrix. The sources of training data include, but are not limited to, existing databases, data obtained from the internet, or data uploaded by users when using the client. When the output of the prediction model meets predetermined requirements, the server 102 can provide the user with a processing service for generating transition frames based on the prediction model. Furthermore, the server 102 can continuously optimize the prediction model based on newly added training data.

[0047] The following combination Figure 1The document correction method according to the exemplary embodiment of the present application is described with reference to the application scenario. It should be noted that the above application scenario is only shown to facilitate understanding of the spirit and principles of the present application, and the implementation of the present application is not limited in this respect. On the contrary, the implementation of the present application can be applied to any applicable scenario.

[0048] refer to Figure 2 , an embodiment of the present application provides a method for generating a transition frame, comprising the following steps:

[0049] S101, obtaining the style features and key frame feature matrix of the animation to be produced.

[0050] In specific implementation, when generating transition frames, the style features of the animation to be produced and the feature matrix of the animation to be produced are first obtained. Optionally, the style features are obtained through user input. The style features are mainly used to enable the prediction model to know what kind of transition frames should be generated when generating transition frames. For example, if the style feature is the feature of a person running, then the corresponding transition frame generated is a transition frame of the person running, or if the style feature is the feature of a person long jumping, then the corresponding transition frame generated is a transition frame of the person long jumping.

[0051] In some embodiments, the characteristic matrix of the key frame includes a displacement matrix and a skeletal rotation matrix, that is, the movement and rotation of the moving object in the key frame are represented by the displacement matrix and the skeletal rotation matrix. It should be noted that the characteristic matrix of the key frame can be set according to the needs of making the animation. The above-mentioned characteristic matrix including the displacement matrix and the skeletal rotation matrix is ​​only a specific implementation method provided by the embodiment of the present application. Those skilled in the art can set a variety of characteristic matrices that can represent the characteristics of the key frame as needed, which is not limited here. For example, the characteristic matrix can be set to the position matrix of multiple skeletal key points in the animation.

[0052] In some embodiments, the key frames of the animation to be produced include the opening key frame and the ending key frame of the animation. Optionally, the key frames of the animation to be produced may include only one key frame, or may include multiple preceding key frames before the opening key frame, or intermediate key frames between the opening key frame and the ending key frame, without limitation herein. Generally speaking, the more key frames input into the prediction model, the higher the accuracy of the resulting transition frame.

[0053] S102: Input the style features and the feature matrix of the key frame into a prediction model obtained through training to obtain a feature matrix of the transition frame of the animation.

[0054] In a specific implementation, after obtaining the style features and feature matrix of the key frames of the animation to be produced, the style features and feature matrix of the key frames are input into the trained prediction model to obtain the feature matrix of the transition frames of the animation.

[0055] In some embodiments, the method of training the prediction model includes:

[0056] Setting multiple style features of the animation and determining a feature matrix of sample key frames corresponding to each style feature;

[0057] The feature matrix of each style feature and its corresponding sample key frame is input into the benchmark model of the prediction model for model training, so that the feature matrix of the transition frame output by the benchmark model during testing corresponds to the feature matrix of the sample key frame.

[0058] During implementation, various style features of the animation to be produced are first set as needed. These style features can include running, long jump, walking, flying, and jumping. Optionally, for ease of input, the style features can be represented by a column of numbers, with different numbers corresponding to different style features. After setting the style features, a feature matrix of corresponding sample key frames is matched for each style feature. For example, a large number of key frames of various running movements are matched for the running style feature. Then, each style feature and its corresponding sample key frame feature matrix are input into the benchmark model of the prediction model for model training, so that the feature matrix of the transition frame output by the benchmark model during testing corresponds to the feature matrix of the sample key frames. Optionally, the benchmark model can select any neural network model as needed, which is not limited here.

[0059] It should be noted that since style features are features set by the user, the model itself cannot recognize each feature well. Therefore, model training is required to enable the model to better recognize each style feature. For example, when training the model, the running style features and the key frames of the running action are used as the input of the model. Then, in the testing phase, when the model receives the running style features and another key frame, the model will also recognize the other key frame as the key frame of the running action by default. This ensures that the model can control the style of the transition frame output by the model through the input style features during the application phase.

[0060] In order to further improve the accuracy of the prediction model, in some embodiments, the baseline model of the prediction model is GAN, and the generator and discriminator of the GAN are both transformers.

[0061] In practice, GANs (Generative Adversarial Networks) are deep learning models in which the generator generates transition frames, while the discriminator determines whether the generator's generated transition frames are correct, further improving the accuracy of the prediction model's generation of transition frames. Unlike RNNs (Recurrent Neural Networks) and CNNs (Convolutional Neural Networks), Transformers utilize a self-attention mechanism to process deep models with variable-length inputs. Their network consists solely of self-attention and feedforward neural networks.

[0062] In some embodiments, the style features and the feature matrix of the key frames are input into a trained prediction model to obtain a feature matrix of the transition frames of the animation, which specifically includes:

[0063] Obtain the target number of transition frames required for the animation;

[0064] The style features, the feature matrix of the key frames, and the target quantity are input into the trained prediction model to obtain the feature matrix of the transition frames of the animation.

[0065] During specific implementation, under normal circumstances, the number of transition frames output by the model is fixed, that is, the number of model output results is controlled by setting the structure of the model. However, for different motion accuracy requirements, the number of transition frames that need to be inserted may be different. At this time, simply fixing the number of transition frames output by the model through the structural setting of the model can no longer meet the needs of users. Therefore, it is necessary to obtain the target number of transition frames required for the animation. Optionally, the target number can be input by the user. After obtaining the target number, the target number is input into the prediction model obtained through training together with the style features and the feature matrix of the key frame, thereby controlling the number of feature matrices of the transition frames of the animation.

[0066] In some embodiments, the style feature, the feature matrix of the key frame, and the target number are input into a trained prediction model to obtain a feature matrix of the transition frame of the animation, specifically including:

[0067] Inputting the style features and the feature matrix of the key frames into the trained prediction model to obtain feature matrices of a preset number of candidate transition frames of the animation; wherein each candidate transition frame corresponds to an arrangement sequence number in chronological order;

[0068] Determine a plurality of arrangement numbers corresponding to the target number, and determine a feature matrix of the transition frame of the animation from a preset number of feature matrices of the candidate transition frames based on the plurality of arrangement numbers; wherein the target number is not greater than the preset number.

[0069] In a specific implementation, the style features and the feature matrices of the key frames are first input into the trained prediction model to obtain feature matrices for a preset number of candidate transition frames of the animation. Optionally, the preset number can be controlled by the model's structural settings. Optionally, the specific number of the preset number can be set as needed. To expand the scope of application of the model, the preset number can be set to a larger value. After obtaining the feature matrices of the preset number of candidate transition frames of the animation, multiple permutation numbers corresponding to the target number are determined, and based on the multiple permutation numbers, the feature matrices of the transition frames of the animation are determined from the feature matrices of the preset number of candidate transition frames; wherein the target number is not greater than the preset number. Each of the preset number of candidate transition frames corresponds to a permutation number in chronological order. For example, if the preset number is 20, the model outputs 20 candidate transition frames, which are then sequentially assigned permutation numbers 1-20 in chronological order. In this case, the maximum number of candidate transition frames output by the model is 20. Then, the obtained target number is used to determine multiple permutation numbers corresponding to the target number. For example, if the target number is 10, then 10 frames are selected from candidate transition frames 1 to 20 as the final output transition frames. Optionally, the process of selecting candidate transition frames can be completed by random selection, or multiple permutation numbers corresponding to each target number can be set as needed, which is not limited here.

[0070] S103: Obtain the transition frame of the animation based on the feature matrix of the transition frame.

[0071] In a specific implementation, after obtaining the characteristic matrix of the transition frame, the transition frame of the animation can be obtained through the characteristic matrix of the transition frame. Optionally, the characteristic matrix of the transition frame can be input into a software system for making an animation, and then the software system for making an animation automatically generates the transition frame.

[0072] In some embodiments, after obtaining the feature matrix of the transition frame of the animation, the method further includes:

[0073] The characteristic matrix of the transition frame of the animation is adjusted based on a forward dynamics algorithm and an inverse dynamics algorithm.

[0074] During specific implementation, in order to make the action of the transition frame finally generated have stable motion performance, the characteristic matrix of the transition frame of the animation can be adjusted by the forward dynamics algorithm (FK) and the inverse dynamics algorithm (IK). Among them, the forward dynamics algorithm (FK) is used to calculate the end position of each joint point of the moving object through the rotation amount of each joint point of the moving object. The inverse dynamics algorithm (IK) is used to infer the rotation amount of each joint point of the moving object through the end position of the joint of the moving object. For example, due to the different lengths of the bones, the feet of a character in an animation will leave the ground in some actions. To deal with this problem, it is usually necessary to lock the feet on the ground, and then use the IK algorithm to solve and correct the positions of all bone points.

[0075] The method for generating transition frames provided in the present application first obtains the style features of the animation to be produced and the feature matrix of the key frames, then inputs the style features and the feature matrix of the key frames into the prediction model obtained through training to obtain the feature matrix of the transition frames of the animation, and obtains the transition frames of the animation based on the feature matrix of the transition frames, so that the feature matrix of the transition frames generated by the prediction model can be controlled by the acquired style features, so that the final transition frames are more accurate. In addition, the method for generating transition frames provided in the present application can realize the application of controllable automatic generation of transition frames in games, assist various gameplay and performance effects, and make it possible to control the generation of high-quality transition frames that were not possible in the past. At the same time, the method for generating transition frames provided in the present application can effectively reduce the workload by assisting the existing methods in the art tool chain. Animators can only provide a small amount of key data to obtain a long action sequence, and use the method of the present application to refine the action, which greatly saves their costs.

[0076] Based on the same inventive concept, corresponding to any of the above-mentioned embodiment methods, the present application also provides a device for generating a transition frame.

[0077] refer to Figure 3 , the generation device of the transition frame includes:

[0078] Acquisition module 201, acquires the style features and key frame feature matrix of the animation to be produced;

[0079] Model module 202, inputs the style features and the feature matrix of the key frame into the trained prediction model to obtain the feature matrix of the transition frame of the animation;

[0080] The generation module 203 generates the transition frame of the animation based on the feature matrix of the transition frame.

[0081] For the convenience of description, the above devices are described as being divided into various modules according to their functions. Of course, when implementing this application, the functions of each module can be implemented in the same or multiple software and / or hardware.

[0082] The apparatus of the above embodiment is used to implement the corresponding method for generating a transition frame in any of the above embodiments, and has the beneficial effects of the corresponding method embodiment, which will not be described in detail here.

[0083] Based on the same inventive concept, corresponding to any of the above-mentioned embodiments and methods, the present application also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and runnable on the processor, wherein when the processor executes the program, the method for generating a transition frame described in any of the above embodiments is implemented.

[0084] Figure 4 10 is a schematic diagram showing a more specific hardware structure of an electronic device provided in this embodiment. The device may include: a processor 1010, a memory 1020, an input / output interface 1030, a communication interface 1040, and a bus 1050. The processor 1010, the memory 1020, the input / output interface 1030, and the communication interface 1040 are communicatively connected to each other within the device via the bus 1050.

[0085] The processor 1010 can be implemented using a general-purpose CPU (Central Processing Unit), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of this specification.

[0086] The memory 1020 can be implemented in the form of ROM (Read Only Memory), RAM (Random Access Memory), static storage devices, dynamic storage devices, etc. The memory 1020 can store an operating system and other application programs. When the technical solutions provided in the embodiments of this specification are implemented through software or firmware, the relevant program code is stored in the memory 1020 and is called and executed by the processor 1010.

[0087] The input / output interface 1030 is used to connect input / output modules to implement information input and output. The input / output modules can be configured as components within the device (not shown in the figure) or can be externally connected to the device to provide corresponding functions. Input devices may include a keyboard, mouse, touch screen, microphone, various sensors, etc., and output devices may include a display, speaker, vibrator, indicator light, etc.

[0088] The communication interface 1040 is used to connect to a communication module (not shown) to enable communication between the device and other devices. The communication module can communicate via a wired method (such as USB, network cable, etc.) or a wireless method (such as mobile network, WiFi, Bluetooth, etc.).

[0089] The bus 1050 comprises a path for transmitting information between the various components of the device (eg, the processor 1010 , the memory 1020 , the input / output interface 1030 , and the communication interface 1040 ).

[0090] It should be noted that although the above device only shows the processor 1010, the memory 1020, the input / output interface 1030, the communication interface 1040, and the bus 1050, in a specific implementation, the device may also include other components necessary for normal operation. In addition, it will be understood by those skilled in the art that the above device may only include the components necessary to implement the embodiments of this specification, and does not necessarily include all the components shown in the figures.

[0091] The electronic device of the above embodiment is used to implement the corresponding transition frame generation method in any of the above embodiments, and has the beneficial effects of the corresponding method embodiment, which will not be described in detail here.

[0092] Based on the same inventive concept, corresponding to any of the above-mentioned embodiment methods, the present application also provides a non-transitory computer-readable storage medium, wherein the non-transitory computer-readable storage medium stores computer instructions, and the computer instructions are used to enable the computer to execute the method for generating a transition frame as described in any of the above embodiments.

[0093] The computer-readable media of this embodiment include permanent and non-permanent, removable and non-removable media that can be used to store information by any method or technology. The information can be computer-readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, read-only compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassettes, tape disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device.

[0094] The computer instructions stored in the storage medium of the above embodiment are used to enable the computer to execute the method for generating a transition frame as described in any of the above embodiments, and have the beneficial effects of the corresponding method embodiments, which will not be repeated here.

[0095] It should be understood by those skilled in the art that the discussion of any of the above embodiments is merely illustrative and is not intended to imply that the scope of the present application (including the claims) is limited to these examples. Within the scope of the present application, the technical features in the above embodiments or different embodiments may be combined, the steps may be implemented in any order, and there are many other variations of the different aspects of the embodiments of the present application as described above, which are not provided in detail for the sake of simplicity.

[0096] In addition, for simplicity of description and discussion, and in order not to make the present application embodiment difficult to understand, the known power supply / ground connection with integrated circuit (IC) chip and other components may or may not be shown in the accompanying drawings provided. In addition, the device can be shown in the form of a block diagram to avoid making the present application embodiment difficult to understand, and this also takes into account the following fact, that is, the details of the embodiment of these block diagram devices are highly dependent on the platform to be implemented in the present application embodiment (that is, these details should be fully within the scope of understanding of members of the art). When specific details (for example, circuit) are set forth to describe exemplary embodiments of the present application, it will be apparent to members of the art that embodiments of the present application can be implemented without these specific details or when these specific details are varied. Therefore, these descriptions should be considered to be illustrative rather than restrictive.

[0097] Although the present invention has been described in conjunction with specific embodiments thereof, many alternatives, modifications, and variations of these embodiments will be apparent to members of ordinary skill in the art based on the foregoing description. For example, other memory architectures (e.g., dynamic RAM (DRAM)) may utilize the embodiments discussed.

[0098] The embodiments of the present application are intended to cover all such substitutions, modifications, and variations that fall within the broad scope of the appended claims. Therefore, any omissions, modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the embodiments of the present application should be included in the scope of protection of this application.

Claims

1. A method for generating a transition frame, characterized in that: include: Obtain the style features and keyframe feature matrix of the animation to be produced; Inputting the style features and the feature matrix of the key frames into the trained prediction model to obtain a feature matrix of the transition frames of the animation; Obtaining a transition frame of the animation based on a feature matrix of the transition frame; The method of training the prediction model includes: Setting multiple style features of the animation and determining a feature matrix of sample key frames corresponding to each style feature; The feature matrix of each style feature and its corresponding sample key frame is input into the benchmark model of the prediction model for model training, so that the feature matrix of the transition frame output by the benchmark model during testing corresponds to the feature matrix of the sample key frame.

2. The method according to claim 1, characterized in that The baseline model of the prediction model is GAN, and the generator and discriminator of the GAN are both transformers.

3. The method according to claim 1, characterized in that Inputting the style features and the feature matrix of the key frames into the trained prediction model to obtain the feature matrix of the transition frames of the animation, specifically further comprising: Obtain the target number of transition frames required for the animation; The style features, the feature matrix of the key frames, and the target quantity are input into the trained prediction model to obtain the feature matrix of the transition frames of the animation.

4. The method according to claim 3, characterized in that Inputting the style features, the feature matrix of the key frames, and the target number into the trained prediction model to obtain the feature matrix of the transition frames of the animation, specifically includes: Inputting the style features and the feature matrix of the key frames into the trained prediction model to obtain feature matrices of a preset number of candidate transition frames of the animation; wherein each candidate transition frame has a corresponding sequence number in chronological order; Determine a plurality of arrangement numbers corresponding to the target number, and determine a feature matrix of the transition frame of the animation from a preset number of feature matrices of the candidate transition frames based on the plurality of arrangement numbers; wherein the target number is not greater than the preset number.

5. The method according to claim 1, wherein The feature matrix includes a displacement matrix and a bone rotation matrix.

6. The method according to claim 1, wherein After obtaining the feature matrix of the transition frame of the animation, the method further includes: The characteristic matrix of the transition frame of the animation is adjusted based on a forward dynamics algorithm and an inverse dynamics algorithm.

7. The method according to claim 1, characterized in that The key frames include a beginning key frame and an ending key frame of the animation.

8. A device for generating a transition frame, characterized in that: include: An acquisition module obtains the style features and keyframe feature matrix of the animation to be produced; A model module, inputting the style features and the feature matrix of the key frame into the trained prediction model to obtain the feature matrix of the transition frame of the animation; A generating module, generating a transition frame of the animation based on a feature matrix of the transition frame; The method of training the prediction model includes: Setting multiple style features of the animation and determining a feature matrix of sample key frames corresponding to each style feature; The feature matrix of each style feature and its corresponding sample key frame is input into the benchmark model of the prediction model for model training, so that the feature matrix of the transition frame output by the benchmark model during testing corresponds to the feature matrix of the sample key frame.

9. An electronic device, characterized in that: The method comprises a memory, a processor, and a computer program stored in the memory and executable by the processor, wherein the processor implements the method according to any one of claims 1 to 7 when executing the program.

10. A non-transitory computer-readable storage medium, characterized in that The non-transitory computer-readable storage medium stores computer instructions for causing a computer to execute the method according to any one of claims 1 to 7.

Images