Video generation method and device, computer program product and electronic equipment

Abstract

The invention relates to the technical field of computers, and relates to a video generation method and device, a computer program product and electronic equipment. The video generation method comprises the following steps: acquiring process information of a game; in response to a current touch operation for a target game role of the game, real-time state information of a current frame is generated according to the current touch operation and the process information, and the real-time state information is used for reflecting real-time states of a plurality of event layers in the game; performing feature coding on the real-time state information, and determining a global feature vector of the current frame according to a coding result; and acquiring global picture information of a previous frame of the current frame, and performing picture generation processing based on the global picture information and the global feature vector to generate a target picture of the current frame. According to the method and the device, the game video quality can be improved, the game picture can smoothly display the expected dynamic picture under the real-time operation of the user, and the visual experience of the game is improved.

Description

Technical Field

[0001] The present disclosure relates to the field of computer technology, and more specifically, to a video generating method, a video generating device, a computer program product, and an electronic device. Background Art

[0002] With the development of computer technology, the speed of AI (Artificial Intelligence) video generation has gradually increased, and may even reach the level of real-time video generation, that is, AI video can be rendered at the moment of input, so it is possible to use AI video for real-time interactive game experience. However, the current AI video generation requires a large amount of text to describe the screen content. For the generation of real-time game screens, the game input becomes very lengthy, and it is difficult to ensure that the game screen can smoothly display the expected dynamic screen under the real-time operation of the user, which affects the user's game visual experience to a certain extent.

[0003] It should be noted that the information disclosed in the above background technology section is only used to enhance the understanding of the background of the present disclosure, and therefore may include information that does not constitute the prior art known to ordinary technicians in the field. Summary of the invention

[0004] The purpose of the present disclosure is to provide a video generation method and device, a computer program product and an electronic device, thereby improving the quality of game videos at least to a certain extent, so that the game screen can smoothly display the expected dynamic pictures under the real-time operation of the user, thereby improving the visual experience of the game.

[0005] Other features and advantages of the present disclosure will become apparent from the following detailed description, or may be learned in part by the practice of the present disclosure.

[0006] According to one aspect of the present disclosure, a video generation method is provided, including: obtaining game process information; responding to a current touch operation on a target game character of the game, generating real-time status information of a current frame according to the current touch operation and the process information, the real-time status information being used to reflect the real-time status of several event layers in the game; feature encoding the real-time status information, and determining a global feature vector of the current frame according to the encoding result; obtaining global picture information of a previous frame of the current frame, and performing picture generation processing based on the global picture information and the global feature vector to generate a target picture of the current frame.

[0007] In an exemplary embodiment of the present disclosure, in response to a current touch operation on a target game character of a game, real-time status information of a current frame is generated according to the current touch operation and process information, including: in response to the current touch operation, the current touch operation is stored in a data buffer, the data buffer is used to store the touch operation, wherein the touch operation stored in the data buffer is divided into a plurality of sub-buffers at preset time intervals; if the current touch operation and the touch operation of a previous frame of the current frame are in the same target sub-buffer, the current touch operation is merged with the touch operation in the target sub-buffer to determine the target operation of the current frame; based on the target operation of the current frame and the game process information, real-time status information is generated.

[0008] In an exemplary embodiment of the present disclosure, the method further includes:

[0009] If the current touch operation and the touch operation of the previous frame are in different sub-buffers, the current touch operation is combined with the target operation corresponding to the previous frame to obtain the target operation corresponding to the current frame.

[0010] In an exemplary embodiment of the present disclosure, the process information includes one or more of game logic, environmental scene information, virtual object information in the game, and virtual camera information.

[0011] In an exemplary embodiment of the present disclosure, real-time status information is generated based on the target operation and process information of the current frame, including: determining the current status information of different event layers based on the process information; updating the current status information of each event layer according to the target operation of the current frame, and determining the real-time status information according to the updated current status information of each event layer; wherein the different event layers include at least one of the action status layer of the target game character of the game, the attribute status layer of the target game character, the environment scene status layer of the game, and the attribute status layer of the virtual camera in the game.

[0012] In an exemplary embodiment of the present disclosure, real-time status information is feature encoded, and the global feature vector of the current frame is determined based on the encoding result, including: for each event layer, the real-time status information of the event layer is encoded to obtain a layer encoding result; the layer encoding results corresponding to each event layer are fused to obtain the global feature vector of the current frame.

[0013] In an exemplary embodiment of the present disclosure, the layer coding results corresponding to each event layer are fused to obtain a global feature vector of the current frame, including: performing vector fusion based on the layer coding results corresponding to the event layers with interactive relationships to obtain a first sub-feature vector; performing vector splicing based on the layer coding results of multiple event layers without interactive relationships to obtain a second sub-feature vector; splicing or fusing the first sub-feature vector and the second sub-feature vector to obtain a global feature vector.

[0014] In an exemplary embodiment of the present disclosure, the method also includes: obtaining historical latent space features corresponding to the global picture information of the previous frame of the current frame; fusing the encoding results corresponding to each event layer, and fusing the obtained fusion results with the historical latent space features to determine the global feature vector of the current frame.

[0015] In an exemplary embodiment of the present disclosure, picture generation processing is performed based on global picture information and global feature vectors to generate a target picture of the current frame, including: based on the global feature vector, using a video diffusion model to diffuse historical latent space features to obtain latent space features corresponding to the current frame; and picture generation is performed according to the latent space features corresponding to the current frame to obtain the target picture of the current frame.

[0016] In an exemplary embodiment of the present disclosure, the method further includes: outputting the latent space features corresponding to the target picture and the current frame to control the generation of a picture of the next frame of the current frame.

[0017] According to one aspect of the present disclosure, there is provided a video generating device, comprising: an information acquiring module, for acquiring progress information of a game; a state acquiring module, for responding to a current touch operation on a target game character of the game, and generating real-time state information of a current frame according to the current touch operation and the progress information, wherein the real-time state information is used to reflect the real-time state of several event layers in the game; a feature encoding module, for performing feature encoding on the real-time state information, and determining a global feature vector of the current frame according to the encoding result; and a video generating module, for acquiring global picture information of a frame previous to the current frame, and performing picture generation processing based on the global picture information and the global feature vector to generate a target picture of the current frame.

[0018] According to one aspect of the present disclosure, a computer program product is provided, including a computer program, and when the computer program is executed by a processor, any one of the above methods is implemented.

[0019] According to one aspect of the present disclosure, an electronic device is provided, comprising: a processor; and a memory for storing executable instructions of the processor; wherein the processor is configured to perform any one of the above methods by executing the executable instructions.

[0020] The video generation method in the exemplary embodiment of the present disclosure, on the one hand, obtains the process information of the game, responds to the current touch operation for the target game character of the game, generates the real-time state information of the current frame according to the current touch operation and the process information, can capture the touch operation of the target game character in real time, ensures that each frame of the generated game screen can respond to the touch operation of the target game character in time, so that the game screen can smoothly display the expected dynamic screen under the real-time operation for the target game character, and combines the process information of the game to generate the real-time state information of the current frame, so that the real-time state of each event layer has coherence between video frames. On the other hand, the real-time state information is feature encoded, and the global feature vector of the current frame is determined according to the encoding result, the global picture information of the previous frame of the current frame is obtained, and the picture generation processing is performed based on the global picture information and the global feature vector to generate the target picture of the current frame, and the coherence and logic between the video frames are further increased by combining the global picture information of the previous frame and the global feature vector of the current frame, while generating the real-time game screen, the coherence of the entire video screen is ensured, and the visual experience of the game is improved.

[0021] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] The above and other objects, features and advantages of the exemplary embodiments of the present disclosure will become readily understood by reading the following detailed description with reference to the accompanying drawings, in which several embodiments of the present disclosure are shown in an exemplary and non-limiting manner.

[0023] Figure 1 An application environment according to an exemplary embodiment of the present disclosure is shown.

[0024] Figure 2 A flowchart of a video generating method according to an exemplary embodiment of the present disclosure is shown.

[0025] Figure 3 A flowchart of an implementation method for generating real-time status information of a current frame according to an exemplary embodiment of the present disclosure is shown.

[0026] Figure 4 A flow chart of an implementation method for generating real-time status information according to an exemplary embodiment of the present disclosure is shown.

[0027] Figure 5 A schematic diagram of determining a global feature vector of a current frame according to an exemplary embodiment of the present disclosure is shown.

[0028] Figure 6A flowchart of determining a global feature vector of a current frame according to an exemplary embodiment of the present disclosure is shown.

[0029] Figure 7 A schematic diagram showing the composition of a video generating device according to an exemplary embodiment of the present disclosure is shown.

[0030] Figure 8 A block diagram of an electronic device according to an exemplary embodiment of the present disclosure is shown.

[0031] In the drawings, the same or corresponding reference numerals represent the same or corresponding parts. DETAILED DESCRIPTION

[0032] The exemplary embodiments will now be described more fully with reference to the accompanying drawings. However, the exemplary embodiments can be implemented in a variety of forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided so that the present disclosure will be more comprehensive and complete and the concepts of the exemplary embodiments will be fully conveyed to those skilled in the art. The same reference numerals in the figures represent the same or similar structures, and thus their detailed description will be omitted.

[0033] In addition, the described features, structures or characteristics may be combined in one or more embodiments in any suitable manner. In the following description, many specific details are provided to provide a full understanding of the embodiments of the present disclosure. However, those skilled in the art will appreciate that the technical solutions of the present disclosure may be practiced without one or more of the specific details, or other methods, components, devices, steps, etc. may be adopted. In other cases, known structures, methods, devices, implementations or operations are not shown or described in detail to avoid blurring the various aspects of the present disclosure.

[0034] The block diagrams shown in the accompanying drawings are merely functional entities and do not necessarily correspond to physically independent entities. That is, these functional entities may be implemented in software form, or these functional entities or parts of functional entities may be implemented in one or more software hardened modules, or these functional entities may be implemented in different networks and / or processor devices and / or microcontroller devices.

[0035] There are many AI video generation methods currently available. The technology of using AI video for real-time interactive game experience can quickly generate endless game content without the need for the long preliminary research and development and material production in the traditional way. However, the current AI video generation methods usually require the use of text to describe the screen content, which is very lengthy for game input and difficult to ensure accuracy and real-time performance. In other words, the game screen cannot be highly consistent with the user's touch operation, affecting the user's gaming visual experience.

[0036] It should be noted that the video generation method and device provided by the exemplary embodiments of the present disclosure can be used in the field of game video generation, such as various interactive games that require real-time screen generation, especially action, role-playing and open world games, which can dynamically generate high-quality game screens that meet expectations according to the real-time input of the player. The video generation method and device provided by the exemplary embodiments of the present disclosure can also be used in any field other than the field of game video generation, such as in virtual reality VR and augmented display AR, which can be immediately fed back to the display screen according to the user's operation, in scenes such as driving simulation or flight simulation, by capturing user operations and generating real-time screen instant feedback, the system's sense of reality and response speed are improved, and immersive simulation is achieved. In scenes where movies and interactive story generation are required, different storyline pictures can be generated in real time according to the user's selection and operation, enhancing the user's sense of participation and immersion, and in scenes involving dynamic scene training, such as medical surgery simulation, mechanical operation training, etc., dynamic training pictures can be generated according to the user's input to provide real-time feedback and practice environment. The exemplary embodiments of the present disclosure do not limit the application field of the provided video generation method and device, and the following description is taken as an example of the field of game video generation.

[0037] The video generation method provided by the exemplary embodiment of the present disclosure can be applied to Figure 1 In the application environment shown, the terminal 101 communicates with the server 102 via a network. The data storage system can store data that the server 102 needs to process. The data storage system can be integrated on the server 102, or placed on the cloud or other network servers.

[0038] In an exemplary embodiment, the video generation method provided by the exemplary embodiment of the present disclosure may be executed by the server 102, and the corresponding video generation device is disposed in the server 102. Correspondingly, in this mode of execution by the server 102, the server 102 may start to execute the steps in the technical solution of the exemplary embodiment of the present disclosure in response to a trigger command, wherein the trigger command may be sent by a terminal used by a user, or may be triggered locally by the server in response to some automation events.

[0039] The server 102 may be an independent physical server, a server cluster or a distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms. The server 102 may perform background tasks.

[0040] Furthermore, in another exemplary embodiment, the terminal 101 may also have similar functions to the server 102 , so as to execute the video generating method provided by the exemplary embodiment of the present disclosure.

[0041] The terminal 101 may be a smart phone, a tablet computer, a laptop computer, a desktop computer, a smart speaker, a smart watch, an IoT device, and a portable wearable device. The IoT device may be a smart speaker, a smart TV, a smart air conditioner, and a smart vehicle-mounted device, etc. The portable wearable device may be a smart watch, a smart bracelet, a head-mounted device, etc. The terminal 101 may also be referred to as a mobile terminal, a terminal device, a mobile device, etc. The exemplary embodiments of the present disclosure do not limit the type of the terminal 101.

[0042] In addition, the technical solution of the exemplary embodiment of the present disclosure can also be executed by the terminal 101 and the server 102 in collaboration. In this way of collaborative execution by the terminal 101 and the server 102, some steps in the technical solution provided by the exemplary embodiment of the present disclosure are executed by the terminal 101, and other steps are executed by the server 102.

[0043] It should be noted that in this mode of collaborative execution by the terminal 101 and the server 102, the steps respectively executed by the terminal 101 and the server 102 can be dynamically adjusted according to actual conditions, and there is no special restriction on this.

[0044] The terminal 101 and the server 102 may be connected directly or indirectly via wireless communication, and the exemplary embodiments of the present disclosure do not impose any special limitation thereto.

[0045] In an exemplary embodiment of the present disclosure, a video generation method is provided, referring to Figure 2 FIG. 1 is a flowchart of a video generation method according to an exemplary embodiment of the present disclosure. The video generation method includes steps S210 to S240, which are specifically as follows:

[0046] In step S210, the game progress information is obtained.

[0047] In an exemplary embodiment of the present disclosure, the game process information refers to the basic information content of the game itself, including but not limited to game logic, environmental scene information, virtual object information in the game, virtual camera information in the game, etc. The game process information is updated in real time according to the game process and preset code settings. The process information can be managed by the game back-end processing engine. The processing logic is not essentially different from that of traditional games, which will not be elaborated on.

[0048] When generating real-time status information, the game process information may be obtained through the game backend processing engine. The exemplary embodiments of the present disclosure do not specifically limit the manner of obtaining the process information.

[0049] In step S220, in response to the current touch operation on the target game character of the game, real-time status information of the current frame is generated according to the current touch operation and the process information.

[0050] In the exemplary embodiments of the present disclosure, the current touch operation refers to the operation performed by the user (also referred to as the player) through an input device such as a handle, keyboard, mouse, etc., that is, the touch operation received by the terminal device based on the logged-in game account, including but not limited to the operation of the direction key, function key, joystick, etc., and the current touch operation is obtained by collecting the input signal of the input device. The current touch operation is, for example, the collected input signal or the corresponding touch operation type (such as a jump operation), and there is no special limitation on this.

[0051] Real-time status information is used to reflect the real-time status of several event layers in the game, that is, the real-time status information may include real-time status information corresponding to different event layers, reflecting the dynamic situation of each event layer. The event layer refers to the level of recording different aspects of status information in the game, including but not limited to the action status layer of the target game character controlled by the user, the attribute status layer of the target game character, the game environment scene status layer, and the attribute status layer of the virtual camera in the game. By integrating these different event layers, a complete game screen is obtained.

[0052] By combining the user's current touch operation and the game's progress information, the real-time status of several event layers is generated, providing an information basis for real-time generation of the current frame's game screen.

[0053] In step S230, the real-time state information is feature-coded, and a global feature vector of the current frame is determined according to the coding result.

[0054] In an exemplary embodiment of the present disclosure, when feature encoding is performed on real-time status information, both numerical data and text data can be encoded as feature vectors, and there is no restriction on the encoding method. The global feature vector is an abstract representation that integrates the context information of the current frame and is used to reflect the complete state of the current frame. The context information includes the above-mentioned process information, etc. The global feature vector of the current frame determined according to the encoding result can be obtained by fusing the real-time status information of different event layers. The fusion method is, for example, vector splicing and vector fusion. Vector splicing refers to simply splicing at least two vectors into one vector. Vector fusion can be achieved by fusing vectors into a representative vector by weighted averaging, attention mechanism, etc.

[0055] In step S240, global picture information of a previous frame of the current frame is obtained, and picture generation processing is performed based on the global picture information and the global feature vector to generate a target picture of the current frame.

[0056] In the exemplary embodiment of the present disclosure, the global picture information of the previous frame of the current frame refers to the information containing the complete content and state of the global picture information of the previous frame, which is the basis for generating the game picture of the current frame. The picture generation based on the global picture information and the global feature information can be implemented by using a video diffusion model, and a new image frame (target picture) is generated by gradually back-diffusion based on the diffusion process. The exemplary embodiment of the present disclosure does not limit the type of diffusion model.

[0057] By combining global screen information and global feature vectors for screen generation processing, the continuity between the previous and next frames (current frame and previous frame) can be better captured, and a target screen that conforms to logic and visual expectations can be generated. The target screen can reflect the user's real-time operations and the dynamic changes of the current game.

[0058] The video generation method in the exemplary embodiment of the present disclosure, on the one hand, obtains the progress information of the game, responds to the current touch operation of the user on the target game character of the game, and generates the real-time status information of the current frame according to the current touch operation information and the game progress information of the user in the current game, and can capture the user's touch operation on the target game character in real time, and ensure that each frame of the generated game screen can respond to the user's touch operation on the target game character in a timely manner, so that the game screen can smoothly display the expected dynamic picture under the user's real-time operation on the target game character, and combines the real-time status information of the current frame generated by the user in the current game, so that the real-time status of each event layer has continuity between video frames. On the other hand, the real-time state information is feature encoded, and the global feature vector of the current frame is determined based on the encoding result, the global picture information of the previous frame of the current frame is obtained, and picture generation processing is performed based on the global picture information and the global feature vector to generate the target picture of the current frame. By combining the global picture information of the previous frame and the global feature vector of the current frame, the coherence and logic between video frames are further increased. While generating the real-time game picture, the coherence of the entire video picture is ensured, thereby improving the visual experience of the game.

[0059] In an exemplary embodiment, a method for generating real-time status information of a current frame is provided. Figure 3 As shown, in response to the current touch operation on the target game character of the game, generating the real-time status information of the current frame according to the current touch operation and process information may include:

[0060] Step S310: In response to the current touch operation, store the current touch operation in a data buffer.

[0061] The data buffer is used for touch operation, wherein the touch operation stored in the data buffer is divided into several sub-buffers at preset time intervals. For example, if the 1st to 4th video frames are in the 1st sub-buffer, and the 5th frame is not in the preset time interval, the 5th frame is stored in the 2nd sub-buffer.

[0062] Among them, the current touch operation can be a continuous input type or a discrete input type, such as continuous directional input of the joystick, indicating continuous movement intention, and a single trigger of a button, indicating an instantaneous action or command (such as jumping, attacking). The exemplary embodiments of the present disclosure will capture any type of touch operation.

[0063] Among them, the data buffer is a memory area that temporarily stores touch operation information. For example, if the user enters the "forward" operation in two consecutive frames and immediately presses the "jump" button, these touch operations are stored in the data buffer to provide basic information for subsequent action determination.

[0064] The information processing mechanism of the data buffer is explained, considering that multiple touch operations occurring in a short period of time can be integrated as an action state to reflect the user's complete operation intention. Therefore, the touch operations of the data buffer can be divided into several sub-buffers according to the preset time interval, that is, the touch operation information of the video frame with a time offset difference within the preset range is stored in a sub-buffer content, and a sub-buffer can be determined at each preset time interval starting from the first video frame. For example, the first frame, the second frame, the third frame and the fourth frame are stored in a sub-buffer, and because the frame difference between the fifth frame and the first frame is greater than the preset time interval, the fifth frame is stored in the next sub-buffer. Among them, the preset time interval can be set according to actual needs, and the preset time interval can be determined according to the frame difference between video frames, and there is no restriction on this.

[0065] Optionally, in order to avoid occupying too much memory, the information in the data buffer used to generate the video frame may be cleared during the process of generating the video.

[0066] Step S320: If the current touch operation and the touch operation of the previous frame of the current frame are in the same target sub-buffer, the current touch operation is merged with the touch operation in the target sub-buffer to determine the target operation of the current frame.

[0067] If it is determined according to the preset time interval that the current touch operation falls into the target sub-buffer where the previous frame is located, the touch operations in the target sub-buffer can be merged into a complex action state, that is, the target operation of the current frame.

[0068] Among them, if the operation type of the current touch operation is the same as that of other touch operations in the target sub-buffer, the operation type of the target operation obtained by merging the operations does not change. For example, the current touch operation and other touch operations in the target sub-buffer are both forward moving operations, and the determined target operation is still a forward moving operation, but the action state of the current frame obtained according to the touch operation in the target sub-buffer is a continuous moving state, that is, the target operation is "running forward".

[0069] If the current touch operation is of a different type from other touch operations in the target sub-buffer, operations of different operation types in the target sub-buffer can be merged to obtain a complex operation. For example, the touch operation corresponding to the first frame is a forward move operation, and the current touch operation (corresponding to the second frame) is a jump operation. The target operation of the second frame obtained by merging the operations is to jump forward in place.

[0070] By merging operations in the same sub-buffer into more complex target operations, the user's operation intention can be more completely captured, thereby improving the accuracy of subsequent generation of game screens.

[0071] The target determination operation is described in detail below with reference to a specific example.

[0072] In Example 1, the user first inputs a forward operation in frame 1, and obtains the current touch operation and stores it in sub-buffer 1. In the process of processing frame 1, the "forward" operation is temporarily identified as the action state of "prepare to move", but it will not be executed immediately, indicating that the target game character controlled by the user is in a state of preparing to move forward.

[0073] In the second frame, the user inputs a "jump" operation (such as pressing the X key), and the current touch operation of the second frame is stored in sub-buffer 1 within a preset time interval between the second frame and the first frame. Since the touch operations corresponding to the second frame and the first frame are both in sub-buffer 1, the current touch operation "jump" and the touch operation "forward" of the first frame are combined, that is, the combined operation of "forward + jump" determines a complex action "fly forward in place", and then determines that the target operation of the current frame (the second frame) is "fly forward in place".

[0074] In some possible embodiments, if the current touch operation and the touch operation of the previous frame are in different sub-buffers, the current touch operation is combined with the target operation corresponding to the previous frame to obtain the target operation corresponding to the current frame.

[0075] The target operation is described in detail below with a specific example.

[0076] In Example 2, the user first inputs a forward operation in frame 1, and obtains the current touch operation and stores it in sub-buffer 1. In the process of processing frame 1, the "forward" operation is temporarily identified as the action state of "prepare to move", but it will not be executed immediately, indicating that the target game character controlled by the user is in a state of preparing to move forward.

[0077] From the 2nd frame to the 4th frame, the user did not input a new type of operation, and the 2nd frame to the 4th frame are all within the first preset time interval. Then, when processing each frame from the 2nd frame to the 4th frame to obtain its target operation, the touch operation of the current frame is merged with the touch operation in the sub-buffer 1, that is, the "forward" operation in the sub-buffer 1 will be gradually processed into an actual moving action. It can be understood that from the 1st frame to the 4th frame, the touch operations corresponding to several consecutive video frames whose time offset difference is within the preset range are merged, and the touch operation in the sub-buffer 1 is always the "forward" operation. Then, the "forward" operation is processed as the continuous moving state of the target game character, that is, the target operation of the 4th frame can be determined as "running forward", and of course, the target operations corresponding to the 2nd and 3rd frames are also "running forward".

[0078] In the 5th frame, the user inputs a "jump" operation (such as pressing the X key). Since the 5th frame exceeds the first preset time interval, the current touch operation corresponding to the 5th frame will be stored in another sub-buffer (such as sub-buffer 2), which is not in the same sub-buffer as the touch operation of the 4th frame.

[0079] Then, the target operation corresponding to the previous frame of the 5th frame is obtained, that is, the target operation corresponding to the 4th frame is "run forward", then the "jump" operation is combined with the "run forward" operation to obtain a complex action "run forward and jump", and then the target operation of the current frame (5th frame) is determined to be "run forward and jump". Similarly, when the user inputs the touch operation of the 6th frame, it is determined whether the touch operation corresponding to the 6th frame is stored in the sub-buffer 2, and it can be processed according to the above-mentioned similar operation flow, which will not be repeated here.

[0080] From the above two examples, it can be seen that the target operation of the current frame can be determined based on whether the current touch operation and the touch operation of the previous frame of the current frame are in the same sub-buffer. The current touch operation and the touch operation of the previous frame of the current frame are in the same target sub-buffer. Regardless of whether these touch operations indicate the same type of touch operation, the touch operations in the target sub-buffer are merged to generate a complex action as the target operation of the current frame. If the current touch operation and the touch operation of the previous frame are in different sub-buffers, the current touch operation of the current frame and the target operation corresponding to the previous frame are processed separately, and the current touch operation is combined with the target operation corresponding to the previous frame to obtain the target operation of the current frame. The reason for combining the current touch operation with the target operation corresponding to the previous frame is to ensure the continuity of the action and the picture between the previous and next frames.

[0081] By adopting a caching mechanism, the user's operating intentions can be fully captured, thereby providing accurate operating information for generating video images, ensuring that the game screen can smoothly display expected dynamic images under the user's real-time operation.

[0082] Step S330: Generate real-time status information based on the target operation and process information of the current frame.

[0083] As mentioned above, the game process information refers to the basic information content of the current game itself, and the process information is updated in real time according to the game process and preset code settings. In other words, the content of the game screen is not only related to the user's touch operation, but also related to the basic information content of the game itself, such as the game screen is adjusted with the game logic, the game screen is adjusted with the scene change, etc. Therefore, after obtaining the target operation of the current frame, real-time status information can be generated based on the target operation and process information of the current frame, which will provide the real-time status of each event in the game to generate an accurate real-time game screen.

[0084] In an exemplary embodiment, a method for generating real-time status information is provided. Figure 4 Generating real-time status information based on target operations and game information may include:

[0085] Step S410: Determine current status information of different event layers based on the process information.

[0086] Step S420: updating the current status information of each event layer according to the target operation, and determining the real-time status information according to the updated current status information of each event layer.

[0087] The different event layers include at least one of the action state layer of the target game character, the attribute state layer of the target game character, the environment scene state layer of the game, and the attribute state layer of the virtual camera in the game. The process information can be obtained through the game backend processing engine, and the current state information of each event layer can be determined according to the process information.

[0088] Exemplarily, the action state information of the virtual object can be generated according to the target operation of the current frame and the game logic information, the character state information can be determined according to the target operation of the current frame and the current attributes of the virtual object (such as health value, position, direction, etc.), and the scene state information (such as lighting changes, scene element movement, etc.) can be determined according to the target operation of the current frame and the game environment setting. In addition, the virtual camera state can be further determined according to the scene state information and the character state information, for example, the angle, position and focal length of the virtual camera are determined. The current state information involved above includes the action state layer (such as walking, running, jumping, etc.), the character attribute layer (such as position, direction, expression, etc.), the environment scene state layer (such as lighting, environmental changes, etc.) and the virtual camera state layer (such as viewing angle, position, focal length, etc.). Of course, the content included in the process information is not limited to this, and the current state information of different event layers obtained is also not limited to this, and they are not listed one by one.

[0089] According to the target operation of the current frame combined with the current state information of different event layers, the real-time state information corresponding to the current frame can be obtained, which is used to generate the conditions for guiding the generation of the picture and ensure the real-time accuracy of the picture generation.

[0090] In an exemplary embodiment, a method for determining a global feature vector of a current frame is provided. Figure 5 As shown, the real-time state information is feature encoded, and the global feature vector of the current frame is determined according to the encoding result, which may include:

[0091] For each event layer, the real-time state information of the event layer is encoded to obtain a layer encoding result; then the layer encoding results corresponding to each event layer are fused to obtain a global feature vector of the current frame.

[0092] Specifically, each event layer (such as action parameters, scene status information, etc.) can be processed by a numerical encoder or a text encoder and converted into a feature vector. For example, converting numerical data such as the character's position information and action speed into a feature vector, or converting text data such as scene descriptions and virtual camera instructions into a feature vector, without specific restrictions. The encoded feature vector retains the key information of each event layer, can reflect the specific status of the corresponding event layer, and is ready for further fusion processing. Figure 5As shown, the layer encoding results of the action layer, the character layer, the environment scene layer and the virtual camera layer can be fused to obtain a global feature vector.

[0093] In an exemplary embodiment, if Figure 6 As shown, the layer encoding results corresponding to each event layer are fused to obtain the global feature vector of the current frame, which may include:

[0094] Step S610: performing vector fusion based on the layer encoding results corresponding to the event layers with interactive relationships to obtain a first sub-feature vector.

[0095] There are interactive relationships between some event layers, such as the relationship between the environment scene layer and the virtual camera layer. The content of the environment scene is related to the position and focal length of the virtual camera. Based on this, the layer encoding results corresponding to the event layers with interactive relationships can be vector fused into a representative vector. The correlation between the layers is ensured through vector fusion, which is conducive to accurately capturing the overall context.

[0096] Among them, vector fusion can be performed by weighted averaging the layer encoding results of each event layer, or by using a self-attention mechanism, etc., and there is no restriction on this.

[0097] Step S620: performing vector concatenation based on the layer encoding results corresponding to multiple event layers without interactive relationship to obtain a second sub-feature vector.

[0098] Multiple event layers without interactive relationships refer to these event layers being independent state layers, for example, there is no interaction or little interaction between the action layer and the role layer. Based on this, these event layers can be directly spliced ​​to obtain a comprehensive vector.

[0099] Step S630: concatenate or merge the first sub-feature vector and the second sub-feature vector to obtain a global feature vector.

[0100] After obtaining the first sub-feature vector and the second sub-feature vector, the two are concatenated or merged, which can be selected according to actual needs and is not limited.

[0101] By fusing the layer coding results of each event layer based on whether there is an interactive relationship, the global feature vector can accurately reflect the status and picture content of the current frame at each layer.

[0102] Optionally, after the first sub-feature vector and the second sub-feature vector are concatenated or fused, the obtained comprehensive features can be further subjected to high-level feature abstraction and processing through a fully connected layer (or multi-layer perceptron layer) to obtain a global feature vector.

[0103] Among them, the comprehensive features can be compressed into a global feature vector with a fixed length through MLP (Multilayer Perceptron), which not only avoids the computational overhead caused by the overly long global feature vector, but also ensures the correlation of each event layer and the overall context information, and can also use the input requirements of the subsequent video diffusion model.

[0104] Based on the aforementioned exemplary embodiments, in order to further ensure the logical coherence between frames, the following may also be included:

[0105] Obtain the historical latent space features corresponding to the global image information of the previous frame of the current frame;

[0106] The encoding results corresponding to each event layer are fused, and the obtained fusion result is fused with the historical latent space features to determine the global feature vector of the current frame.

[0107] The historical latent space features corresponding to the global image information of the previous frame contain important features of the previous frame and can reflect the game state of the previous frame. The encoding results corresponding to each event layer are fused with the historical latent space features of the previous frame. It can be vector concatenation or fusion, and there is no restriction on this. The generation method of the historical latent space features of the previous frame can refer to the implementation method of generating the latent space features of the current frame later.

[0108] Based on this, in the process of generating the current frame of the game screen, the global features of the previous frame of the current frame are also involved, which improves the logical and visual coherence between frames and makes the game screen smoother and more natural.

[0109] In an exemplary embodiment, a method for generating a target picture of a current frame is also provided. The method may include:

[0110] Based on the global feature vector, the video diffusion model is used to diffuse the historical latent space features to obtain the latent space features corresponding to the current frame;

[0111] The image is generated according to the latent space features corresponding to the current frame to obtain the target image of the current frame.

[0112] Among them, the latent space features corresponding to the current frame represent the latent features of the current frame. The processing process of the video diffusion model includes a forward process and a reverse process. It is a generative model that starts from noise data, gradually adds noise, and generates the target picture through a learning denoising process. Based on the global feature vector, the video diffusion model is used to diffuse the historical latent space features. In fact, the global feature vector is used as the conditional input when the video diffusion model is processed. The historical latent space features are diffused based on the input conditions to obtain the latent space features corresponding to the current frame. In this process, not only the input conditions (global feature vector) of the current frame are considered, but also the latent space features of the previous frame are combined to ensure the coherence of the generated picture.

[0113] It should be noted that the exemplary embodiments of the present disclosure do not limit the specific structure and processing method of the video diffusion model. It only needs to ensure that the basic principle architecture of the video diffusion model can be combined with the exemplary embodiments of the present disclosure. However, it is necessary to ensure that the global feature vector is used as the conditional input when processing the video diffusion model, and the historical latent space features are diffused based on the input conditions. No feature restrictions are imposed on other parts.

[0114] Optionally, before the global feature vector is input into the video diffusion model, the global feature vector may be preprocessed to ensure that it can be used efficiently and accurately by the video diffusion model.

[0115] For example, the global feature vector can be normalized to adjust data of different dimensions to the same scale, because information of different event layers (such as actions, roles, scenes, etc.) may have different value ranges. Standardization can avoid the bias of the model on certain dimensions, thereby ensuring that the weights of all input dimensions in the video diffusion model are consistent. For another example, the values ​​in the global feature vector can be normalized to compress the values ​​in the vector to a fixed range (such as [0,1] or [-1,1]), thereby avoiding problems such as gradient disappearance or explosion caused by inputting too large or too small values ​​into the video diffusion model.

[0116] In addition, if the dimension of the global feature vector does not match the input requirement of the A video diffusion model, the dimension of the global feature vector may be further adjusted, such as by performing linear transformation, dimensionality reduction, and dimensionality increase.

[0117] Among them, linear transformation refers to transforming the dimension of the global feature vector to the dimension required by the video diffusion model through a fully connected layer (or linear layer), which can be completed by matrix multiplication to adjust the dimension size of the global feature vector. Dimensionality increase means that in some cases, the video diffusion model may require a higher dimensional feature vector, and the dimension of the feature vector can be expanded by adding redundant information or adding additional features (such as random noise). Dimensionality reduction means that if the vector dimension is too high, the global feature vector can be reduced in dimension through methods such as principal component analysis (PCA) or auto encoder to retain the most representative features. Of course, other dimensionality processing methods can also be selected according to actual needs, and there is no restriction on this.

[0118] After obtaining the latent space features corresponding to the current frame, the latent space features can be used to further generate a target picture of the current frame through a video diffusion model, and the target picture can reflect the real-time operation of the user and the dynamic changes of the game information.

[0119] In an exemplary embodiment, the latent space features corresponding to the target picture and the current picture can also be output to control the generation of the next frame of the current frame. Specifically, after the target picture is generated, the latent space features of the target picture and the current frame can be output. The latent space features of the current frame contain key information of the current frame, which can ensure a natural and smooth transition between pictures when subsequent consecutive frames are generated. The target picture will be displayed to the user in real time in the game.

[0120] In an exemplary embodiment of the present disclosure, during the generation process of each frame of the game screen, the latent space features of the previous frame and the conditions of the current frame (global feature vector) are combined to generate the game screen frame by frame, and the video diffusion model is used to ensure that the generated game screen has coherence and visual consistency, so that the game can smoothly display the expected dynamic pictures under the real-time operation of the player.

[0121] The video generation method in the exemplary embodiment of the present disclosure, on the one hand, obtains the process information of the game, responds to the current touch operation for the target game character of the game, generates the real-time state information of the current frame according to the current touch operation and the process information, can capture the touch operation of the target game character in real time, ensures that each frame of the generated game screen can respond to the touch operation of the target game character in time, so that the game screen can smoothly display the expected dynamic screen under the real-time operation for the target game character, and combines the process information of the game to generate the real-time state information of the current frame, so that the real-time state of each event layer has coherence between video frames. On the other hand, the real-time state information is feature encoded, and the global feature vector of the current frame is determined according to the encoding result, the global picture information of the previous frame of the current frame is obtained, and the picture generation processing is performed based on the global picture information and the global feature vector to generate the target picture of the current frame, and the coherence and logic between the video frames are further increased by combining the global picture information of the previous frame and the global feature vector of the current frame, while generating the real-time game screen, the coherence of the entire video screen is ensured, and the visual experience of the game is improved.

[0122] In an exemplary embodiment of the present disclosure, a video generating device is also provided. Figure 7 As shown, the video generation device 700 may include an information acquisition module 710, a state acquisition module 720, a feature encoding module 730 and a video generation module 740. Specifically:

[0123] The information acquisition module 710 is used to obtain the process information of the game; the state acquisition module 720 is used to respond to the current touch operation on the target game character of the game, and generate the real-time state information of the current frame according to the current touch operation and the process information, and the real-time state information is used to reflect the real-time state of several event layers in the game; the feature encoding module 730 is used to feature encode the real-time state information and determine the global feature vector of the current frame according to the encoding result; the video generation module 740 is used to obtain the global screen information of the previous frame of the current frame, and perform screen generation processing based on the global screen information and the global feature vector to generate the target screen of the current frame.

[0124] In an exemplary embodiment of the present disclosure, the state acquisition module 720 is configured to perform: in response to the current touch operation, storing the current touch operation in a data buffer, the data buffer being used to store touch operations, wherein the touch operations stored in the data buffer are divided into a plurality of sub-buffers at preset time intervals; if the current touch operation and the touch operation of the previous frame of the current frame are in the same target sub-buffer, merging the current touch operation with the touch operation in the target sub-buffer to determine the target operation of the current frame; and generating the real-time state information based on the target operation of the current frame and the game process information.

[0125] In an exemplary embodiment of the present disclosure, the state acquisition module 720 is also configured to execute: if the current touch operation and the touch operation of the previous frame are in different sub-buffers, the current touch operation is combined with the target operation corresponding to the previous frame to obtain the target operation corresponding to the current frame.

[0126] In an exemplary embodiment of the present disclosure, the process information includes one or more of game logic, environmental scene information, virtual object information in the game, and virtual camera information.

[0127] In an exemplary embodiment of the present disclosure, the state acquisition module 720 is configured to perform: determining the current state information of different event layers based on process information; updating the current state information of each event layer according to the target operation of the current frame, and determining the real-time state information based on the updated current state information of each event layer; wherein the different event layers include at least one of the action state layer of the target game character of the game, the attribute state layer of the target game character, the environment scene state layer of the game, and the attribute state layer of the virtual camera in the game.

[0128] In an exemplary embodiment of the present disclosure, feature encoding 730 is configured to perform: for each event layer, encoding the real-time status information of the event layer to obtain a layer encoding result; fusing the layer encoding results corresponding to each event layer to obtain a global feature vector of the current frame.

[0129] In an exemplary embodiment of the present disclosure, feature encoding 730 is configured to perform: vector fusion based on layer encoding results corresponding to event layers with interactive relationships to obtain a first sub-feature vector; vector splicing based on layer encoding results corresponding to multiple event layers without interactive relationships to obtain a second sub-feature vector; splicing or fusing the first sub-feature vector and the second sub-feature vector to obtain a global feature vector.

[0130] In an exemplary embodiment of the present disclosure, feature encoding 730 is configured to perform: obtaining historical latent space features corresponding to global picture information of a previous frame of a current frame; fusing the encoding results corresponding to each event layer, and fusing the obtained fusion results with the historical latent space features to determine a global feature vector of the current frame.

[0131] In an exemplary embodiment of the present disclosure, the video generation module 740 is configured to perform: based on the global feature vector, using the video diffusion model to diffuse the historical latent space features to obtain the latent space features corresponding to the current frame; and generating a picture according to the latent space features corresponding to the current frame to obtain the target picture of the current frame.

[0132] In an exemplary embodiment of the present disclosure, the video generation module 740 is further configured to execute: outputting the latent space features corresponding to the target picture and the current frame to control the generation of the picture of the next frame of the current frame.

[0133] Since the details of the functional modules of the video generating device of the exemplary embodiment of the present disclosure have been described in the exemplary embodiment of the video generating method described above, they will not be repeated here.

[0134] It should be noted that although several modules or units of the video generation device are mentioned in the above detailed description, this division is not mandatory. In fact, according to the embodiments of the present disclosure, the features and functions of two or more modules or units described above can be embodied in one module or unit. Conversely, the features and functions of one module or unit described above can be further divided into multiple modules or units to be embodied.

[0135] The exemplary embodiments of the present disclosure further provide a computer program product, which includes a computer program, and when the computer program is executed by a processor, the above-mentioned video generation method is implemented.

[0136] In one embodiment, the computer program product may be a tangible product containing a computer program, such as a computer-readable storage medium storing a computer program. The readable storage medium may be a storage medium based on electrical, magnetic, optical, electromagnetic, infrared, or other signals, including but not limited to: random access memory (RAM), read-only memory (ROM), magnetic tape, floppy disk, flash memory (Flash), mechanical hard disk (HDD), solid-state drive (SSD), and the like. Exemplarily, the computer program product may be implemented as a non-volatile storage medium storing a computer program, such as a read-only memory, a NAND flash memory, and the like.

[0137] In one embodiment, the computer program product may be an intangible product including a computer program. Exemplarily, the computer program product may be implemented as a virtual digital product, such as a digital file storing an executable file, an installation package, etc. of the computer program.

[0138] The code of the computer program can be written in one or more programming languages. Programming languages ​​such as C language, Java, C++, etc. The program code can be executed entirely on the user computing device, or partially on the user computing device, or as a separate software package, or partially on the user computing device and partially on a remote computing device, or entirely on a remote computing device or server. In the case of a remote computing device, the remote computing device can be connected to the user computing device through any type of network, such as a local area network (LAN), a wide area network (WAN), etc., or can be connected to an external computing device (e.g., an Internet connection provided by an operator).

[0139] Computer programs can be carried or transmitted by electrical, magnetic, optical, electromagnetic, infrared and other signals. Electronic devices can convert signals carrying computer programs into digital signals to run computer programs. When a computer program is running on an electronic device, its code is used to enable the electronic device to execute (more specifically, the processor of the electronic device can be executed) the method steps of various exemplary embodiments of the present disclosure, such as the above-mentioned video generation method, which includes the following steps: obtaining the process information of the game; responding to the current touch operation for the target game character of the game, generating the real-time state information of the current frame according to the current touch operation and process information, and the real-time state information is used to reflect the real-time state of several event layers in the game; feature encoding the real-time state information, and determining the global feature vector of the current frame according to the encoding result; obtaining the global picture information of the previous frame of the current frame, and performing picture generation processing based on the global picture information and the global feature vector to generate the target picture of the current frame.

[0140] The above method steps are executed by a computer program. On the one hand, the process information of the game is obtained, the current touch operation for the target game character of the game is responded to, and the real-time state information of the current frame is generated according to the current touch operation and the process information, so that the touch operation for the target game character can be captured in real time, ensuring that each frame of the generated game screen can respond to the touch operation for the target game character in a timely manner, so that the game screen can smoothly display the expected dynamic screen under the real-time operation for the target game character, and the real-time state information of the current frame is generated in combination with the process information of the game, so that the real-time state of each event layer has coherence between video frames. On the other hand, the real-time state information is feature encoded, and the global feature vector of the current frame is determined according to the encoding result, the global picture information of the previous frame of the current frame is obtained, and the picture generation processing is performed based on the global picture information and the global feature vector to generate the target picture of the current frame, and the coherence and logic between the video frames are further increased by combining the global picture information of the previous frame and the global feature vector of the current frame, while generating the real-time game screen, the coherence of the entire video screen is ensured, and the visual experience of the game is improved.

[0141] In an exemplary embodiment of the present disclosure, in response to a current touch operation on a target game character of a game, real-time status information of a current frame is generated according to the current touch operation and process information, including: in response to the current touch operation, the current touch operation is stored in a data buffer, the data buffer is used to store the touch operation, wherein the touch operation stored in the data buffer is divided into a plurality of sub-buffers at preset time intervals; if the current touch operation and the touch operation of a previous frame of the current frame are in the same target sub-buffer, the current touch operation is merged with the touch operation in the target sub-buffer to determine the target operation of the current frame; based on the target operation of the current frame and the game process information, real-time status information is generated.

[0142] In an exemplary embodiment of the present disclosure, the method further includes:

[0143] If the current touch operation and the touch operation of the previous frame are in different sub-buffers, the current touch operation is combined with the target operation corresponding to the previous frame to obtain the target operation corresponding to the current frame.

[0144] In an exemplary embodiment of the present disclosure, the process information includes one or more of game logic, environmental scene information, virtual object information in the game, and virtual camera information.

[0145] In an exemplary embodiment of the present disclosure, real-time status information is generated based on the target operation and process information of the current frame, including: determining the current status information of different event layers based on the process information; updating the current status information of each event layer according to the target operation of the current frame, and determining the real-time status information according to the updated current status information of each event layer; wherein the different event layers include at least one of the action status layer of the target game character of the game, the attribute status layer of the target game character, the environment scene status layer of the game, and the attribute status layer of the virtual camera in the game.

[0146] In an exemplary embodiment of the present disclosure, real-time status information is feature encoded, and the global feature vector of the current frame is determined based on the encoding result, including: for each event layer, the real-time status information of the event layer is encoded to obtain a layer encoding result; the layer encoding results corresponding to each event layer are fused to obtain the global feature vector of the current frame.

[0147] In an exemplary embodiment of the present disclosure, the layer coding results corresponding to each event layer are fused to obtain a global feature vector of the current frame, including: performing vector fusion based on the layer coding results corresponding to the event layers with interactive relationships to obtain a first sub-feature vector; performing vector splicing based on the layer coding results of multiple event layers without interactive relationships to obtain a second sub-feature vector; splicing or fusing the first sub-feature vector and the second sub-feature vector to obtain a global feature vector.

[0148] In an exemplary embodiment of the present disclosure, the method also includes: obtaining historical latent space features corresponding to the global picture information of the previous frame of the current frame; fusing the encoding results corresponding to each event layer, and fusing the obtained fusion results with the historical latent space features to determine the global feature vector of the current frame.

[0149] In an exemplary embodiment of the present disclosure, picture generation processing is performed based on global picture information and global feature vectors to generate a target picture of the current frame, including: based on the global feature vector, using a video diffusion model to diffuse historical latent space features to obtain latent space features corresponding to the current frame; and picture generation is performed according to the latent space features corresponding to the current frame to obtain the target picture of the current frame.

[0150] In an exemplary embodiment of the present disclosure, the method further includes: outputting the latent space features corresponding to the target picture and the current frame to control the generation of a picture of the next frame of the current frame.

[0151] In addition, in an exemplary embodiment of the present disclosure, an electronic device capable of implementing the above method is also provided. It will be appreciated by those skilled in the art that various aspects of the present disclosure may be implemented as a system, method, or program product. Therefore, various aspects of the present disclosure may be specifically implemented in the following forms, namely: a complete hardware embodiment, a complete software embodiment (including firmware, microcode, etc.), or an embodiment combining hardware and software aspects, which may be collectively referred to herein as a "circuit", "module", or "system".

[0152] Refer to the following Figure 8 800 according to this embodiment of the present disclosure is described. Figure 8 The electronic device 800 shown is merely an example and should not bring any limitation to the functions and scope of use of the embodiments of the present disclosure.

[0153] like Figure 8 As shown, the electronic device 800 is in the form of a general computing device. The components of the electronic device 800 may include, but are not limited to: the at least one processing unit 810, the at least one storage unit 820, a bus 830 connecting different system components (including the storage unit 820 and the processing unit 810), and a display unit 840.

[0154] The storage unit stores a program code, and the program code can be executed by the processing unit 810, so that the processing unit 810 executes the steps described in the above "Exemplary Method" section of this specification according to various exemplary embodiments of the present disclosure. For example, the processing unit 810 can execute the following steps: obtaining the process information of the game; responding to the current touch operation for the target game character of the game, generating the real-time state information of the current frame according to the current touch operation and the process information, and the real-time state information is used to reflect the real-time state of several event layers in the game; feature encoding the real-time state information, and determining the global feature vector of the current frame according to the encoding result; obtaining the global screen information of the previous frame of the current frame, and performing screen generation processing based on the global screen information and the global feature vector to generate the target screen of the current frame.

[0155] In an exemplary embodiment of the present disclosure, in response to a current touch operation on a target game character of a game, real-time status information of a current frame is generated according to the current touch operation and process information, including: in response to the current touch operation, the current touch operation is stored in a data buffer, the data buffer is used to store the touch operation, wherein the touch operation stored in the data buffer is divided into a plurality of sub-buffers at preset time intervals; if the current touch operation and the touch operation of a previous frame of the current frame are in the same target sub-buffer, the current touch operation is merged with the touch operation in the target sub-buffer to determine the target operation of the current frame; based on the target operation of the current frame and the game process information, real-time status information is generated.

[0156] In an exemplary embodiment of the present disclosure, the method further includes:

[0157] If the current touch operation and the touch operation of the previous frame are in different sub-buffers, the current touch operation is combined with the target operation corresponding to the previous frame to obtain the target operation corresponding to the current frame.

[0158] In an exemplary embodiment of the present disclosure, the process information includes one or more of game logic, environmental scene information, virtual object information in the game, and virtual camera information.

[0159] In an exemplary embodiment of the present disclosure, real-time status information is generated based on the target operation and process information of the current frame, including: determining the current status information of different event layers based on the process information; updating the current status information of each event layer according to the target operation of the current frame, and determining the real-time status information according to the updated current status information of each event layer; wherein the different event layers include at least one of the action status layer of the target game character of the game, the attribute status layer of the target game character, the environment scene status layer of the game, and the attribute status layer of the virtual camera in the game.

[0160] In an exemplary embodiment of the present disclosure, real-time status information is feature encoded, and the global feature vector of the current frame is determined based on the encoding result, including: for each event layer, the real-time status information of the event layer is encoded to obtain a layer encoding result; the layer encoding results corresponding to each event layer are fused to obtain the global feature vector of the current frame.

[0161] In an exemplary embodiment of the present disclosure, the layer coding results corresponding to each event layer are fused to obtain a global feature vector of the current frame, including: performing vector fusion based on the layer coding results corresponding to the event layers with interactive relationships to obtain a first sub-feature vector; performing vector splicing based on the layer coding results of multiple event layers without interactive relationships to obtain a second sub-feature vector; splicing or fusing the first sub-feature vector and the second sub-feature vector to obtain a global feature vector.

[0162] In an exemplary embodiment of the present disclosure, the method also includes: obtaining historical latent space features corresponding to the global picture information of the previous frame of the current frame; fusing the encoding results corresponding to each event layer, and fusing the obtained fusion results with the historical latent space features to determine the global feature vector of the current frame.

[0163] In an exemplary embodiment of the present disclosure, picture generation processing is performed based on global picture information and global feature vectors to generate a target picture of the current frame, including: based on the global feature vector, using a video diffusion model to diffuse historical latent space features to obtain latent space features corresponding to the current frame; and picture generation is performed according to the latent space features corresponding to the current frame to obtain the target picture of the current frame.

[0164] In an exemplary embodiment of the present disclosure, the method further includes: outputting the latent space features corresponding to the target picture and the current frame to control the generation of a picture of the next frame of the current frame.

[0165] The storage unit 820 may include a readable medium in the form of a volatile storage unit, such as a random access memory unit (RAM) 821 and / or a cache memory unit 822 , and may further include a read-only memory unit (ROM) 823 .

[0166] The storage unit 820 may also include a program / utility 824 having a set (at least one) of program modules 825, such program modules 825 including but not limited to: an operating system, one or more application programs, other program modules, and program data, each of which or some combination may include an implementation of a network environment.

[0167] Bus 830 may represent one or more of several types of bus structures, including a memory unit bus or memory unit controller, a peripheral bus, an accelerated graphics port, a processing unit, or a local bus using any of a variety of bus architectures.

[0168] The electronic device 800 may also communicate with one or more external devices 900 (e.g., keyboards, pointing devices, Bluetooth devices, etc.), may also communicate with one or more devices that enable a user to interact with the electronic device 800, and / or communicate with any device that enables the electronic device 800 to communicate with one or more other computing devices (e.g., routers, modems, etc.). Such communication may be performed via an input / output (I / O) interface 850. Furthermore, the electronic device 800 may also communicate with one or more networks (e.g., local area networks (LANs), wide area networks (WANs), and / or public networks, such as the Internet) via a network adapter 860. As shown, the network adapter 860 communicates with other modules of the electronic device 800 via a bus 830. It should be understood that, although not shown in the figure, other hardware and / or software modules may be used in conjunction with the electronic device 800, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems, etc.

[0169] Through the description of the above embodiments, it is easy for those skilled in the art to understand that the example embodiments described here can be implemented by software, or by software combined with necessary hardware. Therefore, the technical solution according to the embodiment of the present disclosure can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash drive, a mobile hard disk, etc.) or on a network, and includes several instructions to enable a computing device (which can be a personal computer, a server, a terminal device, or a network device, etc.) to execute the method according to the embodiment of the present disclosure.

[0170] In addition, the above-mentioned figures are only schematic illustrations of the processes included in the method according to the exemplary embodiments of the present disclosure, and are not intended to be limiting. It is easy to understand that the processes shown in the above-mentioned figures do not indicate or limit the time sequence of these processes. In addition, it is also easy to understand that these processes can be performed synchronously or asynchronously, for example, in multiple modules.

[0171] Those skilled in the art will readily appreciate other embodiments of the present disclosure after considering the specification and practicing the invention disclosed herein. The present disclosure is intended to cover any variations, uses, or adaptations of the present disclosure that follow the general principles of the present disclosure and include common knowledge or customary technical means in the art that are not disclosed in the present disclosure. The description and examples are to be considered exemplary only, and the true scope and spirit of the present disclosure are indicated by the claims.

Claims

1. A video generation method, characterized in that: include: Get the game's progress information; In response to a current touch operation on a target game character of the game, generating real-time status information of a current frame according to the current touch operation and the process information, wherein the real-time status information is used to reflect the real-time status of several event layers in the game; Performing feature encoding on the real-time state information, and determining a global feature vector of the current frame according to the encoding result; The global picture information of the previous frame of the current frame is acquired, and picture generation processing is performed based on the global picture information and the global feature vector to generate a target picture of the current frame.

2. The method according to claim 1, characterized in that The response is directed to a current touch operation of a target game character of the game, and generates real-time status information of a current frame according to the current touch operation and the process information, including: In response to the current touch operation, the current touch operation is stored in a data buffer, the data buffer is used to store the touch operation, wherein the touch operation stored in the data buffer is divided into a plurality of sub-buffers at a preset time interval; If the current touch operation and the touch operation of the previous frame of the current frame are in the same target sub-buffer, merging the current touch operation with the touch operation in the target sub-buffer to determine the target operation of the current frame; The real-time status information is generated based on the target operation of the current frame and the process information.

3. The method according to claim 2, characterized in that The method further comprises: If the current touch operation and the touch operation of the previous frame are in different sub-buffers, the current touch operation is combined with the target operation corresponding to the previous frame to obtain the target operation corresponding to the current frame.

4. The method according to claim 2 or 3, characterized in that: The process information includes one or more of game logic, environmental scene information, virtual object information in the game, and virtual camera information.

5. The method according to claim 4, characterized in that The generating the real-time status information based on the target operation of the current frame and the process information includes: Determining current state information of different event layers based on the process information; updating the current state information of each of the event layers respectively according to the target operation of the current frame, and determining the real-time state information according to the updated current state information of each of the event layers; Among them, the different event layers include at least one of the action state layer of the target game character of the game, the attribute state layer of the target game character, the environment scene state layer of the game, and the attribute state layer of the virtual camera in the game.

6. The method according to claim 1, characterized in that The feature encoding of the real-time state information and determining the global feature vector of the current frame according to the encoding result includes: For each event layer, encoding the real-time status information of the event layer to obtain a layer encoding result; The layer encoding results corresponding to each of the event layers are fused to obtain a global feature vector of the current frame.

7. The method according to claim 6, characterized in that The step of fusing the layer encoding results corresponding to each of the event layers to obtain the global feature vector of the current frame includes: Perform vector fusion based on the layer encoding results corresponding to the event layer with interactive relationship to obtain the first sub-feature vector; Perform vector concatenation based on the layer encoding results corresponding to multiple event layers without interactive relationship to obtain a second sub-feature vector; The first sub-feature vector and the second sub-feature vector are concatenated or merged to obtain the global feature vector.

8. The method according to claim 6, characterized in that The method further comprises: Acquire historical latent space features corresponding to global picture information of a previous frame of the current frame; The encoding results corresponding to the event layers are fused, and the obtained fusion result is fused with the historical latent space feature to determine the global feature vector of the current frame.

9. The method according to claim 8, characterized in that The performing picture generation processing based on the global picture information and the global feature vector to generate the target picture of the current frame includes: Based on the global feature vector, the historical latent space feature is diffused by using a video diffusion model to obtain the latent space feature corresponding to the current frame; A picture is generated according to the latent space features corresponding to the current frame to obtain a target picture of the current frame.

10. The method according to claim 9, characterized in that The method further comprises: The latent space features corresponding to the target picture and the current frame are output to control the generation of a picture of the next frame of the current frame.

11. A video generating device, characterized in that: The device comprises: Information acquisition module, used to obtain game process information; A state acquisition module, for responding to a current touch operation on a target game character of the game, and generating real-time state information of a current frame according to the current touch operation and the process information, wherein the real-time state information is used to reflect the real-time states of several event layers in the current game; A feature encoding module, used for performing feature encoding on the real-time state information, and determining a global feature vector of the current frame according to the encoding result; The video generation module is used to obtain the global picture information of the previous frame of the current frame, and perform picture generation processing based on the global picture information and the global feature vector to generate the target picture of the current frame.

12. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the method according to any one of claims 1 to 10 is implemented.

13. An electronic device, characterized in that: include: processor; as well as A memory, configured to store executable instructions of the processor; The processor is configured to perform the method of any one of claims 1 to 10 by executing the executable instructions.

Images