Transition video generation method, virtual character video switching method, device and equipment

Abstract

The invention provides a transition video generation method, a virtual character video switching method, devices and equipment, and is applied to the technical field of artificial intelligence. The transition video generation method comprises the steps that a first animation video and a second animation video are acquired, and the first animation video and the second animation video are different animation videos of the same virtual character; determining an interruption time point on a time axis of the first animation video; and according to the video frame at the interruption time point and the video frame in the second animation video, generating a transition video corresponding to the interruption time point, the transition video being used for a transition stage of switching from playing of the first animation video to playing of the second animation video. Therefore, the fluency of virtual character animation video switching is improved by setting the interruption time point and generating the transition video in advance.

Description

Technical Field

[0001] The present application is applied to the field of artificial intelligence technology, and in particular relates to a transition video generation method, a virtual character video switching method, a device and equipment. Background Art

[0002] With the rapid development of artificial intelligence and computer graphics technology, virtual characters have been widely used in games, education, social media, virtual reality and other fields. Virtual characters can simulate the behavior and emotions of real people and provide users with an immersive experience.

[0003] The related technology responds to the user's interactive operation during the process of playing an animation video of a virtual character, and directly switches to playing another animation video before the animation video is finished, so as to realize the action or state switching of the virtual character.

[0004] However, the above method may cause the virtual character's screen to be discontinuous, resulting in the virtual character's movements or state switching being abrupt and unnatural. Summary of the invention

[0005] In order to solve the above problems, the present application proposes a transition video generation method, a virtual character video switching method, a device and an apparatus, which can improve the smoothness of the virtual character's video switching and make the virtual character's action or state switching more natural.

[0006] The first aspect of the present application provides a method for generating a transition video, including: obtaining a first animation video and a second animation video, wherein the first animation video and the second animation video are different animation videos of the same virtual character; determining an interruption time point on the timeline of the first animation video; and generating a transition video corresponding to the interruption time point based on a video frame at the interruption time point and a video frame in the second animation video, wherein the transition video is used for a transition phase from switching from playing the first animation video to playing the second animation video.

[0007] In some embodiments, there are multiple interruption time points, and there are multiple transition videos, and the multiple interruption time points correspond one-to-one to the multiple transition videos.

[0008] In some embodiments, determining the interruption time points on the timeline of the first animation video includes: determining the multiple interruption time points on the timeline according to a set first time interval.

[0009] In some embodiments, before determining the multiple interruption time points on the timeline according to a set first time interval, it also includes: determining that the first animation video and the second animation video satisfy a first condition; the first condition includes: the video type of the first animation video is a silent type or an interactive type, and the video type of the second animation video is a silent type or an interactive type; and / or, the first condition includes: the number of animation videos under the video type to which the first animation video belongs is less than a first number threshold, and the number of animation videos under the video type to which the second animation video belongs is less than a second number threshold.

[0010] In some embodiments, generating a transition video corresponding to the interruption time point based on the video frame at the interruption time point and the video frame in the second animation video includes: performing optical flow interpolation between the video frames at the multiple interruption time points and the first video frame in the second animation video to generate transition videos corresponding to the multiple interruption time points respectively.

[0011] In some embodiments, determining the interruption time points on the timeline of the first animation video includes: determining the multiple interruption time points on the timeline by matching the first animation video with the second animation video.

[0012] In some embodiments, the process of determining the i-th interruption time point among the multiple interruption time points includes: determining the i-th video segment in the first animation video according to a set second time interval, where i is greater than or equal to 1; matching the video frame in the i-th video segment with the video frame in the starting segment of the second animation video to obtain a matching result, wherein the matching result includes the degree of matching between the video frame in the i-th video segment and the video frame in the starting segment; according to the matching result, confirming the i-th interruption time point on the timeline, wherein the degree of matching between the video frame at the i-th interruption time point and the j-th video frame in the starting segment is the maximum matching degree in the matching result, and j is greater than or equal to 1.

[0013] In some embodiments, the i-th video segment is determined in the first animation video according to the set second time interval, including: when i is equal to 1, the first video frame in the first animation video is determined as the starting frame of the i-th video segment, and the second time interval is determined as the video duration of the i-th video segment, to obtain the i-th video segment; when i is greater than 1, the video frame at the i-1th interruption time point is determined as the starting frame of the i-th video segment, and the second time interval is determined as the video duration of the i-th video segment, to obtain the i-th video segment.

[0014] In some embodiments, the process of generating the i-th transition video among the multiple transition videos includes: performing optical flow interpolation between the video frame at the i-th interruption time point and the j-th video frame to generate the i-th transition video.

[0015] In some embodiments, the matching of video frames in the first animation video with video frames in the second animation video, before determining the multiple interruption time points on the timeline, also includes: determining that the first animation video and the second animation video satisfy a second condition; the second condition includes: the video type of the first animation video is a silent type or an interactive type, and the video type of the second animation video is an action type; and / or the second condition includes: the number of animation videos under the video type to which the first animation video belongs is less than a third quantity threshold, and the number of animation videos under the video type to which the second animation video belongs is greater than a fourth quantity threshold.

[0016] The second aspect of the present application provides a virtual character video switching method, comprising: when pushing a first animation video to a playback end, if an interruption instruction for the first animation video is received, then determining a target interruption time point among the interruption time points included in the timeline of the first animation video, the interruption instruction instructing to interrupt the playback of the first animation video and switch to playing a second animation video, the first animation video and the second animation video being different animation videos of the same virtual character; according to the target interruption time point, determining a target transition video among the transition videos corresponding to the interruption time point, the transition video corresponding to the interruption time point being generated according to the transition video generation method described in the first aspect or any embodiment of the first aspect; when the push progress of the first animation video reaches the target interruption time point, pushing the target transition video to the playback end; when the push progress of the target transition video reaches the end frame of the target transition video, pushing the second animation video to the playback end.

[0017] In some embodiments, the timeline includes multiple interruption time points related to the second animation video, there are multiple transition videos, and the multiple interruption time points correspond one-to-one to the multiple transition videos, and the target interruption time point is determined among the interruption time points included in the timeline of the first animation video, including: among the multiple interruption time points, determining that the target interruption time point is the interruption time point closest to the current playback time point of the first animation video.

[0018] In some embodiments, pushing the second animation video to the playback end includes: starting from a switching video frame of the second animation video, pushing the second animation video to the playback end, the switching video frame being a video frame in the starting segment of the second animation video that has the highest degree of match with the video frame at the target interruption time point.

[0019] The third aspect of the present application provides a transition video generation device, including: a video acquisition unit, used to acquire a first animation video and a second animation video, wherein the first animation video and the second animation video are different animation videos of the same virtual character; an interruption time point determination unit, used to determine the interruption time point on the timeline of the first animation video; a transition video generation unit, used to generate a transition video corresponding to the interruption time point based on the video frame at the interruption time point and the video frame in the second animation video, wherein the transition video is used for the transition stage from switching from playing the first animation video to playing the second animation video.

[0020] The fourth aspect of the present application provides a virtual character video switching device, including: an interruption time point determination unit, which is used to determine a target interruption time point among the interruption time points included in the timeline of the first animation video when pushing a first animation video to a playback end, if an interruption instruction for the first animation video is received, the interruption instruction indicates to interrupt the playback of the first animation video and switch to the playback of the second animation video, the first animation video and the second animation video are different animation videos of the same virtual character; a transition video determination unit, which is used to determine a target transition video in the transition video corresponding to the interruption time point according to the target interruption time point, the transition video corresponding to the interruption time point is generated according to the transition video generation method described in the first aspect or any embodiment of the first aspect; a transition video pushing unit, which is used to push the target transition video to the playback end when the push progress of the first animation video reaches the target interruption time point; and a switching video pushing unit, which is used to push the second animation video to the playback end when the push progress of the target transition video reaches the end frame of the target transition video.

[0021] The fifth aspect of the present application provides an electronic device, comprising a memory and a processor; the memory is connected to the processor and is used to store programs; the processor is used to implement the transition video generation method as described in the first aspect or any embodiment of the first aspect by running the program in the memory, or to implement the virtual character video switching method as described in the second aspect or any embodiment of the second aspect.

[0022] The sixth aspect of the present application provides a computer program product, including a computer program. When the computer program is executed by a processor, it implements the transition video generation method as described in the first aspect or any embodiment of the first aspect, or implements the virtual character video switching method as described in the second aspect or any embodiment of the second aspect.

[0023] The seventh aspect of the present application provides a storage medium having a computer program stored thereon. When the computer program is executed by a processor, the method for generating a transition video as described in the first aspect or any embodiment of the first aspect is implemented, or the method for switching a virtual character video as described in the second aspect or any embodiment of the second aspect is implemented.

[0024] The present application proposes a transition video generation method, a virtual character video switching method, a device and equipment. For the first animation video and the second animation video of the virtual character, the interruption time point is determined on the timeline of the first animation video, and then the video frame at the interruption time point and the video frame in the second animation video are combined to generate a transition video corresponding to the interruption time point. The transition video is used for the transition stage from the first animation video to the second animation video. Compared with directly switching from the first animation video to the second animation video, using the transition video for switching transition at the interruption time point effectively improves the smoothness of switching from the first animation video to the second animation video, making the state switching or action switching of the virtual character more natural, thereby improving the interactive effect of the virtual character. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying any creative work.

[0026] Figure 1 A schematic diagram of an implementation environment involved in an embodiment of the present application;

[0027] Figure 2 The process diagram of the transition video generation method provided in the embodiment of the present application is as follows Figure 1 ;

[0028] Figure 3 The process diagram of the transition video generation method provided in the embodiment of the present application is as follows Figure 2 ;

[0029] Figure 4 This is an example of optical flow interpolation provided according to an embodiment of the present application. Figure 1 ;

[0030] Figure 5 The process diagram of the transition video generation method provided in the embodiment of the present application is as follows Figure 3 ;

[0031] Figure 6 This is an example of optical flow interpolation provided according to an embodiment of the present application. Figure 2 ;

[0032] Figure 7 A schematic diagram of a flow chart of a virtual character video switching method provided according to an embodiment of the present application;

[0033] Figure 8 is a timing diagram of a virtual character video playback provided in an embodiment of the present application;

[0034] Fig. 9 It is a structural schematic diagram of a transition video generating device provided according to an embodiment of the present application;

[0035] Fig.10 It is a structural schematic diagram of a virtual character video switching device provided according to an embodiment of the present application;

[0036] Fig.11 It is a schematic diagram of the structure of an electronic device provided according to an embodiment of the present application. DETAILED DESCRIPTION

[0037] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0038] In the related art, the action switching of the virtual character is mainly performed in the following ways:

[0039] Method 1: Pre-record the action library of the virtual character, in which each action consists of one or more animation videos. In the action library, the animation videos of the virtual character are played and switched based on the set trigger conditions and simple switching rules. For example, after the action is completed, the virtual character automatically returns to the initial state and switches to the animation video of the next action at a fixed time point.

[0040] Method 2: Manage the actions of the virtual character through the animation state machine. The animation state machine includes multiple state nodes and multiple state switching conditions. Each state node represents an action of the virtual character. Different actions are switched according to the events input into the animation state machine and the state switching conditions.

[0041] Method 3, by simulating the actions of the real environment through the physical engine, the virtual character can interact with the virtual environment. For example, the walking action of the virtual character is adjusted according to the inclination angle, surface material, obstacles, etc. of the ground in the virtual environment to achieve action switching.

[0042] Method 4: During the switching of the animation video of the virtual character, the last frame of the animation video before the switching and the first frame of the animation video after the switching are combined to perform optical flow interpolation to generate a transition video, and the animation video switching of the virtual character is achieved through the transition video.

[0043] However, the above methods have the following shortcomings respectively:

[0044] Methods 1 and 2 have the following disadvantages: (1) There may be a significant delay in the switching of animation videos of virtual characters. For example, according to the switching rules or state switching conditions, one action must be completed before another action can be executed, making the time for switching from one action to another uncertain; (2) The animation video switching of virtual characters lacks dynamic processing of action details. For example, when one action is not completed, it directly switches to the next action, resulting in discontinuous images of the virtual characters and abrupt and unnatural animation video switching; (3) The animation video switching is not flexible enough. The animation video switching is based on fixed conditions and cannot be flexibly adjusted according to the context or changes in user input. When the user wants to interrupt an action and switch to another action, there is a lack of corresponding support solutions.

[0045] Methods 3 and 4 have the following disadvantages: whether it is real-time simulation of the actions of the real environment through the physical engine, or real-time generation of transition videos through optical flow interpolation, both have high requirements on computing resources, resulting in high performance consumption, and are difficult to implement on devices with limited computing resources.

[0046] To solve the above problems, the embodiments of the present application provide a transition video generation method, a virtual character video switching method, a device and an apparatus, which are applied to the field of artificial intelligence technology. In the transition video generation method provided in the embodiments of the present application, on the timeline of the first animation video of the virtual character, the interruption time point is determined, and the video frame at the time interruption point and the video frame in the second animation video are combined to generate a transition video at the interruption time point. In the process of switching from the first animation video to the second animation video, the transition video can be played at the interruption time point for transition. On the one hand, by generating a transition video, the continuity of the picture during the switching process of the virtual character animation video is improved, and the smoothness and naturalness of the animation video switching are improved. On the other hand, the transition video can be generated offline, and there is no need to generate it in real time during the switching process, which reduces the requirements for computing resources and reduces the performance consumption during the virtual character display process; on the other hand, the switching effect of the virtual character animation video can be improved by reasonably setting the interruption time point on the timeline, the flexibility of the animation video switching is improved, and the delay of the animation video switching is reduced. Thereby, the interactive experience between the user and the virtual character is effectively improved.

[0047] Exemplary Implementation Environment

[0048] Please refer to Figure 1 , Figure 1 Schematic diagram of the implementation environment involved in the embodiment of the present application. The implementation environment involved in the present application includes a first device 10, a second device 20 and a database 30. The first device 10 may include a display device 11 and a backend device 12. The database 30 may be used to store animation videos of virtual characters and transition videos for switching animation videos. The first device 10 and the second device 20 may perform data transmission with the database 30 respectively.

[0049] During the offline generation of the virtual character animation video, the second device 20 can obtain the animation video of the virtual character from the database 30, and based on the animation video of the virtual character, use the transition video generation method provided in the embodiment of the present application to generate a transition video, and store the transition video in the database 30.

[0050] During the display process of the virtual character, the back-end device 12 can obtain the animation video of the virtual character from the database 30 and push the animation video to the display device 11; if the animation video needs to be switched, the back-end device 12 can obtain the transition video from the database 30 and push the transition video to the display device 11.

[0051] It should be noted that the animation video and transition video of the virtual character can also be stored in different databases.

[0052] The first device 10 and the second device 20 may be terminals or servers, or the first device 10 may include a terminal and a server, in which case the terminal serves as a display device 11 and the server serves as a backend device 12 . Figure 1 For example, the first device 10 includes a terminal and a server, and the second device 20 is a server.

[0053] This embodiment can be applied to a variety of scenarios: live broadcasting scenarios, electronic game scenarios, voice interaction scenarios (such as voice translation scenarios, intelligent question-and-answer scenarios), map navigation scenarios, etc. The virtual characters in this embodiment are, for example, virtual anchors in live broadcasting scenarios, game characters in electronic game scenarios, voice assistants in voice interaction scenarios, and navigation assistants in map navigation scenarios.

[0054] As an example, in a voice interaction scenario, the first device 10 may be a smart terminal device, and a virtual character of a voice assistant may be displayed on the first device 10. By displaying different animation videos of the virtual character, the virtual character may make different actions, such as nodding, shaking head, waving, sitting down, etc.

[0055] Exemplary Methods

[0056] See also Figure 2 In an exemplary embodiment, a transition video generation method is provided, and the transition video generation method comprises the following steps:

[0057] S201, obtaining a first animation video and a second animation video, where the first animation video and the second animation video are different animation videos of the same virtual character.

[0058] The virtual character may be a two-dimensional virtual character or a three-dimensional virtual character, such as a two-dimensional virtual person, a two-dimensional virtual pet, and the like.

[0059] In this embodiment, the first animation video and the second animation video can be obtained from a database. The database contains multiple animation videos of the same virtual character, the first animation video can be one of the multiple animation videos, and the second animation video can be one of the remaining animation videos in the multiple animation videos except the first animation video.

[0060] It should be noted that, in this embodiment, generation of a transition video for switching from playing a first animation video to playing a second animation video is taken as an example. In actual scenarios, this embodiment can be referred to to generate a transition video for switching between multiple animation videos of a virtual character, such as generating a transition video for switching between all or part (greater than 2) of the animation videos in a database.

[0061] S202, determining an interruption time point on the time axis of the first animation video.

[0062] Among them, the timeline of the first animation video refers to the progress bar of the first animation video, which can also be called the time bar or timeline of the first animation video, reflecting the length of the first animation video and the time point corresponding to each video frame in the first animation video.

[0063] The interruption time point refers to a time point at which the playback of the first animation video is allowed to be interrupted (interrupted, stopped or ended), and may also be referred to as a switching time point.

[0064] In this embodiment, the interruption time points can be determined on the time axis of the first animation video, and the virtual character's video can be switched with reference to these interruption time points. For example, the interruption time points can be randomly determined on the time axis, or the interruption time points can be selected from multiple time points included in the time axis according to the set interruption point selection rules.

[0065] S203, generating a transition video corresponding to the interruption time point according to the video frame at the interruption time point and the video frame in the second animation video, wherein the transition video is used for a transition phase of switching from playing the first animation video to playing the second animation video.

[0066] The video frames in the second animation video are, for example, the first frame in the second animation video, or the video frames included in the starting segment (a time interval of the starting segment is preset) of the second animation video.

[0067] In this embodiment, multiple video frames can be generated between the video frame at the interruption time point and the video frame in the second animation video, and the multiple video frames are combined to obtain a transition video corresponding to the interruption time point. The transition video can be subsequently used in the transition stage from switching from playing the first animation video to playing the second animation video, making the switch from the first animation video to the second animation video more natural and smooth.

[0068] In one example, optical flow interpolation can be performed between the video frame at the interruption time point and the video frame in the second animation video to generate a transition video corresponding to the interruption time point. Thus, the quality of the generated transition video is improved through the optical flow interpolation technology.

[0069] In another example, the video frame at the interruption time point and the video frame in the second animation video can be input into a pre-trained neural network model, and a transition video corresponding to the interruption time point can be generated by the neural network model. Thus, the generation quality of the transition video can be improved by using the neural network model.

[0070] In the embodiment of the present application, an interruption time point is determined on the timeline of the first animation video, and a transition video corresponding to the interruption time point is generated according to the video frame at the interruption time point and the video frame in the second animation video. On the one hand, by using the transition video corresponding to the interruption time point, the picture continuity from the first animation video to the second animation video is improved, making the switch from the first animation video to the second animation video more natural and smooth, and improving the switching effect of the virtual character action; on the other hand, by switching the animation video at the interruption time point, there is no need to wait for the first animation video to be played, and the flexibility of switching from the first animation video to the second animation video can be improved by reasonably setting the interruption time point, and the time delay of the animation video switching can be reduced; on the other hand, by generating the transition video offline, the i performance consumption in the virtual character display process is reduced, and the requirements for computing resources for real-time display of virtual characters are reduced.

[0071] In some embodiments, there are multiple interruption time points, multiple transition videos, and multiple interruption time points correspond to multiple transition videos one by one. That is, for each interruption time point, a transition video corresponding to the interruption time point can be generated according to the video frame at the interruption time point and the video frame in the second animation video, thereby obtaining transition videos corresponding to multiple interruption time points. Thus, through multiple interruption time points and transition videos corresponding to multiple interruption time points, the flexibility of switching from playing the first animation video to playing the second animation video is further improved, the time delay of animation video switching is reduced, and the switching from playing the first animation video to playing the second animation video is more natural and smooth, which improves the switching effect of virtual character actions. .

[0072] In one example, multiple interruption time points are randomly determined on the time axis.

[0073] In another example, on the time axis, a plurality of interruption time points are determined at equal intervals.

[0074] In another example, on the time axis, multiple interruption time points are determined at unequal intervals according to a set interruption time point selection rule. For example, the interruption time point selection rule requires that the video frame at the interruption time point meets certain conditions, or that the video frames at adjacent time points of the interruption time point meet certain conditions, or that the intervals between multiple interruption time points meet certain conditions.

[0075] In another example, the first animation video and the second animation video may be matched to determine multiple interruption time points on the event axis. For details, please refer to the description of the corresponding embodiments below.

[0076] See also Figure 3 In another exemplary embodiment, a transition video generation method is provided, the transition video generation method comprising the following steps:

[0077] S301, obtaining a first animation video and a second animation video, where the first animation video and the second animation video are different animation videos of the same virtual character.

[0078] The implementation principle and technical effects of S301 may refer to the aforementioned embodiments and will not be described in detail.

[0079] S302: Determine whether the first animation video and the second animation video satisfy a first condition.

[0080] S302 is an optional step. The first condition may not be set, and S303 may be directly executed after S301.

[0081] In this embodiment, it is determined whether the first animation video and the second animation video meet the first condition. If the first animation video and the second animation video meet the first condition, S303 is executed. If the first animation video and the second animation video do not meet the first condition, the generation of a transition video from the first animation video to the second animation video may be rejected, or other methods may be used to determine the interruption time point.

[0082] In one example, the first condition includes: the video type of the first animation video is a silent type or an interactive type, and the video type of the second animation video is a silent type or an interactive type.

[0083] Among them, in the database, multiple animation videos of virtual characters and type tags corresponding to the multiple animation videos may be included. The type tag of the animation video indicates the video type of the animation video. The video type of the animation video may include a silent type, an interactive type, and an action type. In a silent type animation video, the virtual character is in a silent state or a standby state. In the silent state, the virtual character can wait for the user to input information, without performing any action or speaking. In the standby state, the virtual character can be still or perform a preset standby action. In an interactive type animation video, the virtual character is in an interactive state and can output interactive actions or have a conversation with the user; in an action type animation video, the virtual character is in a state of performing the operations included in the task event, such as singing, dancing, or translating the user's input content.

[0084] In this example, compared with action-type animation videos, the number of silent-type animation videos and interactive-type animation videos is relatively small, so the number of transitional videos required for animation video switching between silent-type animation videos is relatively small. Therefore, when the first animation video and the second animation video meet the first condition, that is, when the video type of the first animation video is silent type or interactive type and the video type of the second animation video is silent type or interactive type, a transitional video can be generated for switching the first animation video to the second animation video in the manner shown in subsequent S303 and S304, without incurring a large amount of data processing.

[0085] In another example, the first condition includes: the number of animation videos in the video type to which the first animation video belongs is less than a first number threshold, and the number of animation videos in the video type to which the second animation video belongs is less than a second number threshold. The first number threshold and the second number threshold are the same or different.

[0086] For example, the video type of the first animation video is a silent type, and the number of animation videos under the video type to which the first animation video belongs refers to the number of animation videos belonging to the silent type.

[0087] In this example, the video type to which the first animation video belongs and the video type to which the second animation video belongs are constrained directly by means of a quantity threshold. The first animation video can be any animation video under the video type to which the first animation video belongs, and the second animation video can be any animation video under the video type to which the second animation video belongs. When the number of animation videos under the video type to which the first animation video belongs is less than the first quantity threshold, and the number of animation videos under the video type to which the second animation video belongs is less than the second quantity threshold, the number of transition videos required for video switching between the animation videos under the video type to which the first animation video belongs and the animation videos under the video type to which the second animation video belongs will not be large, and the transition videos can be generated in the manner shown in subsequent S303 and S304, without causing a large amount of data processing.

[0088] It should be noted that the first conditions provided by the above two examples can be combined with each other. S303 can be executed when the first animation video and the second animation video meet the first conditions provided by any of the above two examples, or when the first animation video and the second animation video meet the first conditions provided by both examples.

[0089] S303: Determine multiple interruption time points on the time axis according to the set first time interval.

[0090] The first time interval is a fixed time interval.

[0091] In one example, the first time interval may be determined with reference to the time length of the time axis of the first animation video to improve the rationality of the first time interval.

[0092] In this embodiment, on the time axis of the first animation video, multiple interruption time points are evenly determined according to the set first time interval, so that the multiple interruption time points are evenly distributed on the time axis of the first animation video. In the process of switching from playing the first animation video to playing the second animation video, a suitable interruption time point can be selected from the evenly distributed interruption time points to switch the video, thereby improving the smoothness of switching from the first animation video to the second animation video.

[0093] S304: Generate transition videos corresponding to a plurality of interruption time points respectively according to the video frames at the plurality of interruption time points and the video frames in the second animation video.

[0094] The video frames in the second animation video are, for example, the first frame in the second animation video, or the video frames included in the starting segment (a time interval of the starting segment is preset) of the second animation video.

[0095] In this embodiment, for each interruption time point, multiple video frames may be generated between the video frame at the interruption time point and the video frame in the second animation video, and the transition video corresponding to the interruption time point may be obtained by combining the multiple video frames.

[0096] In one example, optical flow interpolation may be performed between video frames at multiple interruption time points and the first video frame in the second animation video to generate transition videos corresponding to the multiple interruption time points.

[0097] In this example, for each interruption time point, optical flow interpolation can be performed between the video frame at the interruption time point and the video frame in the second animation video to obtain multiple video frames, and the multiple video frames are combined to obtain the transition video corresponding to the interruption time point. Thus, the video generation quality of multiple transition videos is improved through the optical flow interpolation technology.

[0098] In the embodiment of the present application, when the first animation video and the second animation video meet the first condition, multiple interruption time points can be determined on the time axis of the first animation video at a fixed time interval, and multiple transition videos corresponding to the interruption time points can be generated according to the video frames at the multiple interruption time points and the video frames in the second animation video. Thus, sufficient interruption time points to choose from are provided for switching from the first animation video to the second animation video, and sufficient transition videos to choose from are also provided, which effectively improves the flexibility and smoothness of the animation video switching of the virtual character, makes the animation video switching of the virtual character more natural, and improves the user experience.

[0099] As an example, Figure 4This is an example of optical flow interpolation provided according to an embodiment of the present application. Figure 1 .like Figure 4 As shown, on the time axis of the first animation video, a plurality of breakpoint frames (ie, video frames at the interrupted time points) are determined according to a fixed time interval (eg, a first time interval), for example Figure 4 There is a breakpoint frame every 1 second, and optical flow interpolation is performed between each breakpoint frame and the target action frame (i.e., the video frame in the second animation video, such as the first frame in the second animation video) to obtain multiple optical flow interpolation results (i.e., multiple transition videos, each optical flow interpolation result includes multiple video frames obtained by interpolation).

[0100] See also Figure 5 In another exemplary embodiment, a transition video generation method is provided, the transition video generation method comprising the following steps:

[0101] S501, obtaining a first animation video and a second animation video, where the first animation video and the second animation video are different animation videos of the same virtual character.

[0102] The implementation principle and technical effects of S501 may refer to the aforementioned embodiments and will not be described in detail.

[0103] S502: Determine whether the first animation video and the second animation video satisfy a second condition.

[0104] Among them, S502 is an optional step, and the second condition may not be set, and S503 is directly executed after S501.

[0105] In this embodiment, it is determined whether the first animation video and the second animation video meet the second condition. If the first animation video and the second animation video meet the second condition, S503 is executed. If the first animation video and the second animation video do not meet the second condition, the generation of a transition video from the first animation video to the second animation video may be rejected, or other methods may be used to determine the interruption time point.

[0106] In one example, the second condition includes: the video type of the first animation video is a silent type or an interactive type, and the video type of the second animation video is an action type.

[0107] Among them, the database may include multiple animation videos of virtual characters and type labels corresponding to the multiple animation videos. The type label of the animation video indicates the video type of the animation video. The video type of the animation video may include a silent type, an interactive type and an action type. For these video types, please refer to the description of the aforementioned embodiments and will not be repeated here.

[0108] In this example, for virtual characters, there are a large number of action-type animation videos, and the virtual characters in the action-type animation videos may have obvious movements and more changes in movements. If the interruption time points are evenly determined on the timeline at fixed time intervals, the positions of the interruption time points may be unreasonable, which may also lead to a large number of transition videos and excessive resource consumption of the transition video generation task. Therefore, when the video type of the first animation video is a silent type or an interactive type, and the video type of the second animation video is an action type, the interruption time points can be determined and the transition videos can be generated in the manner shown in subsequent S403 and S404, which can improve the rationality of the position of the interruption time points on the timeline and the rationality of the number of interruption time points.

[0109] In another example, the second condition includes: the number of animation videos in the video type to which the first animation video belongs is less than a third number threshold, and the number of animation videos in the video type to which the second animation video belongs is greater than a fourth number threshold. The third number threshold may be less than the fourth number threshold.

[0110] In this example, the video type to which the first animation video belongs and the video type to which the second animation video belongs are constrained directly by means of quantity thresholds. The first animation video can be any animation video under the video type to which the first animation video belongs, and the second animation video can be any animation video under the video type to which the second animation video belongs. When the number of animation videos under the video type to which the first animation video belongs is less than the third quantity threshold, and the number of animation videos under the video type to which the second animation video belongs is less than the fourth quantity threshold, the interruption time point can be determined and the transition video can be generated in the manner shown in subsequent S403 and S404, which can improve the rationality of the position of the interruption time point on the timeline and the rationality of the number of interruption time points.

[0111] It should be noted that the second conditions provided by the above two examples can be combined with each other. S503 can be executed when the first animation video and the second animation video meet the second conditions provided by any of the above two examples, or S503 can be executed when the first animation video and the second animation video meet the second conditions provided by both examples.

[0112] The third quantity threshold and the fourth quantity threshold have no relation with the first quantity threshold and the second quantity threshold. For example, the third quantity threshold may be the same as or different from the first quantity threshold.

[0113] S503: Determine a plurality of interruption time points on the time axis by matching the first animation video with the second animation video.

[0114] In this embodiment, multiple video frames in the first animation video are matched with multiple video frames in the second animation video to obtain matching results. The matching results may include the matching degree (i.e., similarity) between the multiple video frames in the first animation video and the multiple video frames in the second animation video. Based on the matching results, multiple interruption time points are determined on the timeline of the first animation video.

[0115] In one example, among multiple video frames of a first animation video, multiple video frames whose matching degree with one of the video frames of a second animation video is greater than a degree threshold are selected, and time points corresponding to the multiple video frames are determined as multiple interruption time points.

[0116] In another example, a process of determining the i-th interruption time point among multiple interruption time points includes: determining the i-th video segment in the first animation video according to a set second time interval, where i is greater than or equal to 1; matching the video frame in the i-th video segment with the video frame in the starting segment of the second animation video to obtain a matching result, the matching result including the degree of matching between the video frame in the i-th video segment and the video frame in the starting segment; based on the matching result, confirming the i-th interruption time point on the timeline, the degree of matching between the video frame at the i-th interruption time point and the j-th video frame in the starting segment is the maximum matching degree in the matching result, where j is greater than or equal to 1.

[0117] In this example, in order to improve the accuracy of each interruption time point, a second time interval is set for the first animation video, and a time range corresponding to the start segment is set for the second animation video. In the process of determining the ith interruption time point, first, the ith video segment with a video duration of the second time interval in the first animation video can be determined in chronological order, and the start segment of the second animation video can be determined according to the time range corresponding to the start segment; then, the video frames in the ith video segment can be matched with the video frames in the start segment of the second animation video to obtain the degree of match, i.e., similarity, between the video frames in the ith video segment and the video frames in the start segment; then, the maximum degree of match is selected from the degree of match between the video frames in the ith video segment and the video frames in the start segment; in the ith video segment, the video frame corresponding to the maximum degree of match is determined, and the time point corresponding to the video frame is determined to be the ith interruption time point; in the start segment of the second animation video, the video frame corresponding to the maximum degree of match is determined, i.e., the jth video frame in the start segment.

[0118] Afterwards, the i+1th video segment can be determined in the first animation video according to the set second time interval, and the i+1th interruption time point can be determined by matching the video frame in the i+1th video segment with the video frame in the start segment of the second animation video. This is done until the video segment selected according to the second time interval reaches the end of the timeline of the first animation video.

[0119] Thus, by sequentially selecting video segments in the first animation video and matching them with the starting segments of the second animation video for video frame matching, dynamic determination of interruption time points on the timeline of the first animation video is achieved, thereby improving the accuracy of determining interruption time points in the first animation video and the rationality of the position distribution of interruption time points on the timeline.

[0120] In one example, when i is equal to 1, the first video frame in the first animation video is determined as the starting frame of the ith video segment, and the second time interval is determined as the video duration of the ith video segment, to obtain the ith video segment; when i is greater than 1, the last video frame of the i-1th video segment is determined as the starting frame of the ith video segment, and the second time interval is determined as the video duration of the ith video segment, to obtain the ith video segment. It can be seen that the multiple video segments selected in the first animation video are continuous and non-overlapping on the timeline, and the video duration is fixed to the second time interval.

[0121] In another example, when i is equal to 1, the first video frame in the first animation video is determined as the starting frame of the ith video segment, and the second time interval is determined as the video duration of the ith video segment, to obtain the ith video segment; when i is greater than 1, the video frame at the i-1th interruption time point is determined as the starting frame of the ith video segment, and the second time interval is determined as the video duration of the ith video segment, to obtain the ith video segment. It can be seen that the multiple video segments selected in the first animation video are continuous on the timeline and the video duration is fixed to the second time interval, and there is a time overlap between adjacent video segments in the multiple video segments.

[0122] Compared with the previous example, this example takes into account that there may be multiple video frames in the i-1th video segment that have a high degree of match with the video frames in the starting segment of the second animation video, while there may be a period of video frames after the i-1th video segment that have a low degree of match with the video frames in the starting segment. Therefore, after determining the i-1th interruption time point, the video frame at the i-1th interruption time point is determined as the starting frame of the i-th video segment, so that the time point corresponding to the video frame with a higher degree of match is selected as the interruption time point as much as possible, which improves the accuracy of determining the interruption time point in the first animation video and also improves the rationality of the position distribution of the interruption time points on the timeline.

[0123] S504: Generate transition videos corresponding to a plurality of interruption time points respectively according to the video frames at the plurality of interruption time points and the video frames in the second animation video.

[0124] The video frames in the second animation video are, for example, the first frame in the second animation video, or the video frames included in the starting segment (a time interval of the starting segment is preset) of the second animation video.

[0125] In this embodiment, for each interruption time point, multiple video frames may be generated between the video frame at the interruption time point and the video frame in the second animation video, and the transition video corresponding to the interruption time point may be obtained by combining the multiple video frames.

[0126] In one example, optical flow interpolation may be performed between video frames at multiple interruption time points and the first video frame in the second animation video to generate transition videos corresponding to the multiple interruption time points.

[0127] In this example, for each interruption time point, optical flow interpolation can be performed between the video frame at the interruption time point and the video frame in the second animation video to obtain multiple video frames, and the multiple video frames are combined to obtain the transition video corresponding to the interruption time point. Thus, the video generation quality of multiple transition videos is improved through the optical flow interpolation technology.

[0128] In another example, the process of generating the i-th transition video among multiple transition videos includes: performing optical flow interpolation between the video frame at the i-th interruption time point and the j-th video frame in the start segment of the second animation video to generate the i-th transition video. Wherein, i is greater than or equal to 1, the j-th video frame is the video frame in the start segment that has the highest matching degree with the video frame at the i-th interruption time point, and the process of determining the j-th video frame can refer to the aforementioned example of video frame matching, which will not be repeated here.

[0129] Thus, by generating a transition video between video frames with a high degree of matching between the first animation video and the second animation video, the picture changes of the transition video can be made smaller, the picture discontinuity can be reduced, and the smoothness of switching from the first animation video to the second animation video can be improved. In the subsequent switching process, after the switching, it is not necessary to start playing from the first video frame of the second animation video, but to start playing from the jth video frame, which can improve the efficiency and intelligence of the virtual character action switching.

[0130] In the embodiment of the present application, when the first animation video and the second animation video meet the second condition, multiple interruption time points can be dynamically determined on the timeline of the first animation video by matching the video frames between the first animation video and the second animation video, and multiple transition videos corresponding to the interruption time points are generated according to the video frames at the multiple interruption time points and the video frames in the second animation video. Thus, by dynamically determining multiple interruption time points, the rationality and accuracy of the number and position of the interruption time points are improved, which is conducive to improving the flexibility and smoothness of the animation video switching of the virtual character, making the animation video switching of the virtual character more natural and improving the user experience.

[0131] As an example, Figure 6 This is an example of optical flow interpolation provided according to an embodiment of the present application. Figure 2 .like Figure 6 As shown, a plurality of breakpoint frames (i.e., video frames at the interrupted time points) with irregular time intervals are dynamically determined on the time axis of the first animation video, for example, Figure 6 The multiple breakpoint frames are distributed at 0.5s (seconds), 1.9s, 3.2s, 3.6s, 4.8s, 5.8s, 6.8s, 8.5s, and 9s. Optical flow interpolation is performed between each breakpoint frame and the target action frame (the video frame with the highest matching degree with the breakpoint frame in the beginning segment of the second animation video) to obtain multiple optical flow interpolation results (i.e., multiple transition videos, each optical flow interpolation result includes multiple video frames obtained by interpolation).

[0132] Based on any of the foregoing embodiments, the following optional solutions may also be provided:

[0133] In some embodiments, after determining the interruption time point on the timeline of the first animation video, the correspondence between the interruption time point and the identification information of the second animation video can be saved, so that in the process of switching from the first animation video to the second animation video, the interruption time point related to the second animation video can be obtained in time on the timeline of the first animation video.

[0134] In some embodiments, before determining the interruption time point on the timeline of the first animation video, it can be determined that the video type of the first animation video and the video type of the second animation video are not both action types. Considering that there are a large number of action-type animation videos of virtual characters, there are many combinations of switching from an action-type animation video to another action-type animation video, and a large number of transition videos need to be generated, which will require a large amount of data processing resources and storage resources. Therefore, in the case where the video type of the first animation video and the video type of the second animation video are both action types, it can be selected not to generate the transition video offline to save software and hardware resources and reduce the amount of data storage.

[0135] Below, an embodiment of a virtual character video switching process is provided.

[0136] See also Figure 7 In another exemplary embodiment, a virtual character video switching method is provided, and the virtual character video switching method comprises the following steps:

[0137] S701, when a first animation video is pushed to a playback end, if an interruption instruction for the first animation video is received, a target interruption time point is determined from the interruption time points included in the timeline of the first animation video.

[0138] Among them, the interruption instruction indicates to interrupt the playback of the first animation video and switch to the playback of the second animation video. The first animation video and the second animation video are different animation videos of the same virtual character. The first animation video and the second animation video can refer to the description of the aforementioned embodiment and will not be repeated here.

[0139] The interruption instruction may come from the playback end, and is generated by the playback end after receiving the user's interactive operation, such as the playback end generating the interruption instruction after receiving the user's voice instruction or screen operation. The playback end refers to the display end of the virtual character.

[0140] Among them, the interruption time point included in the timeline of the first animation video is the interruption time point related to the second animation video, or in other words, it is the interruption time point used to switch from the first animation video to the second animation video. The process of determining the interruption time point can refer to the description of the aforementioned embodiment and will not be repeated here.

[0141] In this embodiment, when the first animation video is pushed to the playback end, that is, when the first animation video is displayed on the playback end, if an interruption instruction for the first animation video is received, a target interruption time point is determined among the interruption time points included in the timeline of the first animation video. The target interruption time point is located after the push progress of the first animation video (that is, after the current playback time).

[0142] In one example, the timeline of the first animation video includes an interruption time point, and the target interruption time point can be determined to be the interruption time point.

[0143] In another example, the timeline of the first animation video includes multiple interruption time points, and a target interruption time point can be selected from the multiple interruption time points.

[0144] In one example, the interruption instruction includes a target action to be performed by the virtual character to instruct switching to the animation video corresponding to the target action, that is, the second animation video.

[0145] In another example, the interruption instruction includes interactive information input by the user, such as the user's voice instruction, touch screen operation, etc. The interactive information can be analyzed and processed to determine the second animation video.

[0146] In this example, after receiving the interruption instruction, the target action to be performed by the virtual character is obtained by analyzing the user's interaction information, and the animation video corresponding to the target action, i.e., the second animation video, is determined based on the mapping relationship between the virtual character's action and the animation video.

[0147] In one example, the first animation video may also include interruption time points related to other animation videos other than the second animation video. Before determining the target interruption time point, the interruption time point corresponding to the identification information of the second animation video included in the timeline of the first animation video may be determined. The correspondence between the second animation video and the interruption time point may be configured in advance, and the specifics may refer to the relevant steps in the transition video generation method provided in the aforementioned embodiment.

[0148] S702: According to the target interruption time point, determine a target transition video from the transition videos corresponding to the interruption time point.

[0149] The transition video corresponding to the interruption time point is generated according to the video frame at the interruption time point and the video frame of the second animation video.

[0150] Furthermore, the transition video corresponding to the interruption time point can be generated according to the transition video generation method provided in any of the aforementioned embodiments, which will not be described in detail here.

[0151] In this embodiment, the transition video corresponding to the target interruption time point can be searched for in the transition video corresponding to the interruption time point, and the target transition video is the transition video corresponding to the target interruption time point.

[0152] S703: When the push progress of the first animation video reaches the target interruption time point, the target transition video is pushed to the playback end.

[0153] In this embodiment, when the push progress of the first animation video reaches the target interruption time point, that is, when the current playback progress (current playback time) of the first animation video reaches the target interruption time point, the target transition video is pushed to the playback end, so that the playback end switches from playing the first animation video of the virtual character to playing the target transition video at the target interruption time point.

[0154] S704: When the push progress of the target transition video reaches the end frame of the target transition video, push the second animation video to the playback end.

[0155] In this embodiment, when the push progress of the target transition video reaches the end frame of the target transition video, that is, when the target transition video is finished playing, the second animation video is pushed to the playback end, so that the playback end switches from playing the target transition video to playing the first animation video of the virtual character.

[0156] In the embodiment of the present application, when the first animation video of the virtual character is played to the target interruption time point, the transition video corresponding to the target interruption time point is switched to be played, and after the transition video is played, the second animation video of the virtual character is switched to be played. Since the transition video is generated based on the video frame at the target interruption time point and the video frame of the second animation video, the switching between the first animation video and the second animation video is achieved through the transition video at the target interruption time point, which can effectively improve the picture continuity of the virtual character video switching, make the virtual character action or state switching more smooth and natural, and improve the user's interactive experience with the virtual character.

[0157] Below, some implementation methods are provided for some steps of the above embodiment.

[0158] In some embodiments, the timeline includes multiple interruption time points related to the second animation video, there are multiple transition videos, and the multiple interruption time points correspond to the multiple transition videos one by one. Based on this, S701 includes: among the multiple interruption time points, determining that the target interruption time point is the interruption time point closest to the current playback time point of the first animation video. The current playback time point of the first animation video refers to the time point corresponding to the current pushed video frame in the first animation video.

[0159] In this embodiment, the interruption time point closest to the current playback time point of the first animation video is selected as the target interruption time point to improve the animation video switching efficiency from the first animation video to the second animation video and reduce the delay of animation video switching.

[0160] In addition to the above method, an interruption time point located after the current playback time point of the first animation video may be randomly selected from multiple interruption time points as the target interruption time point.

[0161] In some embodiments, S704 includes: when the push progress of the target transition video reaches the end frame of the target transition video, starting from the switching video frame of the second animation video, pushing the second animation video to the playback end, the switching video frame being the video frame in the starting segment of the second animation video that has the highest degree of match with the video frame at the target interruption time point.

[0162] Taking the target interruption time point as the i-th interruption time point among multiple interruption time points as an example, the switching video frame is the j-th video frame in the starting segment of the second animation video, wherein the i-th interruption time point and the j-th video frame can refer to the description of the previous embodiment and will not be repeated here.

[0163] In this embodiment, after the target transition video is pushed, it is not necessary to push the second animation video from the first video frame of the second animation video, but to push the second animation video to the playback end from the switching video frame of the second animation video, so that the second animation video starts playing from the switching video frame. On the one hand, the playback efficiency of the second animation video is improved; on the other hand, since the switching video frame is the video frame in the starting segment of the second animation video that has the highest degree of matching with the video frame at the target interruption time point, playing the second animation video from the switching video frame can further improve the picture smoothness of the transition video switching to the second animation video.

[0164] As an example, Figure 8 2 is a timing diagram of a virtual character video playback according to an embodiment of the present application. Figure 8 As shown, in accordance with the timeline sequence, the current action of the virtual character is played at the beginning (i.e., the animation video corresponding to the current action, such as the first animation video); as the current action is played, an interruption instruction is received at the current playback time point of the virtual character; in response to the interruption instruction, the nearest interruption time point is searched backward on the timeline, and the transition video corresponding to the nearest interruption time point is searched; the animation video corresponding to the current action continues to be played, and when the current playback time point reaches the nearest interruption time point, the transition segment action is played, i.e., the transition video corresponding to the nearest interruption time point is played; when the transition segment action is finished playing, the target action is started to be played, i.e., the target animation video (e.g., the second animation video) indicated by the interruption instruction is played.

[0165] Exemplary Devices

[0166] Correspondingly, an embodiment of the present application also provides a transition video generating device.

[0167] See also Fig. 9 In an exemplary embodiment, a transition video generating device 900 is provided, and the transition video generating device 900 includes: a video acquiring unit 901, an interruption time point determining unit 902 and a transition video generating unit 903, wherein:

[0168] The video acquisition unit 901 is used to acquire a first animation video and a second animation video, where the first animation video and the second animation video are different animation videos of the same virtual character; the interruption time point determination unit 902 is used to determine the interruption time point on the timeline of the first animation video; the transition video generation unit 903 is used to generate a transition video corresponding to the interruption time point based on the video frame at the interruption time point and the video frame in the second animation video, and the transition video is used for the transition stage from switching from playing the first animation video to playing the second animation video.

[0169] In some embodiments, there are multiple interruption time points, and there are multiple transition videos, and the multiple interruption time points correspond one-to-one to the multiple transition videos.

[0170] In some embodiments, the interruption time point determination unit 902 is specifically used to: determine a plurality of interruption time points on the time axis according to a set first time interval.

[0171] In some embodiments, the transition video generating device 900 also includes: a first condition judgment unit (not shown in the figure), which is used to determine on the timeline, before determining multiple interruption time points according to a set first time interval, that the first animation video and the second animation video meet a first condition; the first condition includes: the video type of the first animation video is a silent type or an interactive type, and the video type of the second animation video is a silent type or an interactive type; and / or, the first condition includes: the number of animation videos under the video type to which the first animation video belongs is less than a first number threshold, and the number of animation videos under the video type to which the second animation video belongs is less than a second number threshold.

[0172] In some embodiments, the transition video generation unit 903 is specifically used to perform optical flow interpolation between video frames at multiple interruption time points and the first video frame in the second animation video to generate transition videos corresponding to multiple interruption time points.

[0173] In some embodiments, the interruption time point determination unit 902 is specifically used to determine a plurality of interruption time points on the timeline by matching the first animation video with the second animation video.

[0174] In some embodiments, a process of determining the i-th interruption time point among multiple interruption time points includes: determining the i-th video segment in the first animation video according to a set second time interval, where i is greater than or equal to 1; matching the video frame in the i-th video segment with the video frame in the starting segment of the second animation video to obtain a matching result, the matching result including the degree of matching between the video frame in the i-th video segment and the video frame in the starting segment; based on the matching result, confirming the i-th interruption time point on the timeline, the degree of matching between the video frame at the i-th interruption time point and the j-th video frame in the starting segment is the maximum matching degree in the matching result, where j is greater than or equal to 1.

[0175] In some embodiments, the i-th video segment is determined in the first animation video according to a set second time interval, including: when i is equal to 1, the first video frame in the first animation video is determined as the starting frame of the i-th video segment, and the second time interval is determined as the video duration of the i-th video segment, to obtain the i-th video segment; when i is greater than 1, the video frame at the i-1th interruption time point is determined as the starting frame of the i-th video segment, and the second time interval is determined as the video duration of the i-th video segment, to obtain the i-th video segment.

[0176] In some embodiments, the process of generating the i-th transition video among multiple transition videos includes: performing optical flow interpolation between a video frame at an i-th interruption time point and a j-th video frame to generate the i-th transition video.

[0177] In some embodiments, the transition video generating device 900 also includes: a second condition judgment unit (not shown in the figure), which is used to: determine whether the first animation video and the second animation video meet the second condition before determining multiple interruption time points on the timeline by matching the video frames in the first animation video with the video frames in the second animation video; the second condition includes: the video type of the first animation video is a silent type or an interactive type, and the video type of the second animation video is an action type; and / or, the second condition includes: the number of animation videos under the video type to which the first animation video belongs is less than a third number threshold, and the number of animation videos under the video type to which the second animation video belongs is greater than a fourth number threshold.

[0178] The transition video generation device provided in this embodiment belongs to the same application concept as the transition video generation method provided in the above embodiments of this application, and can execute the transition video generation method provided in any of the above embodiments of this application, and has the corresponding functional modules and beneficial effects of the execution method. For technical details not fully described in this embodiment, please refer to the specific processing content of the transition video generation method provided in the above embodiments of this application, and will not be repeated here.

[0179] Correspondingly, an embodiment of the present application also provides a virtual character video switching device.

[0180] See also Fig.10 In an exemplary embodiment, a virtual character video switching device 1000 is provided, and the virtual character video switching device 1000 includes: an interruption time point determination unit 1001, a transition video determination unit 1002, a transition video push unit 1003 and a switching video push unit 1004, wherein:

[0181] An interruption time point determination unit 1001 is used to determine a target interruption time point in the interruption time points included in the timeline of the first animation video when an interruption instruction for the first animation video is received when the first animation video is pushed to the playback end, the interruption instruction indicates to interrupt the playback of the first animation video and switch to the playback of the second animation video, the first animation video and the second animation video are different animation videos of the same virtual character; a transition video determination unit 1002 is used to determine a target transition video in the transition video corresponding to the interruption time point according to the target interruption time point, the transition video corresponding to the interruption time point is generated according to the transition video generation method of the first aspect or any embodiment of the first aspect; a transition video pushing unit 1003 is used to push the target transition video to the playback end when the push progress of the first animation video reaches the target interruption time point; a switching video pushing unit 1004 is used to push the second animation video to the playback end when the push progress of the target transition video reaches the end frame of the target transition video.

[0182] In some embodiments, the timeline includes multiple interruption time points related to the second animation video, there are multiple transition videos, and the multiple interruption time points correspond one-to-one to the multiple transition videos. The interruption time point determination unit 1001 is specifically used to: among the multiple interruption time points, determine that the target interruption time point is the interruption time point closest to the current playback time point of the first animation video.

[0183] In some embodiments, the switching video pushing unit 1004 is specifically used to: push the second animation video to the playback end starting from the switching video frame of the second animation video, the switching video frame being the video frame in the starting segment of the second animation video that has the highest degree of match with the video frame at the target interruption time point.

[0184] The virtual character video switching device provided in this embodiment belongs to the same application concept as the virtual character video switching method provided in the above embodiments of this application, and can execute the virtual character video switching method provided in any of the above embodiments of this application, and has the corresponding functional modules and beneficial effects of the execution method. For technical details not fully described in this embodiment, please refer to the specific processing content of the virtual character video switching method provided in the above embodiments of this application, and will not be repeated here.

[0185] The functions implemented by the various units in the device (transition video generating device or virtual character video switching device) provided in the above embodiments may be implemented by the same or different processors, respectively, and the embodiments of the present application are not limited thereto.

[0186] It should be understood that each unit in the device provided in the above embodiment can be implemented in the form of a processor calling software. For example, the device includes a processor, the processor is connected to a memory, and instructions are stored in the memory. The processor calls the instructions stored in the memory to implement any of the above methods or realize the functions of each unit of the device, wherein the processor can be a general-purpose processor, such as a CPU or a microprocessor, etc., and the memory can be a memory in the device or a memory outside the device. Alternatively, the unit in the device can be implemented in the form of a hardware circuit, and the functions of some or all units can be realized by designing the hardware circuit. The hardware circuit can be understood as one or more processors; for example, in one implementation, the hardware circuit is an ASIC, and the functions of some or all of the above units are realized by designing the logical relationship of the components in the circuit; for another example, in another implementation, the hardware circuit can be implemented by PLD, taking FPGA as an example, which can include a large number of logic gate circuits, and the connection relationship between the logic gate circuits is configured by a configuration file, so as to realize the functions of some or all of the above units. All units of the above device can be implemented in the form of a processor calling software, or in the form of a hardware circuit, or in part by a processor calling software, and the remaining part is implemented in the form of a hardware circuit.

[0187] In an embodiment of the present application, a processor is a circuit with the ability to process signals. In one implementation, the processor may be a circuit with the ability to read and run instructions, such as a CPU, a microprocessor, a GPU, or a DSP; in another implementation, the processor may implement certain functions through the logical relationship of a hardware circuit, and the logical relationship of the hardware circuit is fixed or reconfigurable, such as a hardware circuit implemented by an ASIC or PLD, such as an FPGA. In a reconfigurable hardware circuit, the process of the processor loading a configuration document to implement the hardware circuit configuration can be understood as the process of the processor loading instructions to implement the functions of some or all of the above units. In addition, it can also be a hardware circuit designed for artificial intelligence, which can be understood as an ASIC, such as an NPU, TPU, DPU, etc.

[0188] It can be seen that each unit in the above device can be one or more processors (or processing circuits) configured to implement the above method, such as: CPU, GPU, NPU, TPU, DPU, microprocessor, DSP, ASIC, FPGA, or a combination of at least two of these processor forms.

[0189] In addition, all or part of the units in the above device can be integrated together, or can be implemented independently. In one implementation, these units are integrated together and implemented in the form of a SOC. The SOC may include at least one processor for implementing any of the above methods or implementing the functions of each unit of the device. The type of the at least one processor may be different, for example, including a CPU and an FPGA, a CPU and an artificial intelligence processor, a CPU and a GPU, etc.

[0190] Exemplary Electronic Devices

[0191] Another embodiment of the present application also provides an electronic device. Fig.11 As shown, the electronic device includes: a memory 1100 and a processor 1110; wherein the memory 1100 is connected to the processor 1110 for storing programs; the processor 1110 is used to implement the transition video generation method or virtual character video switching method disclosed in any of the above embodiments by running the program stored in the memory 1100.

[0192] Specifically, the electronic device may further include: a bus, a communication interface 1120 , an input device 1130 and an output device 1140 .

[0193] The processor 1110, the memory 1100, the communication interface 1120, the input device 1130 and the output device 1140 are connected to each other via a bus.

[0194] A bus may include a pathway that transfers information between components of a computer system.

[0195] The processor 1110 may be a general-purpose processor, such as a general-purpose central processing unit (CPU), a microprocessor, etc., or an application-specific integrated circuit (ASIC), or one or more integrated circuits for controlling the execution of the program of the present application. It may also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.

[0196] The processor 1110 may include a main processor, and may also include a baseband chip, a modem, and the like.

[0197] The memory 1100 stores a program for executing the technical solution of the present application, and may also store an operating system and other key services. Specifically, the program may include a program code, and the program code includes a computer operation instruction. More specifically, the memory 1100 may include a read-only memory (ROM), other types of static storage devices that can store static information and instructions, a random access memory (RAM), other types of dynamic storage devices that can store information and instructions, a disk storage, a flash, and the like.

[0198] The input device 1130 may include a device for receiving data and information input by a user, such as a keyboard, a mouse, a camera, a scanner, a light pen, a voice input device, a touch screen, a pedometer, or a gravity sensor.

[0199] Output device 1140 may include devices that allow information to be output to a user, such as a display screen, printer, speaker, etc.

[0200] The communication interface 1120 may include any transceiver or the like to communicate with other devices or communication networks, such as Ethernet, a radio access network (RAN), a wireless local area network (WLAN), etc.

[0201] The processor 1110 executes the program stored in the memory 1100 and calls other devices, which can be used to implement the various steps of any transition video generation method or any virtual character video switching method provided in the above embodiments of the present application.

[0202] An embodiment of the present application also proposes a chip, which includes a processor and a data interface. The processor reads and runs a program stored in a memory through the data interface to execute the transition video generation method or the virtual character video switching method introduced in any of the above embodiments. The specific processing process and its beneficial effects can be found in the embodiment introduction of the above-mentioned transition video generation method or the virtual character video switching method.

[0203] Exemplary computer program products and storage media

[0204] In addition to the above-mentioned methods and devices, an embodiment of the present application may also be a computer program product, which includes computer program instructions, which, when executed by a processor, enable the processor to execute the steps of the transition video generation method according to various embodiments of the present application or the steps of the virtual character video switching method according to various embodiments of the present application described in any of the above embodiments of this specification.

[0205] The computer program product may be written in any combination of one or more programming languages ​​to write program codes for performing the operations of the embodiments of the present application, including object-oriented programming languages, such as Java, C++, etc., and conventional procedural programming languages, such as "C" language or similar programming languages. The program code may be executed entirely on the user computing device, partially on the user device, as an independent software package, partially on the user computing device and partially on a remote computing device, or entirely on a remote computing device or server.

[0206] In addition, an embodiment of the present application may also be a storage medium on which a computer program is stored, and the computer program is executed by a processor to execute the steps of the transition video generation method according to various embodiments of the present application or the steps of the virtual character video switching method according to various embodiments of the present application described in any of the above embodiments of this specification.

[0207] For the aforementioned method embodiments, for the sake of simplicity, they are all described as a series of action combinations, but those skilled in the art should be aware that the present application is not limited by the order of the actions described, because according to the present application, some steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the present application.

[0208] It should be noted that each embodiment in this specification is described in a progressive manner, and each embodiment focuses on the differences from other embodiments, and the same or similar parts between the embodiments can be referred to each other. For the device embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0209] The steps in the methods of each embodiment of the present application can be adjusted in order, combined and deleted according to actual needs, and the technical features recorded in each embodiment can be replaced or combined.

[0210] The modules and sub-modules in the devices and terminals in the various embodiments of the present application can be combined, divided and deleted according to actual needs.

[0211] In the several embodiments provided in the present application, it should be understood that the disclosed terminals, devices and methods can be implemented in other ways. For example, the terminal embodiments described above are only schematic, for example, the division of modules or submodules is only a logical function division, and there may be other division methods in actual implementation, for example, multiple submodules or modules can be combined or integrated into another module, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or modules, which can be electrical, mechanical or other forms.

[0212] The modules or submodules described as separate components may or may not be physically separated, and the components of the modules or submodules may or may not be physical modules or submodules, that is, they may be located in one place, or they may be distributed on multiple network modules or submodules. Some or all of the modules or submodules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0213] In addition, each functional module or submodule in each embodiment of the present application may be integrated into one processing module, or each module or submodule may exist physically separately, or two or more modules or submodules may be integrated into one module. The above-mentioned integrated modules or submodules may be implemented in the form of hardware or in the form of software functional modules or submodules.

[0214] Professionals may further appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in the above description according to function. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians may use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0215] The steps of the method or algorithm described in conjunction with the embodiments disclosed herein may be implemented directly by hardware, software units executed by a processor, or a combination of the two. The software units may be placed in a random access memory (RAM), a memory, a read-only memory (ROM), an electrically programmable ROM, an electrically erasable programmable ROM, a register, a hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art.

[0216] Finally, it should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the presence of other identical elements in the process, method, article or device including the elements.

[0217] The above description of the disclosed embodiments enables those skilled in the art to implement or use the present application. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application will not be limited to the embodiments shown herein, but will conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for generating a transition video, characterized in that: include: Acquire a first animation video and a second animation video, wherein the first animation video and the second animation video are different animation videos of the same virtual character; Determine an interruption time point on the timeline of the first animation video; A transition video corresponding to the interruption time point is generated according to the video frame at the interruption time point and the video frame in the second animation video, and the transition video is used for the transition stage from switching from playing the first animation video to playing the second animation video.

2. The method for generating transitional video according to claim 1, characterized in that: There are multiple interruption time points, and there are multiple transition videos. The multiple interruption time points correspond one to one with the multiple transition videos.

3. The method for generating transitional video according to claim 2, characterized in that: Determining the interruption time point on the time axis of the first animation video includes: On the time axis, the plurality of interruption time points are determined according to a set first time interval.

4. The method for generating transitional video according to claim 3, characterized in that: Before determining the plurality of interruption time points on the time axis according to the set first time interval, the method further includes: Determining that the first animation video and the second animation video satisfy a first condition; The first condition includes: the video type of the first animation video is a silent type or an interactive type, and the video type of the second animation video is a silent type or an interactive type; And / or, the first condition includes: the number of animation videos under the video type to which the first animation video belongs is less than a first quantity threshold, and the number of animation videos under the video type to which the second animation video belongs is less than a second quantity threshold.

5. The method for generating transitional video according to claim 3, characterized in that: The step of generating a transition video corresponding to the interruption time point according to the video frame at the interruption time point and the video frame in the second animation video includes: Optical flow interpolation is performed between the video frames at the multiple interruption time points and the first video frame in the second animation video to generate transition videos corresponding to the multiple interruption time points respectively.

6. The method for generating transitional video according to claim 2, characterized in that: Determining the interruption time point on the time axis of the first animation video includes: The plurality of interruption time points are determined on the timeline by matching the first animation video with the second animation video.

7. The method for generating transitional video according to claim 6, characterized in that: The process of determining the i-th interruption time point among the multiple interruption time points includes: According to the set second time interval, determining an i-th video segment in the first animation video, where i is greater than or equal to 1; Matching the video frames in the i-th video segment with the video frames in the starting segment of the second animation video to obtain a matching result, wherein the matching result includes a matching degree between the video frames in the i-th video segment and the video frames in the starting segment; According to the matching result, the i-th interruption time point is confirmed on the timeline, and the matching degree between the video frame at the i-th interruption time point and the j-th video frame in the starting segment is the maximum matching degree in the matching result, and j is greater than or equal to 1.

8. The method for generating transitional video according to claim 7, characterized in that: The step of determining the i-th video segment in the first animation video according to the set second time interval includes: When i is equal to 1, the first video frame in the first animation video is determined as the starting frame of the i-th video segment, and the second time interval is determined as the video duration of the i-th video segment, to obtain the i-th video segment; When i is greater than 1, the video frame at the i-1th interruption time point is determined as the starting frame of the i-th video segment, and the second time interval is determined as the video duration of the i-th video segment to obtain the i-th video segment.

9. The method for generating transitional video according to claim 7, characterized in that: The process of generating the i-th transition video among the multiple transition videos includes: Optical flow interpolation is performed between the video frame at the i-th interruption time point and the j-th video frame to generate the i-th transition video.

10. The method for generating transitional video according to claim 6, characterized in that: Before determining the plurality of interruption time points on the timeline by matching the video frames in the first animation video with the video frames in the second animation video, the method further includes: Determining that the first animation video and the second animation video satisfy a second condition; The second condition includes: the video type of the first animation video is a silent type or an interactive type, and the video type of the second animation video is an action type; And / or, the second condition includes: the number of animation videos in the video type to which the first animation video belongs is less than a third quantity threshold, and the number of animation videos in the video type to which the second animation video belongs is greater than a fourth quantity threshold.

11. A method for switching a virtual character video, characterized in that: include: In the case of pushing a first animation video to a playback end, if an interruption instruction for the first animation video is received, a target interruption time point is determined in the interruption time points included in the timeline of the first animation video, the interruption instruction instructs to interrupt the playback of the first animation video and switch to the playback of the second animation video, the first animation video and the second animation video being different animation videos of the same virtual character; According to the target interruption time point, determining a target transition video in the transition video corresponding to the interruption time point, wherein the transition video corresponding to the interruption time point is generated by the transition video generation method according to any one of claims 1 to 10; When the push progress of the first animation video reaches the target interruption time point, push the target transition video to the playback end; When the push progress of the target transition video reaches the end frame of the target transition video, the second animation video is pushed to the playback end.

12. The method for switching virtual character videos according to claim 11, characterized in that: The timeline includes a plurality of interruption time points related to the second animation video, the transition videos are multiple, the plurality of interruption time points correspond to the plurality of transition videos one by one, and determining a target interruption time point among the interruption time points included in the timeline of the first animation video includes: Among the multiple interruption time points, it is determined that the target interruption time point is the interruption time point closest to the current playing time point of the first animation video.

13. The method for switching virtual character videos according to claim 11, characterized in that: The pushing the second animation video to the playback end includes: Starting from the switching video frame of the second animation video, the second animation video is pushed to the playback end, and the switching video frame is the video frame in the starting segment of the second animation video that has the highest matching degree with the video frame at the target interruption time point.

14. A transition video generating device, characterized in that: include: A video acquisition unit, configured to acquire a first animation video and a second animation video, wherein the first animation video and the second animation video are different animation videos of the same virtual character; An interruption time point determination unit, used to determine an interruption time point on the time axis of the first animation video; A transition video generation unit is used to generate a transition video corresponding to the interruption time point based on the video frame at the interruption time point and the video frame in the second animation video, and the transition video is used for the transition stage from switching from playing the first animation video to playing the second animation video.

15. A virtual character video switching device, characterized in that: include: an interruption time point determination unit, configured to, when a first animation video is pushed to a playback end, determine a target interruption time point among the interruption time points included in the timeline of the first animation video if an interruption instruction for the first animation video is received, wherein the interruption instruction indicates interrupting the playback of the first animation video and switching to the playback of a second animation video, wherein the first animation video and the second animation video are different animation videos of the same virtual character; a transition video determination unit, configured to determine, according to the target interruption time point, a target transition video in the transition video corresponding to the interruption time point, wherein the transition video corresponding to the interruption time point is generated by the transition video generation method according to any one of claims 1 to 10; A transition video pushing unit, configured to push the target transition video to the playback end when the pushing progress of the first animation video reaches the target interruption time point; The switching video pushing unit is used to push the second animation video to the playback end when the pushing progress of the target transition video reaches the end frame of the target transition video.

16. An electronic device, characterized in that: including memory and processor; The memory is connected to the processor and is used to store programs; The processor is used to implement the transition video generation method as described in any one of claims 1 to 10, or the virtual character video switching method as described in any one of claims 11 to 13, by running the program in the memory.

17. A computer program product, characterized in that It comprises a computer program, which, when executed by a processor, implements the transition video generation method as described in any one of claims 1 to 10, or implements the virtual character video switching method as described in any one of claims 11 to 13.

Images