Techniques for motion editing for character animation

Abstract

A technique for modifying a character animation via a user interface includes displaying a timeline that includes frame numbers as units of time; displaying, along the timeline, a set of key poses associated with a set of frame numbers for a character; displaying a set of tracks overlaid on the set of key poses, where each track in the set of tracks is displayed as a curve that passes through a particular joint of the character across the set of key poses; and receiving, via the user interface, a modification to a first key pose included in the set of key poses. A technique for automatically extracting a set of key poses from a character animation based on a plurality of joint tracks. A technique for retiming a segment of a character animation.

Description

[0001] Cross-references to related applications This application claims the benefit of U.S. Provisional Application No. 63 / 597,652, filed November 9, 2023, entitled “TECHNIQUES FOR INTEGRATING SPATIAL AND TEMPORAL MOTION EDITING FOR CHARACTER ANIMATION IN VIRTUALREALITY”, and U.S. Application No. 18 / 916,485, filed October 15, 2024, entitled “TECHNIQUES FOR MOTION EDITING FOR CHARACTER ANIMATIONS”. This application claims the rights of U.S. application “TECHNIQUES FORMOTION EDITING FOR CHARACTER ANIMATIONS”, serial number 18 / 916,487, filed October 15, 2024, and also claims the rights of U.S. application “TECHNIQUES FORMOTION EDITING FOR CHARACTER ANIMATIONS”, serial number 18 / 916,488. The subjects of these related applications are hereby incorporated by reference. Technical Field

[0002] The various embodiments generally relate to computer science, and more specifically, to techniques for motion editing of character animation. Background Technology

[0003] As creation tools for motion capture in animation become computationally cheaper and technically easier to use, 3D animation is being implemented more and more frequently across a variety of applications. For example, animated 3D characters are becoming increasingly common in online videos, video games, and virtual environments. At this point, motion capture technology has become much easier for both experienced and non-expert users, who can now create character animations using a wide range of available motion capture devices such as cameras, monocular video, and sensors on virtual reality (VR) headsets. Using these types of devices, users can easily generate character animations consisting of a set of frames, where each frame includes motion capture data specifying a particular pose of the character. The character's pose typically changes across the set of frames to produce animated clips of the character. Once the character animation has been created, it can be edited / modified via an animation editing application, which allows editing / modification of the spatial and temporal data associated with the character animation through spatial and temporal controls provided in desktop-based or VR-based user interfaces.

[0004] Despite the fact that motion capture technology and related devices have made it easier for users to create / create character animations, the standard editing applications used to edit the resulting character animations remain unintuitive and difficult to learn for non-professional users. For example, in many editing applications, spatial and temporal controls for editing spatial and temporal data are not integrated / embedded in the same area of ​​the user interface. Instead, these controls are implemented separately within the user interface in an unrelated manner. Therefore, when editing character animation, users typically must first interact with spatial controls, such as a configurable 3D model of the character, to edit the spatial aspects of the character animation, and then must separately interact with unrelated temporal controls, such as a configurable animation timeline, to separately edit the temporal aspects of the character animation. Thus, to modify the character's movement within a segment of animation, the user must constantly switch between the spatial view of the 3D model and the temporal view of the animation timeline to modify multiple frames of the animation. In addition, non-professional users often find it difficult to achieve the overall desired motion of a character because editing applications typically do not indicate how changes made to the 3D model in the current frame affect the 3D model in adjacent frames along the animation timeline, and non-professional users do not have enough experience to understand what changes need to be made to the 3D model in adjacent frames along the animation timeline to effectively create the overall desired motion of a character.

[0005] Another drawback of conventional editing applications is that they typically don't offer any way to automatically extract key poses from character animations for editing. More specifically, a given character animation can include a large number of different poses for various characters within that animation. However, most of those poses are often "non-essential," such as static or near-static poses, which are often unimportant when editing character animations. Instead, a relatively small subset of the character poses in an animation is considered the necessary "key poses," such as transitional poses between static or near-static poses, which represent the overall motion in the character animation. Editing the key poses included in a character animation is generally more effective and efficient in changing the overall motion of the animation than also editing the non-essential poses. However, for most editing applications, users need a certain level of expertise to accurately extract the relevant key poses from the character animation, and for non-expert users, this often leads to misidentification of key poses, reducing the effectiveness of editing such key poses.

[0006] Another drawback of conventional editing applications is that they don't provide novice users with a simple and intuitive way to retime specific segments of character animation. Besides modifying poses in character animation, another common editing task users perform is retimeting segments of character animation. In this task, users want to slow down or speed up the character's movement in a specific segment of animation. Conventionally, to slow down the character's movement in a specific segment, a user can add a new frame to that segment. Conversely, to speed up the character's movement in a specific segment, a user can remove a frame from that segment. However, users need expertise to know where to add a new frame or remove a current frame within the retimed segment to provide smooth and natural movement for the character. Non-expert users who perform such retimed operations using conventional editing applications will often fail to accurately add and remove frames to provide smooth and natural movement for the character within the retimed segment, thus degrading the quality of the character animation.

[0007] As explained above, there is a need in the art for more efficient techniques for editing character animations. Summary of the Invention

[0008] Various embodiments include a computer-implemented method for modifying character animation via a user interface. The computer-implemented method includes: displaying a timeline including frame numbers as units of time; displaying a set of key poses along the timeline based on a set of frame numbers associated with a set of key poses of the character; displaying a set of trajectories superimposed on the set of key poses, wherein each trajectory in the set of trajectories is displayed as a curve passing through a specific joint of the character across the set of key poses; and receiving a modification to a first key pose included in the set of key poses via the user interface.

[0009] The disclosed technology has at least one technical advantage over existing technologies in that it allows spatial and temporal controls for editing character animation to be integrated and housed together within a 3D timeline user interface (UI) of the animation editing application. The 3D timeline UI simultaneously displays a set of key poses and a set of trajectories overlaid on the animation timeline. Each displayed trajectory crosses specific joints of the character across the set of key poses, helping the user visualize the character's movement in different poses between key poses. The user can modify the current key pose displayed along the animation timeline, and in response, the 3D timeline UI automatically propagates the corresponding change to poses near the key pose and indicates these changes via an updated set of trajectories that helps the user visualize the modified movement of the character along the animation timeline. Utilizing this functionality, the 3D timeline UI simplifies animation editing for non-professional users compared to existing methods because it eliminates the need for users to constantly switch between spatial controls / views of poses and temporal controls / views of the animation timeline to modify the character's animation movement. Furthermore, when a change is made to the current key pose, the 3D timeline UI automatically indicates the resulting changes to nearby poses via an updated trajectory, eliminating the need for users to accurately predict how a change in one pose will affect nearby poses, as required by existing methods. In this way, the disclosed technology provides an accessible and intuitive interface that allows non-professional users to edit character animations more easily and efficiently compared to conventional editing applications. These technological advantages offer one or more technological improvements over existing methods.

[0010] Various embodiments include a computer-implemented method for automatically extracting a set of key poses from a character animation. The computer-implemented method includes: determining multiple trajectories for multiple joints associated with a character included in the character animation, wherein each trajectory includes a set of positions of a set of poses of the corresponding joints across the character animation; extracting a set of key poses from the character animation based on the multiple trajectories; and displaying the set of key poses within a user interface for further processing.

[0011] At least one technical advantage of the disclosed technology over existing technologies is that it enables animation editing applications to automatically and accurately extract key pose sets from character animations, which is impossible using existing methods. Therefore, the disclosed technology avoids non-professional users inaccurately selecting key pose sets from character animations. Specifically, the disclosed technology enables animation editing applications to extract key pose sets that accurately represent the motion of a character animation based on the trajectories of multiple joints in various poses of the character. Automatically identifying and extracting accurate key pose sets that represent character animations also improves the overall accuracy and quality of editing work performed on character animations. These technical advantages provide one or more technical improvements over existing methods.

[0012] Various embodiments include a computer-implemented method for retiming segments included in a character animation. The computer-implemented method includes: displaying a set of key poses associated with the character animation along an animation timeline; receiving a selection of a plurality of key poses included in the set of key poses, the plurality of key poses defining the selected segments included in the character animation; receiving a command for expanding or contracting the selected segments along the animation timeline, wherein the command specifies the updated segments; and performing one or more retiming operations on the selected segments to generate the updated segments.

[0013] The disclosed technology has at least one technical advantage over existing technologies in that it implements a retiming function in animation editing applications. This retiming function can be automatically applied to selected segments of character animation to slow down or speed up the character's movement in the selected segments, which is impossible to achieve using existing methods. In operation, the user expands or contracts the selected segment to specify the updated segment. In response, the retiming function automatically generates a velocity curve function based on the amount of expansion or contraction and applies the velocity curve function to the selected segment to add new frames or remove frames from the selected segment, thereby automatically generating an updated segment in which the character has smooth and natural movement. In this way, the disclosed technology provides a retiming function that allows both professional and non-professional users to retiming given segments of character animation more accurately and efficiently, thereby improving the quality of character animation compared to what can be achieved using existing methods. These technical advantages provide one or more technical improvements over existing methods. Attached Figure Description

[0014] To gain a detailed understanding of the features described above in the various embodiments, reference can be made to various embodiments to describe in more detail the inventive concept briefly outlined above, some of which are illustrated in the accompanying drawings. However, it should be noted that the drawings illustrate only typical embodiments of the inventive concept and should therefore not be construed as limiting the scope in any way, and other equivalent embodiments exist.

[0015] Figure 1 A system configured to implement one or more aspects of various embodiments is shown; Figure 2 According to various embodiments Figure 1 Screenshot of the 3D timeline UI; Figure 3 Based on the various embodiments and Figure 2 A screenshot of the selected joint associated with the current key pose, included in the 3D timeline UI. Figure 4 According to various embodiments Figure 3 A screenshot showing the modifications made to the selected joint; Figures 5A to 5B A flowchart illustrating the steps of a method for editing character animation via a 3D timeline UI, according to various embodiments; Figure 6 A flowchart illustrating the steps of a method for displaying a visual representation of an activity trajectory dataset within a 3D timeline UI, according to various embodiments; Figure 7 A flowchart illustrating the steps of a method for automatically extracting a set of key poses from character animation, according to various embodiments; Figure 8 The retiming function elements are illustrated according to various embodiments. Figure 1 Screenshot of the 3D timeline UI; Figure 9 The activated retiming function element is shown according to various other embodiments. Figure 8 Screenshot of the 3D timeline UI; Figure 10 The extended fragment is shown according to various other embodiments. Figure 9 Screenshot of the 3D timeline UI; Figure 11 The contracted segment is shown according to various other embodiments. Figure 9 Screenshot of the 3D timeline UI; Figure 12 A flowchart illustrating the steps of a method for retiming selected segments of character animation according to various embodiments is provided. Figure 13The velocity profiles of the expanded and updated segments implemented via a retiming function according to various embodiments are shown; and Figure 14 The velocity profiles of the shrunken, updated segments implemented via a retiming function according to various embodiments are shown. Detailed Implementation

[0016] In the following description, numerous specific details are set forth to provide a more thorough understanding of various embodiments. However, it will be apparent to those skilled in the art that the inventive concepts can be practiced without one or more of these specific details.

[0017] As used in this article, a "trajectory" is associated with a specific joint of a character in a character animation, which comprises a set of poses depicted by the character across a set of frames. A trajectory for a specific joint comprises a dataset containing a set of joint positions (3D spatial coordinates) of the poses across the set of frames in the character animation. The joint positions in the trajectory's set of joint positions are ordered chronologically based on the frame number corresponding to each joint position. The representation of a trajectory for a specific joint can be displayed on a 3D timeline UI as a 3D curve passing through the specific joint for each key pose across the character animation; this key pose is also displayed in the 3D timeline UI to help users visualize the character's motion between key poses.

[0018] As used herein, a "key pose set" for a character in a character animation comprises a subset of all poses of the character in the animation, representing the character's overall motion. The remaining subset of poses in the character animation may be referred to as the "non-essential pose set" or "non-key pose set." The key pose set is extracted / identified from the character animation to depict the character's overall motion in a concise view and highlight important poses within the animation. Frames containing key poses may be referred to as "keyframes." The key pose set is extracted / identified from the character animation based on one or more trajectories of one or more joints of the character. In some embodiments, the key pose set is extracted / identified from the character animation based on multiple trajectories corresponding to multiple selected joints of the character.

[0019] As used herein, “IE interface” includes 3D-specific hardware and software components for interacting with a 3D immersive environment (IE). For example, 3D hardware may include a 3D display, one or more 3D controllers that operate in 3D, one or more tracking devices, and one or more cameras. For example, 3D software may include a 3D timeline UI engine that generates an immersive 3D timeline UI on a 3D display. Examples of IE interfaces include virtual reality (VR) interfaces and augmented reality (AR) interfaces.

[0020] The disclosed technology enables a 3D timeline UI in animation editing applications to integrate spatial and temporal controls for editing character animation. Character animation comprises a set of poses, including a set of optional poses and a set of key poses representing the character's overall movement within the animation. The 3D timeline UI displays an animation timeline including a time axis with frame numbers as the unit of time. The 3D timeline UI also displays key pose sets overlaid / overlaid at various points on the animation timeline based on the frame numbers of the animation timeline and the frame numbers associated with the key pose sets. The 3D timeline UI further displays a set of trajectories overlaid / overlaid on the key pose sets based on the character's joint sets. Each displayed trajectory consists of a continuous curve traversing a specific joint of the character across the key pose set, helping users visualize the passage of time and the character's movement between poses. Users can directly edit the current key pose displayed in the 3D timeline UI. In response, the 3D timeline UI automatically propagates the corresponding change to one or more poses near the current key pose and automatically updates the trajectory set based on changes to the current key pose and nearby poses. The 3D timeline UI also automatically displays an updated set of trajectories to indicate changes to the current key pose and nearby poses, helping users visualize the updated motion of the character that occurs between key poses. Changes to the current key pose and nearby poses generate modified character animations, which can be stored and further modified based on user edits.

[0021] The disclosed technology also enables animation editing applications to automatically extract a set of key poses from character animation. This set of key poses is automatically extracted from the character animation based on a set of joint trajectories associated with the character animation. The animation editing application initially generates a trajectory for each joint of the character. The trajectory for a specific joint includes a dataset comprising a sorted set of positions for all poses of that joint across all frames of the character animation. The set of trajectory positions is sorted chronologically based on the frame number corresponding to that position set. The animation editing application can generate and store the trajectory dataset for each joint of the character. The animation editing application then receives selections of multiple joints of the character (either by default or by the user) and retrieves multiple selected trajectories corresponding to the multiple selected joints. The animation editing application then extracts / identifies a set of key poses from the character animation based on the multiple selected trajectories. The user can also select multiple new joints of the character via a joint graph, causing the animation editing application to dynamically identify a new set of key poses for the character animation based on the new multiple selected joints. The animation editing application also displays the set of key poses, allowing the user to select and directly edit key poses within the set to modify the character animation.

[0022] The disclosed technology further implements a retiming function in animation editing applications, which is applied to selected segments of character animation to slow down or speed up the character's movement in the selected segments. The animation editing application's 3D timeline UI displays an animation timeline including a time axis, with frame numbers as the time unit. The 3D timeline UI also displays a set of key poses superimposed on the animation timeline, each key pose superimposed at the frame number corresponding to a specific key pose. Users can select segments of character animation by selecting a pair of key poses displayed in the 3D timeline UI; the selected pair of key poses defines the selected segment. Users can then expand (increase) or contract (decrease) the frame distance between the selected pair of key poses along the animation timeline to slow down or speed up the character's movement in the selected segment, respectively. Expanding or contracting the selected segment along the animation timeline specifies the updated segment. In response, the animation editing application automatically performs one or more retiming operations on the selected segment based on the degree to which the selected segment is expanded or contracted along the animation timeline. Specifically, the 3D timeline UI automatically generates a velocity curve function based on the amount of expansion or contraction. This velocity curve function is automatically applied to the pose / frame of the selected segment to provide a smooth and natural slowing down or speeding up of the character's movement in the updated segment.

[0023] System Overview Generally, two-dimensional (2D) computing environments are provided by computing devices that execute 2D applications via 2D interfaces, such as desktop environments implemented via desktop interfaces. Conversely, three-dimensional (3D) immersive environments (IEs) are provided by computing devices that execute 3D applications via IE interfaces, such as VR or AR environments implemented via virtual reality (VR) interfaces or augmented reality (AR) interfaces, respectively. Compared to performing animation editing via 2D environments and 2D interfaces, performing animation editing via 3D environments and IE interfaces can provide users with a better sense of space and scale, and improve productivity. In the embodiments described below, animation editing applications can be implemented in a 3D environment via IE interfaces.

[0024] Figure 1A system 100 configured to implement one or more aspects of various embodiments is illustrated. As shown, the IE system 100 includes, but is not limited to, computer systems 106 interconnected via a network 192, an animation database 180, and an animation server 190, which may be a wide area network (WAN) (such as the Internet), a local area network (LAN), or any other suitable network. The animation server 190 may include computer hardware (such as a processor, memory, and storage devices) for performing animation editing support services, such as frame interpolation, when requested by the computer system 106. According to the embodiments described herein, the animation database 180 may store various character animations that can be loaded into the computer system 106 for editing.

[0025] Computer system 106 may include at least one processor 102, input / output (I / O) device 108, and memory unit 104 coupled together. Computer system 106 may include a server, personal computer, laptop or tablet computer, mobile computer system, or any other device suitable for practicing the various embodiments described herein. For example, computer system 106 may include a cloud server to provide animation editing as a cloud service to other computer systems on network 192. Generally, each processor 102 may be any technically feasible processing device or hardware unit capable of processing data and executing software applications and program code. Each processor 102 executes software and performs the functions and operations set forth in the embodiments described herein. For example, processor 102 may include a general-purpose processor (such as a central processing unit), a special-purpose processor (such as a graphics processing unit), an application-specific processor, a field-programmable gate array or other programmable logic device, discrete gate or transistor logic, discrete hardware components, or any combination of different processing units.

[0026] Memory unit 104 may include a hard disk, a random access memory (RAM) module, a flash memory unit, or any other type of memory unit or combination thereof. Processor 102 and I / O device 108 read data from memory 104 and write data to memory. Memory unit 104 stores software applications and data. Instructions derived from the software configuration within memory unit 104 are executed by processor 102 to implement the inventive operations and functions described herein.

[0027] I / O device 108 is also coupled to memory 104 and may include devices capable of receiving input and devices capable of providing output. I / O device 108 may include input and output devices not specifically listed in IE hardware 170, such as network cards for connecting to network 192, speakers, manufacturing equipment (such as 3D printers), etc. Additionally, I / O devices may include devices capable of both receiving input and providing output, such as touchscreens, Universal Serial Bus (USB) ports, etc.

[0028] As shown, computer system 106 is also connected to various IE hardware 170, including but not limited to IE headset 172, one or more IE controllers 176, and one or more tracking devices 178. Each IE controller 176 includes IE-tracking devices tracked by tracking devices 178, which determine the 3D position / location information of the IE controller 176. For example, IE controller 176 may include a 6-DOF controller operating in 3D. IE headset 172 may display images in 3D stereoscopic images, such as a 3D timeline UI 130. IE headset 172 includes IE-tracking devices tracked by tracking devices 178, which determine the 3D position / location information of the IE headset 172. In some embodiments, tracking devices 178 track the 3D position of the user's viewpoint by tracking the 3D position of the IE headset 172. In some embodiments, IE hardware 170 includes VR hardware 170, including but not limited to VR headset 172, one or more VR controllers 176, and one or more VR tracking devices 178. In other embodiments, IE hardware 170 includes AR hardware 170, including but not limited to AR headset 172, one or more AR controllers 176, and one or more AR tracking devices 178. In further embodiments, IE hardware 170 includes other types of IE hardware for displaying and interacting with other types of 3D immersive environments.

[0029] Memory unit 104 stores animation editing application 120, 3D timeline UI 130, character animation 150, multiple trajectory datasets 160, and key pose sets 162. Animation editing application 120 includes a 3D timeline UI engine 110 and a retiming engine 140. Although shown as separate software components, the 3D timeline UI engine 110 and the retiming engine 140 can be integrated into a single software component. For example, in other embodiments, the retiming engine 140 can be integrated with the 3D timeline UI engine 110. In a further embodiment, animation editing application 120 can be stored on and executed on an IE head-mounted device 172.

[0030] Animation editing application 120 (such as stored in memory unit 104 and by...) Figure 1The processor 102 (executing the process) includes a 3D timeline UI engine 110 for generating a 3D timeline UI 130 displayed in the IE head-mounted device 172. The 3D timeline UI engine 110 also provides the underlying functionality of the 3D timeline UI 130 described herein. The 3D timeline UI 130 may include a 3D virtual environment that a user can interact with via the IE controller 176. For example, a user can directly edit key poses displayed in the 3D timeline UI 130 via the IE controller 176. The animation editing application 120 also includes a retiming engine 110 for performing retiming functions. The animation editing application 120 operates on character animation 150, which may be downloaded, for example, from an animation database 180. The character animation 150 may be captured, for example, using a vision system (such as a color camera or depth camera) that captures frame-by-frame motion capture data of the character. The animation editing application 120 operates on the character animation 150 to generate multiple trajectory datasets 160 and a set of key poses 162.

[0031] Character animation 150 includes source files containing frames of motion capture data for animation clips of the character, each frame having an associated frame number indicating its temporal order within character animation 150. Character animation 150 includes a spatiotemporal dataset comprising multiple frames (representing the temporal aspect of the animation), each frame containing metadata specifying a particular pose of the character (representing the spatial aspect of the animation). Specifically, each frame specifies a 3D model of the character configured in a particular pose. The 3D model of the character may include multiple joints organized and connected in a tree-like hierarchical data structure. The 3D model may include a set of motion joints forming a kinematic chain, such that a modification to the position and / or rotation of one joint can cause the position and / or rotation modification to be propagated to one or more other joints in that set of motion joints. Therefore, the joints of the 3D model may be dependent on each other and correlated with each other in terms of position and / or rotation. The 3D model of the character may include, for example, skeletal rigging, joint frames, articulated characters, unrigged characters, etc.

[0032] A character's joints can be any predetermined part / point on the character that can be used as an effector, including points that are generally considered "joints" (such as the right elbow, right shoulder, left knee, etc.). However, a character's joints can also be predetermined parts / points on the character that are not generally considered "joints" (such as key vertices) (such as the forehead, right eye, left ear, right forearm, etc.). Furthermore, a character can be any type of 3D object, including humanoid objects or non-humanoid objects, such as animals, cars, airplanes, ships, etc. Therefore, for non-humanoid objects, a character's joints can be any predetermined part / point on the character, such as a logo on a car, a steering wheel on a car, a specific window on an airplane, etc.

[0033] Each frame of the character animation 150 specifies metadata for each joint of the character, including the position and rotation of each joint in the frame. The position of a joint can be represented by 3D spatial coordinates (xyz coordinates) in the local coordinate system of the 3D model. The rotation of a joint can be represented by three rotation values ​​relative to the three possible axes of rotation (xyz axes). The character animation 150 may also include metadata describing the character's joints, the total number of joints in the character, and how the joints are organized and connected in a tree-hierarchical data structure.

[0034] Animation editing application 120 performs operations on character animation 150 to generate trajectory datasets 160, which include trajectory datasets for each joint of the character. Specifically, animation editing application 120 processes character animation 150 to extract the 3D position of the joint in each pose / frame of character animation 150 for each joint of the character. Therefore, the trajectory dataset of the joints includes a sorted set of 3D positions of the joints across the set of poses / frames of the character animation, which is sorted chronologically based on the frame number corresponding to the 3D position set.

[0035] Then, the animation editing application 120 can use the following about Figure 6 The trajectory display technique described in Equation 1 is used to display the representation of the trajectory dataset 160 as a 3D curve in the 3D timeline UI 130. In some embodiments, to avoid visual clutter, only some, not all, of the trajectory datasets 160 are displayed as 3D curves in the 3D timeline UI 130, and only the "active" trajectory datasets of "active" joints are displayed in the 3D timeline UI 130. Active joints can be selected by default by the animation editing application 120 or selected by the user via a joint graph. In some embodiments, active joints include a subset of all joints of the character. In some embodiments, multiple active trajectory datasets of multiple active joints are displayed in the 3D timeline UI 130. Additionally, the animation editing application 120 may use the following... Figure 7 The key pose extraction technique described in Equations 2-3 automatically extracts / identifies a set of key poses 162 from the character animation 150 based on an activity trajectory dataset. The key pose set 162 is also displayed in the 3D timeline UI 130, whereby the activity trajectories of the active joints are superimposed onto the key pose set 162. See below for more details. Figure 2 As described, the 3D Timeline UI 130 also displays a timeline with a time axis that uses frame numbers as the time unit. The 3D Timeline UI 130 is displayed on the IE headset 172.

[0036] Animation editing within the 3D timeline UI Figure 2 According to various embodiments Figure 1A screenshot of the 3D Timeline UI 130. The 3D Timeline UI 130 is generated based on a humanoid character animation 150, which depicts the humanoid character's movements while dancing. As shown, the 3D Timeline UI 130 displays an animation timeline 210, one or more continuous curve representations of the activity trajectories 220 (such as 220a and 220b) of one or more active joints of the character, one or more key poses 230 of the character animation (such as 230a, 230b, 230c, etc.), a joint diagram 250, playback control buttons 260, view control buttons 270, and an optional retiming button 840 (discussed below in the section related to the retiming function). In some embodiments, the animation timeline 210, one or more curve representations of the activity trajectories 220, and one or more key poses 230 are displayed concurrently in the 3D Timeline UI 130 at overlapping times.

[0037] The animation timeline 210 includes a time axis (shown horizontally along the x-axis) that uses frame numbers as the unit of time. In some embodiments, the animation timeline 210 displays the frame number of each key pose 230 along the animation timeline 210, and does not display the frame numbers of any non-key poses along the animation timeline 210. In some embodiments, each displayed frame number of a key pose 230 includes a selectable frame number for navigation to and selection of the corresponding key pose 230. For example, in response to receiving a selection of a first selectable frame number corresponding to a first key pose 230a, the animation editing application 120 can navigate to and select the first key pose 230a on the animation timeline 210 for editing.

[0038] exist Figure 2 In the example, the active joints include the character's right wrist and left ankle joints. Joint diagram 250 shows a visualization of all the character's joints (e.g., 10 joints in total) arranged in a tree hierarchy, which shows the organization of the joints and the relationships between them (e.g., parent-child relationships). The character's joints can include a set of mobile joints. Joint diagram 250 visually represents each joint as a selectable circle in the joint tree hierarchy, where dark circles represent selected / active joints and light circles represent unselected / inactive joints.

[0039] As shown in the figure, joint diagram 250 indicates that the character's right wrist joint and left ankle joint are selected as active joints. In some embodiments, the 3D timeline UI 130 does not display a representation of the trajectory of all the character's joints, but only a representation of the trajectory of the character's active joints. Therefore, the representation of the active trajectory 220 includes a representation of the first active trajectory 220a of the character's right wrist joint and a representation of the second active trajectory 220b of the left ankle joint. The 3D timeline UI 130 uses the following information about... Figure 6 The trajectory display technique described in Equation 1 is used to display the representation of the activity trajectory 220 as a 3D animation curve. The activity trajectory 220 is displayed along the animation timeline 210 based on the frame number of the animation timeline 210 and the frame number associated with the activity trajectory 220. Each activity trajectory 220 shows the 3D position (xyz coordinates) of the corresponding active joint along the animation timeline 210.

[0040] The character's key poses 230 are overlaid / overlaid on the animation timeline 210 and displayed along the animation timeline 210 based on the frame number of the animation timeline 210 and the frame number associated with the key pose 230. Specifically, the 3D model of the character's key poses 230 is displayed along the animation timeline 210. Note that the curve representations of the activity trajectories 220 are simultaneously / concurrently overlaid on the key poses 230, where each trajectory 220 is displayed as a 3D curve passing through a specific joint of the character across all key poses 230. For example, the representation of the first activity trajectory 220a is displayed as a first 3D curve passing through the character's right wrist joint across all key poses 230, and the representation of the second activity trajectory 220b is displayed as a second 3D curve passing through the character's left ankle joint across all key frames 230.

[0041] Each key pose 230 is displayed along the animation timeline 210 at the frame number corresponding to the frame in the character animation 150 that includes the key pose 230. The frame number corresponding to each key pose 230 is also highlighted on the animation timeline 210 (e.g., shown as the frame number within a selectable circular button). In some embodiments, only the frame number corresponding to the key pose 230 is displayed on the animation timeline 210, while the frame numbers corresponding to non-essential poses (non-key poses) are not displayed on the animation timeline 210. As shown, the displayed key poses 230 include the current key pose 230a, a pair of adjacent key poses (key poses 230b and 230c) that are adjacent to the current key pose 230 on both sides of the current key pose 230 on the animation timeline 210, and a pair of next-adjacent key poses (key poses 230d and 230e) that are next to the current key pose 230 on both sides of the current key pose 230 on the animation timeline 210. Specifically, the pair of adjacent key poses includes a key pose 230b that is adjacent to the current key pose 230a on the first side of the current key pose 230a along the animation timeline 210, and another key pose 230c that is adjacent to the current key pose 230a on the second side of the current key pose 230a.

[0042] The current key pose 230a is the key pose currently selected by the user and displayed towards the center of the 3D timeline UI 130. In some embodiments, the current key pose 230a is displayed with visual characteristics / appearance different from other non-current key poses 230. For example, the current key pose 230a may be displayed with a highlighted appearance, and other non-current key poses 230 may be displayed with a non-highlighted appearance. In some embodiments, other non-current key poses 230 are displayed with a specific visual appearance based on their frame distance from the current key pose 230a. For example, the current key pose 230a may be displayed with a first appearance, the pair of adjacent key poses 230 (such as key poses 230b and 230c) adjacent to the current key pose 230 on both sides may be displayed with a second appearance, and the pair of next-next-next key poses 230 (such as key poses 230d and 230e) adjacent to the current key pose 230a on both sides may be displayed with a third appearance, and so on, whereby the first appearance, the second appearance, and the third appearance are each different visual appearances. For example, the current key pose 230a can be displayed in a dark shade (first appearance), the pair of adjacent key poses 230 can be displayed in a light shade (second appearance), and the pair of next-next-next key poses 230 can be displayed in outline form (third appearance), such as... Figure 2 As shown.

[0043] Animation editing application 120 can use the following information: Figure 7 The key pose extraction technique described in Equations 2-3 automatically extracts / identifies key poses 230 from character animation 150 based on activity trajectory 220. Then, metadata describing the identified key poses 230 stored in the corresponding frame is used to render and display each identified key pose 230 in the 3D timeline UI 130. Specifically, metadata describing the 3D model stored in the corresponding frame is used to render and display the 3D model of each identified key pose 230 in the 3D timeline UI 130. Each key pose 230 can be identified via the frame number of the frame including the key pose 230, thereby a list of frame numbers identifying the key poses 230 can be stored in a key pose set 162. In some embodiments, the key pose set 162 may also store metadata required for rendering and displaying each key pose 230.

[0044] Active joints can be selected by the animation editing application 120 by default or by the user via joint graph 250, which in turn determines which tracks 220 are active and will be displayed in the 3D timeline UI 130. The active tracks 220 also determine which poses are identified as key poses 230 in the character animation 150 and displayed in the 3D timeline UI 130. Joint graph 250 allows the user to focus on specific joint movements by activating only the joints and tracks of interest and then viewing the key poses generated from the active joints and tracks. In this regard, the user can select a new set of active joints via joint graph 250, which in turn causes the animation editing application 120 to dynamically determine a new set of active tracks 220 corresponding to the new set of active joints, and based on the new set of active tracks 220, determine a new set of key poses 230 in the character animation 150, thereby dynamically displaying the new set of active tracks 220 and the new set of key poses 230 in the 3D timeline UI 130.

[0045] The playback control button 260 includes selectable buttons for guiding the frames of character animation 150 along the animation timeline 210. The playback control button 260 includes a play / pause button, a forward button, and a rewind button. Selecting the play button causes character animation 150 to be played back. In some embodiments, key poses 230 are not displayed during playback to avoid visual clutter, and are thus displayed when the animation is paused. In some embodiments, the activity track 220 is displayed both during playback and when the animation is paused. The forward and rewind buttons can be selected to guide the viewer to the next or previous frame of character animation 150 along the animation timeline 210, respectively. Furthermore, each key pose 230 has a frame number displayed within a selectable circular button along the animation timeline 210, wherein selecting the circular button guides the viewer to the selected key pose 230 (thus making the selected key pose 230 the current key pose 230).

[0046] The view control buttons 270 include a rotate button, a zoom-in button, and a zoom-out button. The rotate button allows the user to rotate the displayed key pose 230 in place, providing a view of the movement in the character animation 150 from various angles. The zoom-in and zoom-out buttons allow the user to adjust the number of frames visible in the 3D timeline UI 130. The zoom-in button reduces the number of frames visible in the 3D timeline UI 130 to allow the user to focus on a smaller number of frames. The zoom-out button increases the number of frames visible in the 3D timeline UI 130 to allow the user to view a larger range of frames.

[0047] exist Figure 2 In the example, animation timeline 210 includes a time axis (shown horizontally along the x-axis) with frame numbers as the time unit, such that differences in the horizontal position between key poses 230 indicate the number of unnecessary poses between key poses 230. Note that although the horizontal time axis at the bottom of animation timeline 210 serves as a time reference, the top of animation timeline 210 specifies the 3D space coordinate region in which the 3D model displaying key poses 230 is located. In other embodiments, animation timeline 210 may be configured to be drawn and displayed along the y-axis or z-axis instead of the x-axis. Figure 2 In the example, the animation timeline 210 has a straight line shape suitable for short to medium-length character animations 150. However, the straight line shape of the animation timeline 210 may not be ideal for relatively longer character animations 150, as the user may find it difficult to reach key poses that are far from the current key pose. In other embodiments, the animation timeline 210 may have different types of shapes, such as curved shapes, circular shapes, or spiral shapes for visualizing character animations 150. These different types of shapes may be more ideal for relatively longer character animations 150 and make it easier for the user to reach key poses that are far from the current key pose.

[0048] After key pose 230 is displayed in the 3D timeline UI 130, the user can navigate through the key poses 230 of the character animation 150 along the animation timeline 210 and select a specific key pose 230 for editing. The currently selected key pose 230 is referred to as the current key pose 230a. Upon receiving a user selection of the current key pose 230a, in response, the 3D timeline UI 130 displays selectable widgets 280 (such as 280a and 280b) on each active joint of the current key pose 230a. Figure 2In the example, the 3D timeline UI 130 displays a first widget 280a on the right wrist joint of the current key pose 230a and a second widget 280b on the left ankle joint of the current key pose 230a. The selectable widgets 280 allow the user to manipulate the 3D model of the key pose 230 by, for example, directly manipulating the joints of the 3D model via the IE controller 176. Advantageously, the IE controller 176 utilizes 6 degrees of freedom (6DOF) input and hand tracking to make it easier for the user to manipulate the 3D model.

[0049] Figure 3 Based on various embodiments and Figure 2 The screenshot shows the selected joint associated with the current key pose included in the 3D timeline UI. For example, the user selection of the left ankle joint in the current key pose 230a is received via IE controller 176. The user can then modify the left ankle joint in the current key pose 230a using a second widget 280b on the left ankle joint. The user can select a selectable widget 280 on a specific joint to modify the 3D position and / or rotation of that joint. In some embodiments, the selectable widget 280 includes a spherical widget comprising a center point and three rotation axes (x, y, z axes), each displayed in a different color (such as blue, green, and red). The user can move the selectable widget 280 to a new 3D position to modify the current 3D position of the corresponding joint. The user can rotate the selectable widget 280 around one or more axes to a new rotation to modify the current rotation of the corresponding joint.

[0050] Figure 4 According to various embodiments Figure 3 A screenshot showing the modifications made to the selected joint. As shown, the user has already modified the joint. Figure 3 The initial 3D position of the left ankle joint shown has changed to Figure 4 The updated 3D position of the left ankle joint shown is modified via the second widget 280b. In response to the user modification / change performed on the selected left ankle joint, the animation editing application 120 performs a series of operations, including: propagating the modification to other joints in the current key pose 230a; propagating the modification to the current key pose 230a to other poses near the current key pose 230a; saving the modified character animation 150; updating all trajectory datasets 160 of the modified character animation 150; updating the representation of the activity trajectory 220 displayed in the 3D timeline UI 130 based on the updated trajectory dataset 160; re-extracting the key pose set 162 from the updated character animation 150 based on the updated activity trajectory 220; and updating the rendered key pose 230 displayed in the 3D timeline UI 130.

[0051] As discussed above, a character's 3D model can include a set of motion joints forming a kinematic chain, such that a modification to the position and / or rotation of one joint can cause that modification to be propagated to one or more other joints in the set of motion joints. Therefore, any modification to a particular joint in the current key pose 230a may cause that modification to be propagated to other joints in the current key pose 230a, the extent to which the modification is propagated to other joints depending on the extent of the modification to the particular joint and the kinematic characteristics of the character's kinematic chain (skeleton rigging). Various solvers can be used to calculate the position and / or rotation modifications propagated to other joints, such as inverse kinematic solvers or other types of solvers.

[0052] Modifications to the selected joint and one or more other joints in the current key pose 230a are then propagated to other poses near the current key pose 230a in the character animation 150. Each key pose is included in a keyframe, and the current key pose is included in the current keyframe of the character animation 150. In some embodiments, modifications to joints made in the current key pose in the current keyframe are propagated to all non-essential poses included in frames adjacent to the current frame, as defined by a pair of adjacent key poses 230; these modified non-essential poses are referred to as “neighboring” poses. In these embodiments, neighboring poses / frames modified based on modifications to the current key pose / keyframe include all poses / frames in the character animation 150 between the current key pose / keyframe and the adjacent previous key pose / keyframe, as shown along the animation timeline 210, and all poses / frames between the current key pose / keyframe and the adjacent next key pose / keyframe.

[0053] exist Figure 4 In the example, the current key pose 230a has been modified, and these modifications are propagated to a first set of poses 410 between the current key pose 230a and the adjacent previous key pose 230b, and a second set of poses 420 between the current key pose 230a and the adjacent next key pose 230c, as shown along the animation timeline 210. In this way, the propagation effect of the modification to the current key pose 230a can be controlled within a limited range of neighboring poses / frames. In other embodiments, the pair of key poses 230s defining the neighboring poses / frames can be user-selected to expand the range of poses / frames modified based on the modification to the current key pose / keyframe.

[0054] In some embodiments, weighted linear interpolation and weighted spherical interpolation are used to propagate position and / or rotation modifications to the current key pose / keyframe to neighboring poses / frames. Weighted linear interpolation is used to propagate position modifications to joints in the current key pose to joints in neighboring poses, while weighted spherical interpolation is used to propagate rotation modifications to joints in the current key pose to joints in neighboring poses. A Gaussian distribution can be used to determine the weights, where the peak is centered on the current key pose / keyframe where the modification occurred. Therefore, the peak / highest weight is at the current key pose / keyframe, and the weights of neighboring poses / frames decrease as the neighboring poses / frames become farther (in frames / time) from the current key pose / keyframe. Thus, the maximum amount of propagated modification occurs at the nearest (in frames / time) neighboring pose / frame to the current key pose / keyframe, and the minimum amount of propagated modification occurs at the farthest (in frames / time) neighboring pose / frame from the current key pose / keyframe. Figure 4 In the example, the maximum amount of the propagated modification occurs at the nearest neighboring poses to the current critical pose 230a on both sides, and the minimum amount of the propagated modification occurs at the nearest neighboring pose / frame to the adjacent previous critical pose 230b and the adjacent next critical pose 230c.

[0055] Since one or more poses / frames in character animation 150 have now been modified, a modified character animation 150 has been generated, and the animation editing application 120 then stores the modified character animation 150 in memory 104. When the trajectory of one or more joints in character animation 150 has been modified, the animation editing application 120 also recalculates and updates all trajectory datasets 160 for all joints in the modified character animation 150. Each trajectory dataset for a joint includes a time-ordered set of positions for all poses / frames of the joint across the character animation, as described above regarding... Figure 1 The subject of discussion.

[0056] The animation editing application 120 can then retrieve the updated activity trajectory dataset 220 from the updated trajectory dataset 160, and display the representation of the updated activity trajectory dataset 220 as a 3D curve in the 3D timeline UI 130. Figure 4 In the example, the active joints are still the right wrist joint and the left ankle joint, so the animation editing application 120 retrieves the updated first active trajectory 220a of the right wrist joint and the updated second active trajectory 220b of the left ankle joint, and re-displays the updated representations of the updated first active trajectory 220a and the updated second active trajectory 220b as 3D curves.

[0057] As discussed above, the key pose set 162 is identified based on one or more activity trajectories of one or more active joints. Since the activity trajectory 220 has been updated in the trajectory dataset 160, the animation editing application 120 also re-extracts the key pose set 162 based on the updated activity trajectory 220. The animation editing application 120 also re-renders and re-displays the newly extracted key poses 230 in the 3D timeline UI 130.

[0058] The above process can be performed by the animation editing application 120 for each user modification received by the joints of the current key pose 230a. Advantageously, the user can immediately see the effect of the joint modification in the spatial domain through the displayed changes to the 3D model of the current key pose 230a, and immediately see the effect of the joint modification in the temporal domain through the displayed changes to the representative 3D curve of the activity trajectory 220 of the active joints of the key pose 230 along the animation timeline 210. In this way, the user can easily modify the key poses in the character animation 150 and view the changes in the character animation 150 in an intuitive and efficient manner via the 3D timeline UI 130.

[0059] Figures 5A to 5B Flowcharts illustrating method steps for editing character animation via a 3D timeline UI, according to various embodiments, are provided. (Although combined...) Figures 1 to 4 The system describes these method steps, but those skilled in the art will understand that these method steps can be performed by any system in any order. In some embodiments, method 500 can be performed by an animation editing application 120 that includes a 3D timeline UI engine 110.

[0060] When animation editing application 120 receives (at step 502) a source file including character animation 150, which includes multiple motion capture data frames of multiple poses of the character, each frame having an associated frame number indicating its temporal order within character animation 150. Each frame captures a specific pose of the character. Specifically, each frame includes metadata specifying the pose of a 3D model of the character, which includes multiple joints organized in a tree hierarchy. For example, character animation 150 may be downloaded from animation database 180 and stored in memory 104.

[0061] At step 504, the animation editing application 120 extracts multiple trajectory datasets 160 for multiple joints from the character animation 150, each trajectory dataset corresponding to a specific joint of the character. Each trajectory dataset for a joint includes multiple ordered 3D positions of the joint across multiple poses / frames of the character animation 150, the multiple 3D positions being ordered chronologically based on the frame numbers corresponding to the multiple 3D positions. The multiple extracted trajectory datasets 160 can be stored in memory 104.

[0062] At step 506, the animation editing application 120 generates and displays a 3D timeline UI 130 including an animation timeline 210, articulation diagrams 250, playback control buttons 260, and view control buttons 270, as shown below. Figure 2 As shown in the example. The animation timeline 210 includes a time axis with frame numbers as the time unit. The joint diagram 250 displays a visualization of all the character's joints and the relationships between them. The joint diagram 250 also indicates which joints are "active" joints. The set of active joints can be selected by default by the animation editing application 120 or selected by the user via the joint diagram. The playback control buttons 260 include selectable buttons for guiding the character's animation 150 frames along the animation timeline 210. The view control buttons 270 include a rotate button, a zoom-in button, and a zoom-out button.

[0063] At step 508, the animation editing application 120 loads a collection of activity trajectory datasets 220 corresponding to a collection of activity joints from a plurality of trajectory datasets 160. In some embodiments, the collection of activity joints includes a plurality of activity joints, and the collection of activity trajectory datasets 220 includes a plurality of activity trajectory datasets 220. In some embodiments, the collection of activity joints includes some, but not all, of a character’s plurality of joints, and the collection of activity trajectory datasets 220 includes some, but not all, of a plurality of trajectory datasets 160.

[0064] At step 510, the animation editing application 120 uses the following about Figure 6 The trajectory display technique described in Equation 1 is used to display the representation of each activity trajectory dataset 220 as a 3D curve in the 3D timeline UI 130. At step 512, the animation editing application 120 uses the following... Figure 7The key pose extraction technique described in Equations 2 and 3 automatically extracts / identifies a set 162 of key poses from the character animation 150 based on the set of activity trajectory datasets 220. The identified key poses 230 are also displayed in the 3D timeline UI 130. In some embodiments, each key pose 230 is overlaid on the animation timeline 210 and displayed at a specific point on the animation timeline 210 based on the frame number associated with the key pose 230. In some embodiments, a curve representation of the activity trajectory dataset 220 is overlaid on the displayed key poses 230 to help the user visualize the motion of the character in the poses between the displayed key poses 230. Additionally, the animation timeline 210 includes selectable frame numbers adjacent to / below each key pose 230, which correspond to the frame numbers of the frames in the character animation 150 that include the key pose 230.

[0065] At step 514, the animation editing application 120 receives a user selection for a new / different set of active joints via the joint graph 250. At step 516, in response to receiving the selection for a new / different set of active joints, the animation editing application 120 loads a set of new / different activity trajectory datasets 220 corresponding to the new / different set of active joints from multiple trajectory datasets 160, displays a representation of each new / different activity trajectory dataset 220 in the 3D timeline UI 130, extracts a new / different set of key poses 162 from the character animation 150 based on the set of new / different activity trajectory datasets 220, and displays each key pose 230 in the 3D timeline UI 130. In this way, the animation editing application 120 can dynamically display the activity trajectory datasets 220 and key poses 230 in the 3D timeline UI 130 in response to a user selection for a new / different active joint via the joint graph 250. Note that whenever the user selects a new / different active joint via joint diagram 250, the animation editing application 120 can dynamically repeat steps 514 to 516.

[0066] At step 518, the animation editing application 120 receives a user selection of a specific key pose 230 displayed along the animation timeline 210, the selected key pose 230 being referred to as the current key pose 230a. For example, the user can navigate to and select the current key pose 230a using the playback control button 260, or the user can select the frame number displayed below the current key pose 230a on the animation timeline 210. In response, at step 520, the animation editing application 120 displays selectable widgets 280 on each active joint of the current key pose 230a in the 3D timeline UI 130.

[0067] At step 522, the animation editing application 120 receives user modifications to the active joints of the current key pose 230a via a corresponding selectable widget 280. The received modifications may include changes to the position or rotation of the active joints. In response to receiving user modifications at step 522, the animation editing application 120 performs the following steps 524 to 530.

[0068] At step 524, the animation editing application 120 modifies the character animation 150 to generate a modified character animation 150 based on the received modifications by propagating the received modifications to the active joints to one or more other joints in the current key pose 230a and the modifications to the active joints and one or more other joints in the current key pose 230a to the adjacent poses of the current key pose 230a. In some embodiments, the adjacent poses of the current key pose 230a include poses near the current key pose 230a defined by a pair of adjacent key poses 230. The animation editing application 120 then saves the modified character animation 150, which includes the modifications to the active joints and one or more other joints in the current key pose 230a, as well as the modifications to the adjacent poses of the current key pose 230a.

[0069] At step 526, the animation editing application 120 extracts updated multiple trajectory datasets 160 for multiple joints from the modified character animation 150. At step 528, the animation editing application 120 loads a set of updated active trajectory datasets 220 corresponding to the set of active joints from the updated multiple trajectory datasets 160. At step 530, the animation editing application 120 displays an updated representation of each updated active trajectory dataset 220 in the 3D timeline UI 130. At step 532, the animation editing application 120 automatically extracts / identifies an updated set of key poses 162 from the modified character animation 150 based on the set of updated active trajectory datasets 220, and displays the identified key poses 230 in the 3D timeline UI 130. Note that the animation editing application 120 may repeat steps 522 to 530 whenever the user modifies a joint in the current key pose 230a. Method 500 then ends.

[0070] Figure 6 A flowchart illustrating method steps for displaying a visual representation of an activity trajectory dataset within a 3D timeline UI, according to various embodiments, is provided. Although combined with... Figures 1 to 4The system describes these method steps, but those skilled in the art will understand that these method steps can be performed by any system in any order. In some embodiments, method 600 can be performed by animation editing application 120 including 3D timeline UI engine 110. In some embodiments, method 600 can be performed by animation editing application 120 whenever the techniques disclosed herein display a visual representation of activity trajectory dataset 220 along animation timeline 210 in 3D timeline UI 130.

[0071] Method 600 begins when animation editing application 120 loads (at step 610) a collection of active trajectory datasets 220 corresponding to the active joint set from multiple trajectory datasets 160 stored in memory 104. Each active trajectory dataset 220 includes a set of joint positions (3D spatial coordinates) for all poses of the character across all frames of the character animation 150 corresponding to a joint, whereby the set of joint positions is ordered chronologically based on the frame number corresponding to each joint position. In some embodiments, the active joint set includes multiple active joints, and the collection of active trajectory datasets 220 includes multiple active trajectory datasets 220.

[0072] At step 620, the animation editing application 120 generates a 3D curve representation of the current activity trajectory dataset 220 included in the set of activity trajectory datasets 220 to be displayed along the animation timeline 210 in the 3D timeline UI 130 by applying Equation 1 to the current activity trajectory dataset 220. in: t =Time (represented by frame number); t c =Current time / frame; =Time t The location of the joint; Joint position = 3D spatial coordinates (xyz coordinates); = The corresponding position on the 3D curve representation; = unit vector; and =A constant indicating the scaling of the timeline.

[0073] Apply Equation 1 to the joint At each position across all frames (time), to convert each 3D position of the joint into a position on a 3D curve representation. To be displayed along the animation timeline 210 in the 3D timeline UI 130. Vector This is a unit vector aligned with the direction of timeline 120. In the above embodiment, this unit vector includes a unit vector along the x-axis, and timeline 120 is horizontally aligned along the x-axis, with time (frame number) incrementing to the right. In other embodiments, this unit vector may be a unit vector along the y- or z-axis, and animation timeline 210 may be aligned with the y- or z-axis instead of the x-axis. A constant indicates the unit vector. The amplitude of . The constant indicates the scaling of the animation timeline 210 and reflects the length / size of the animation timeline 210. The larger the value, the longer the animation timeline 210; and a value of 0 indicates that there is no animation timeline 210.

[0074] Using unit vectors The animation timeline 210 has a linear shape suitable for short to medium-length character animations 150 (such as in...). Figure 2 (In the middle). However, the straight-line shape of the animation timeline 210 may not be ideal for relatively longer character animations 150, as users may find it difficult to reach key poses far from the current key pose. In other embodiments, instead of a straight line, the animation timeline 210 is generated using variable vectors in space, such as curved, circular, or spiral shapes used to visualize character animations 150. These different types of non-linear shapes may be more ideal for relatively longer character animations 150 and make it easier for users to reach key poses far from the current key pose.

[0075] At step 630, the animation editing application 120 then displays the resulting 3D curve representation of the current activity trajectory dataset 220 along the current active joint of the animation timeline 210 in the 3D timeline UI 130. At step 640, the animation editing application 120 determines whether there are any more activity trajectory datasets 220 to process in the set of activity trajectory datasets 220. If yes (at step 640 – Yes), the animation editing application 120 continues processing the next current activity trajectory dataset 220 included in the set of activity trajectory datasets 220 at step 620. If not (at step 640 – No), method 600 ends.

[0076] Figure 7 A flowchart illustrating the steps of a method for automatically extracting a set of key poses from character animation, according to various embodiments, is provided. Although combined with... Figures 1 to 4The system describes these method steps, but those skilled in the art will understand that these method steps can be performed by any system in any order. In some embodiments, method 700 can be performed by an animation editing application 120 including a 3D timeline UI engine 110. In some embodiments, method 700 can be performed by the animation editing application 120 whenever the techniques disclosed herein extract a set of key poses 162 from character animation 150. Generally, the animation editing application 120 can use key pose extraction techniques to automatically extract the set of key poses 162 from character animation 150 based on one or more activity trajectories corresponding to one or more active joints. Specifically, each activity trajectory includes a set of local extremum locations / points, and the set of key poses 162 of character animation 150 can be identified based on the set of local extremum locations / points associated with the set of activity trajectories. The set of local extremum locations is a set of local target locations that the animation editing application 120 seeks to identify and utilize to extract the set of key poses 162 from character animation 150.

[0077] Method 700 begins when animation editing application 120 loads (at step 710) a collection of active trajectory datasets 220 corresponding to the active joint set from multiple trajectory datasets 160 stored in memory 104. Each active trajectory dataset 220 includes a set of joint positions (3D spatial coordinates) for all poses of the character across all frames of the character animation 150 corresponding to a joint, whereby the set of joint positions is ordered chronologically based on the frame number corresponding to each joint position. In some embodiments, the active joint set includes multiple active joints, and the collection of active trajectory datasets 220 includes multiple active trajectory datasets 220.

[0078] At step 720, the animation editing application 120 extracts / identifies individual key poses in the character animation 150 by applying Equations 2 and 3 to the set of activity trajectory datasets 220 during a single iteration of the key pose extraction technique. in: T k ( t ) = Joint k ( k = 0, ... N - 1) The trajectory that changes over time (indicated by frame number); =Trajectory T k ( t The smooth form of ) and w k =Weight.

[0079] In equations 2 and 3, the movable joint k The activity trajectory dataset is represented as T k ( t Trajectory T k ( t The smooth version / form of ) is represented as This smooth version / form is achieved by using the trajectory T k ( t The animation was generated using a Gaussian smoothing function, as shown in Equation 3. For each active joint / track, the animation editor applied 120 by calculating... T k ( t )and To calculate the Euclidean distance between them T k ( t )and Between over time ( t The difference in change.

[0080] As shown in Equations 2 and 3, in each iteration, the animation editor application 120 calculates the combined difference of all activity trajectories / joints. d ( t This is the normalized weighted sum of the Euclidean differences of all movement trajectories for all moving joints. The Euclidean differences of the movement trajectories include the movement trajectories... T k ( t ) and the smooth form of the activity trajectory The difference between them. Weight ( w k ) is normalized because when all weight values ​​( w k When these are added together, the sum equals 1 (as indicated on the right side of Equation 2). Therefore, the combined difference... d ( t This includes the normalized weighted sum of the Euclidean differences of all activity trajectories / joints.

[0081] As shown on the right side of Equation 2, each weight ( w k () is a movable joint k / trajectory kThe ratio of the motion / movement amplitude to the sum of the motion / movement amplitudes of all active joints / trajectories. Therefore, when multiple joints / trajectories are active, the active joints / trajectories with larger motion / movement amplitudes within character animation 150 (as indicated by the joint positions ordered by time in the active trajectory) will have a greater weight / influence on the selection of key poses than active joints / trajectories with smaller motion / movement amplitudes within character animation 150 (as indicated by the joint positions ordered by time in the active trajectory). The weight values ​​calculated for active joints / trajectories ( w k The value reflects / represents the weight / influence of each active joint / trajectory pair on the selection of key poses.

[0082] function d ( t The time function is used, and at each iteration, the animation editing application 120 will adjust the time... t m Find the largest d ( t m As shown in Equations 2 and 3, in each iteration, in order to find the local extremum location / point of the activity trajectory, the animation editing application 120 selects the position with the largest difference among all poses / frames of the character animation 150. d ( t m (time) t m At each iteration, the chosen time... t m Added to the list of selected locations / points, at the selected time t m Specifies the time as the corresponding frame number. t m The frame numbers include the frame numbers of the identified key poses of character animation 150. Therefore, the list of selected positions / points may include a list of frame numbers identifying the set of key poses 162.

[0083] At step 730, after the frame number of the identified key pose is added to the key pose set 162 in the current iteration, the animation editing application 120 updates the smooth form for the next iteration of the key pose extraction technique according to Equation 3. Generally speaking, smooth form Through calculation T k ( t )and The weighted average of the values ​​is used to update the smoothed form for the next iteration. To ensure the same frame number in the current iterationt m It will not be selected as a key pose in the next iteration. Specifically, the weights α ( t ) is time ( t The function of the weights for a small constant window δ Inner separation in time t m The pose / frame closer to the selected key pose / keyframe has a greater weight. The weights in Equation 3... α ( t ) Use to t = t m The Gaussian function centered at the center is used for calculation, such that when t = t m At that time, weight α ( t The value equals 1. In the next iteration, when αt = t m At that time, the updated smooth form Will equal to T k ( t m Therefore, in the next iteration, in Equation 2, T k ( t ) - exist t = t m The value will be equal to 0, thus ensuring the frame number. t m It will not be selected as a key pose in the next iteration.

[0084] At step 740, the animation editing application 120 determines whether the stopping condition of the key pose extraction technique has been met. Because the key pose extraction technique is an iterative process, a stopping / terminating condition is implemented. In some embodiments, the stopping condition is met if a predetermined number of key poses are identified in the key pose set 162. Note that each iteration of the key pose extraction technique adds key poses to the key pose set 162, and the keyframe extraction technique is repeated until a sufficient number of key poses are identified. In some embodiments, if the maximum difference... d ( t m If the difference falls below a threshold, the stopping condition is met. This condition defines the situation when there is no key pose to select, such as when... d ( tThe maximum difference is an extreme case of a constant function of 0, such that no maximum location / point is chosen as the critical pose. Therefore, the maximum difference... d ( t m The lower bound of ) can be defined as the threshold difference.

[0085] If the stopping condition is not met (at step 740 – No), the animation editing application 120 continues to the next iteration at step 720. If the stopping condition is met (at step 740 – Yes), at step 750, the animation editing application 120 stores a list of frame numbers identifying key poses as a key pose set 162 in memory 104. In some embodiments, the animation editing application 120 may also retrieve metadata required for rendering and displaying each key pose from each keyframe including the key poses and store it in the key pose set 162. At step 760, the animation editing application 120 displays each key pose in the key pose set 162 in the 3D timeline UI 130. Method 700 then ends.

[0086] In this way, method 700 can automatically extract a set of key poses 162 from character animation 150 based on multiple trajectories / joints. The resulting set of key poses 162 represents the motion of character animation 150 more accurately than a set of key poses 162 extracted based on a single trajectory / joint, because multiple trajectories of multiple joints represent the motion of character animation 150 more accurately than a single trajectory of a single joint. Advantageously, editing the more accurate set of key poses 162 representing character animation 150 can significantly reduce the total time and effort required by the user editing character animation 150.

[0087] Retiming animation clips In some embodiments, the animation editing application 120 implements a retiming function that can be applied to selected segments of character animation 150 within the 3D timeline UI 130 to slow down or speed up the character's movement in the selected segment. As discussed above, the 3D timeline UI 130 displays a set of key poses superimposed on the animation timeline 210, with each key pose superimposed at the frame number corresponding to a specific key pose on the animation timeline 210. The user can select a segment of character animation by selecting a pair of key poses displayed in the 3D timeline UI, the selected pair defining the selected segment. The user can then expand (increase) or contract (decrease) the frame distance between the selected pair of key poses along the animation timeline 210 to slow down or speed up the character's movement in the selected segment, respectively. In response to the expansion or contraction of the selected segment, the animation editing application automatically performs one or more retiming operations on the selected segment based on the degree of expansion or contraction of the selected segment along the timeline.

[0088] Figure 8 The retiming function elements are shown according to various embodiments. Figure 1 A screenshot of the 3D timeline UI. The 3D timeline UI 130 is generated based on a humanoid character from character animation 150, which depicts the humanoid character's movement while jumping. As shown, the 3D timeline UI 130 displays an animation timeline 210, one or more curve representations of the movement trajectories 820 (such as 820a and 820b) of one or more movable joints of the character, one or more displayed key poses 830 (such as 830a, 830b, 830c, etc.) extracted from character animation 150, and an optional retiming button 840. In some embodiments, the animation timeline 210, one or more curve representations of the movement trajectories 820, and one or more key poses 830 are displayed concurrently in the 3D timeline UI 130 at overlapping times.

[0089] The animation timeline 210 includes a timeline using frame numbers as the time unit. In some embodiments, the animation timeline 210 displays the frame number of each key pose 230 along the animation timeline 210 (such as the frame number within a selectable button), and does not display the frame numbers of any non-key poses along the animation timeline 210. Key poses 830 of the character are overlaid on the animation timeline 210 and displayed along the animation timeline 210 based on the frame numbers of the animation timeline 210 and the frame numbers associated with the key poses 830. Specifically, each key pose 830 is displayed along the animation timeline 210 at the frame number corresponding to the frame in the character animation 150 that includes the key pose 830. As shown in the figure, the displayed key poses 830 include the current key pose 830a, a pair of adjacent key poses (key poses 830b and 830c) that are adjacent to the current key pose 830 on both sides of the current key pose 830 on the animation timeline 210, and a pair of secondary adjacent key poses (key poses 830d and 830e) that are secondary adjacent to the current key pose 830 on both sides of the current key pose 830 on the animation timeline 210.

[0090] Figure 9 The following is an illustration of the activated retiming function element according to various embodiments. Figure 8 A screenshot of the 3D timeline UI. User selection of the selectable retime button 840 activates the retime function. In response to user selection of the selectable retime button 840, the animation editing application 120 enables the user (e.g., using an IE controller 176) to select a pair of key poses 830 via the 3D timeline UI 130. Figure 9In the example, the user has selected the pair of adjacent key poses (key pose 830b and key pose 830c) on both sides of the current key pose 830 on the animation timeline 210.

[0091] The selected pair of key poses 830 includes a start key pose 830b and an end key pose 830c defining the selected segment. The start key pose 830b is included in a “start frame” within the character animation 150 with an initial start frame number (such as 33), and the end key pose 830c is included in an “end frame” within the character animation 150 with an initial end frame number (such as 56), which is higher than the initial start frame number. Note that in this context, “start frame” and “end frame” do not refer to the start and end frames of the character animation 150, but rather to the start and end frames of the selected segment. The user can then expand (enlarge) or contract (shrink) the selected initial segment along the animation timeline 210 to slow down or speed up the character's movement within the selected initial segment, respectively. Users can do this by expanding (increasing) or contracting (decreasing) the frame distance between the selected pair of key poses along the animation timeline 210 to specify an updated segment with an updated start frame number, an updated end frame number, and an updated start frame number. This updated segment is the target segment that the user wishes to achieve by retiming the selected segment. The animation editing application 120 then automatically performs one or more retiming operations on the selected segment to achieve the updated segment.

[0092] Figure 10 This is an illustration of an expanded fragment according to various other embodiments. Figure 9 A screenshot of the 3D timeline UI. As shown, the selected initial segment has been expanded by the user to specify an expanded, updated segment. The user can do this by expanding (increasing) the frame distance between the selected pair of key poses 830 along the animation timeline 210 by moving the selected pair of key poses 830 outward from their initial corresponding positions on the animation timeline 210 (i.e., moving each of the selected pair of key poses 830 away from the center pose located in the middle between the selected pair of key poses 830 along the animation timeline 210). Expanding the selected initial segment along the animation timeline 210 specifies an expanded, updated segment defined by a start key pose 830b with an updated start frame number (such as 30) and an end key pose 830c with an updated end frame number (such as 59), the updated end frame number being higher than the updated start frame number.

[0093] The selected initial segment has an initial frame distance of 1010, which is equal to the difference between the initial end frame number and the initial start frame number (i.e., the initial end frame number minus the initial start frame number). The expanded, updated segment has an updated frame distance of 1020, which is equal to the difference between the updated end frame number and the updated start frame number (i.e., the updated end frame number minus the updated start frame number). Figure 10 In the example, the selected initial segment has an initial start frame number of 33, an initial end frame number of 56, and an initial frame distance of 1010 equal to 23 (56 minus 33). Figure 10 In the example, the expanded updated fragment has an updated start frame number equal to 30, an updated end frame number equal to 59, and an updated frame distance of 1020 equal to 29 (59 minus 30). In some embodiments, the expanded updated fragment includes and contains a selected initial fragment, whereby the selected initial fragment is a sub-part of the expanded updated fragment.

[0094] Figure 11 The contracted segment is shown according to various other embodiments. Figure 9 A screenshot of the 3D timeline UI. As shown, the selected initial segment has been shrunk by the user to specify a shrunk, updated segment. The user can do this by shrinking (reducing) the frame distance between the selected pair of key poses 830 along the animation timeline 210 by moving the selected pair of key poses 830 inward from their initial corresponding positions on the animation timeline 210 (i.e., moving each of the selected pair of key poses 830 along the animation timeline 210 toward a center pose located in the middle between the selected pair of key poses 830). Shrinking the selected initial segment along the animation timeline 210 specifies a shrunk, updated segment defined by a start key pose 830b with an updated start frame number (such as 37) and an end key pose 830c with an updated end frame number (such as 52), the updated end frame number being higher than the updated start frame number.

[0095] The selected initial segment has an initial frame distance of 1010, which is equal to the difference between the initial end frame number and the initial start frame number. The shrunken, updated segment has an updated frame distance of 1120, which is equal to the difference between the updated end frame number and the updated start frame number. Figure 11 In the example, the selected initial segment has an initial start frame number of 33, an initial end frame number of 56, and an initial frame distance of 1010 equal to 23 (56 minus 33). Figure 11In the example, the shrunken updated segment has an updated start frame number equal to 37, an updated end frame number equal to 52, and an updated frame distance 1120 equal to 15 (52 minus 37). In some embodiments, the selected initial segment includes and contains the shrunken updated segment, whereby the shrunken updated segment is a sub-part of the selected initial segment.

[0096] In response to receiving an expanded or shrunken updated segment, animation editing application 120 automatically performs one or more retiming operations on the selected initial segment to achieve the expanded or shrunken updated segment. In some embodiments, in response to receiving an updated segment, animation editing application 120 uses the following... Figure 12 The retiming technique described in Equation 4 automatically performs one or more retiming operations on a selected initial segment based on the degree of expansion or contraction along the animation timeline 210. Specifically, the animation editing application 120 automatically generates a retiming ratio based on the amount of expansion or contraction, and then generates a velocity curve function based on the retiming ratio. The retiming ratio is equal to the ratio between the initial frame distance and the updated frame distance (initial frame distance divided by the updated frame distance), whereby a retiming ratio less than 1 indicates expansion (slowing down), and a retiming ratio greater than 1 indicates contraction (speeding up). The velocity curve function is then automatically applied to the poses / frames of the selected segment to provide smooth and natural movement of the character in the selected segment. Specifically, the velocity curve function is automatically applied to all poses / frames of the character animation 150 between the selected pair of key poses 830 to generate an updated segment that includes the retimed segment.

[0097] exist Figure 10 In the example, the selected initial segment is expanded to generate an expanded, updated segment. Therefore, the retiming ratio (initial frame distance divided by the updated frame distance) is less than 1 (2³ divided by 2⁹). In this case, the animation editing application 120 automatically generates new poses / frames to add to the selected segment to achieve the expanded, updated segment, thereby slowing down the character's movement within the selected segment. Specifically, the animation editing application 120 will automatically generate and add multiple new poses / frames that increase the initial frame distance of the selected initial segment to reach / achieve the updated frame distance of the expanded, updated segment. Figure 10In the example, animation editing application 120 can generate and add six new poses / frames to increase the initial frame distance 23 of the selected initial segment, thereby achieving / achieving the updated frame distance 29 of the expanded updated segment. Additionally, a velocity curve function indicates where to add the new poses / frames within the selected segment. In some embodiments, the velocity curve function indicates that relatively fewer new poses / frames will be added near the selected pair of key poses 830, and relatively more new poses / frames will be added towards the middle of the pair of key poses 830 (i.e., towards the center pose in the middle between the selected pair of key poses 830).

[0098] exist Figure 11 In the example, the selected initial segment is shrunk to generate a shrunk, updated segment. Therefore, the retiming ratio (initial frame distance divided by updated frame distance) is greater than 1 (2³ divided by 15). In this case, the animation editing application 120 automatically removes poses / frames from the selected segment to achieve the shrunk, updated segment, thereby speeding up the character's movement within the selected segment. Specifically, the animation editing application 120 will automatically remove multiple poses / frames from the selected initial segment that reduce the initial frame distance of the selected initial segment to achieve / realize the updated frame distance of the shrunk, updated segment. Figure 11 In the example, animation editing application 120 can remove 8 poses / frames to reduce the initial frame distance 23 of the selected initial segment, thereby achieving / achieving the updated frame distance 15 of the shrunken updated segment. Additionally, a velocity curve function indicates where to remove poses / frames within the selected segment. In some embodiments, the velocity curve function indicates that relatively fewer poses / frames will be removed near the selected pair of key poses 830, and relatively more poses / frames will be removed towards the middle of the pair of key poses 830 (i.e., towards the center pose in the middle between the selected pair of key poses 830).

[0099] Figure 12 A flowchart illustrating the steps of a method for retiming selected segments of character animation according to various embodiments is provided. Although combined with... Figures 1 to 4 and Figures 8 to 11 The system describes these method steps, but those skilled in the art will understand that these method steps can be performed by any system in any order. In some embodiments, method 1200 can be performed by an animation editing application 120 including a 3D timeline UI engine 110 and a retiming engine 140. Note that the retiming method 1200 can be performed at any time during method 500 via the above-described... Figures 5A to 5B The discussion focuses on using a 3D timeline UI to edit character animations.

[0100] Method 1200 begins when the animation editing application 120 receives activation of the retiming function via the retiming button 840 of the 3D timeline UI 130 of the character animation 150 (at step 1210). At step 1220, the animation editing application 120 receives a selection of a pair of key poses 830 of the character animation 150 displayed on the animation timeline 210 of the 3D timeline UI 130. The selected pair of key poses 830 includes a start key pose 830b and an end key pose 830c, which define a selected initial segment of the character animation 150 for retiming operations. At step 1230, the animation editing application 120 determines information about the selected initial segment, including an initial start frame number, an initial end frame number, and an initial frame distance. At this point, the start key pose 830b is included in the start frame of the character animation 150 with the initial start frame number, and the end key pose 830c is included in the end frame of the character animation 150 with the initial end frame number. The selected initial segment has an initial frame distance of 1010, which is equal to the difference between the initial end frame number and the initial start frame number.

[0101] At step 1240, the animation editing application 120 receives from the user the selection of an updated segment by expanding (increasing) or contracting (decreasing) the selected initial segment along the animation timeline 210, for example via the IE controller 176, the frame distance between the selected pair of key poses. At step 1250, the animation editing application 120 determines information about the updated segment, including an updated start frame number, an updated end frame number, and an updated frame distance equal to the difference between the updated end frame number and the updated start frame number.

[0102] At step 1260, the animation editing application 120 calculates a retiming ratio, which includes the ratio between the initial frame distance and the updated frame distance (initial frame distance divided by the updated frame distance). A retiming ratio less than 1 indicates an expansion of the initial segment used to slow down the character's movement in character animation 150. A retiming ratio greater than 1 indicates a contraction of the initial segment used to speed up the character's movement in character animation 150. Note that a retiming ratio equal to 1 indicates no change in the initial segment.

[0103] In step 1270, the animation editing application 120 calculates the velocity curve function based on the retiming ratio. Speed (t) Note that when the selected segment does not expand or contract, the velocity curve is a function of a constant 1, making... Speed ​​(t)=1. The velocity curve function defines the speed pattern of the character's movement within an updated segment. Generally, if the character's movement speed is constant, the movement may appear unnatural for some activities, such as jumping. In some embodiments, the velocity curve function alters the speed of the movement within the updated segment to give the movement a smoother and more natural feel. For example, the velocity curve function can achieve a "fade-in, fade-out" effect, where the movement in the animation starts slowly, accelerates midway, and decelerates at the end. In some embodiments, the velocity curve function is an incremental curve calculated using a quadratic Bézier curve, as shown in Equation 4: in: t =Time; =Time( t The velocity at point ) and =Retiming ratio.

[0104] As shown in Equation 4, the velocity curve function indicates the change in velocity over time within the updated segment ( t The rate of change Retiring ratio Specify the magnitude of the increment curve. Note that in Equation 4, time = 0 represents the start pose / frame of the updated segment, and time = 1 represents the end pose / frame of the updated segment.

[0105] Figure 13 The velocity profiles of expanded and updated segments implemented via a retiming function according to various embodiments are shown. As shown, the expanded velocity profile 1310 of the expanded and updated segment will have an amplitude less than 1. At this point, a retiming ratio less than 1 indicates the expansion of the initial segment that generates the expanded and updated segment to slow down the motion in the character animation 150. For a retiming ratio less than 1, the amplitude of the velocity profile will be less than 1 and always greater than 0.

[0106] Figure 14 The velocity profiles of the shrunken, updated segment implemented via a retiming function according to various embodiments are shown. As shown, the shrunken velocity profile 1410 of the shrunken, updated segment will have an amplitude greater than 1. At this point, a retiming ratio greater than 1 indicates the shrinking of the initial segment that generates the shrunken, updated segment to accelerate movement in character animation 150. For a retiming ratio greater than 1, the amplitude of the velocity profile will be greater than 1.

[0107] At step 1280, the animation editing application 120 modifies a selected segment of character animation 150 based on a velocity curve function to generate an updated segment of character animation 150, which includes a retimed version of the selected segment. Specifically, the velocity curve function is automatically applied to the poses / frames of character animation 150 between the selected pair of key poses 830 to generate a retimed segment. Note that at step 1270, the animation editing application 120 calculates the velocity curve function of the character's motion to be realized by the updated segment. Then, at step 1280, the velocity curve function is applied to the selected segment to add or remove poses / frames from the selected segment based on the velocity curve function, in order to achieve the desired updated segment with the velocity curve function calculated at step 1270. Therefore, the resulting updated segment includes a retimed version of the selected segment.

[0108] At step 1280, for the expanded and updated segment, the animation editing application 120 achieves / realizes the expanded and updated segment's updated frame distance (e.g., by generating and adding an increased initial frame distance to the selected initial segment) through generating and adding an increased initial frame distance. Figure 10 As shown in the example, the selected segment requires multiple new poses / frames to modify it. Therefore, the expanded, updated segment will have a larger number of poses / frames compared to the selected segment. To generate new poses / frames, the animation editing application 120 can implement, for example, interpolation techniques to generate new poses / frames between the current poses / frames of the selected segment. Various frame interpolation techniques for generating new frames between existing frames can be used, such as motion estimation, blending, warping, etc. In other embodiments, the animation editing application 120 implements other techniques to generate new poses / frames for adding to the selected segment. In other embodiments, the animation editing application 120 utilizes the animation services of the animation server 190 to generate new poses / frames for adding to the selected segment.

[0109] Additionally, the extended velocity curve function 1310 instructs the animation editing application 120 where to generate and add new poses / frames within the selected segment. For example... Figure 13 As shown, the expanded velocity curve 1310 of the expanded updated segment indicates that the velocity at the beginning and end of the updated segment near the selected pair of key poses 830 (thus requiring relatively fewer poses / frames) is relatively greater than that in the middle of the updated segment with a relatively lower velocity (thus requiring relatively more poses / frames). Therefore, the expanded velocity curve function 1310 indicates that more new poses / frames should be added in the middle of the selected segment, and in contrast, fewer new poses / frames should be added in the beginning and end of the selected segment to achieve the desired updated segment.

[0110] At step 1280, for the shrunken updated segment, the animation editing application 120 modifies the selected segment by removing multiple poses / frames from the selected segment to achieve the updated frame distance of the shrunken updated segment (e.g., ...). Figure 11 As shown in the example, this speeds up the movement of the character within the selected segment. Therefore, the selected segment will have a greater number of poses / frames compared to the shrunken, updated segment. Additionally, the shrunken velocity curve function 1410 instructs the animation editing application 120 where to remove poses / frames within the selected segment. Figure 14 As shown, the shrunken velocity curve 1410 of the shrunken updated segment indicates that the velocity at the beginning and end of the updated segment near the selected pair of key poses 830 (thus requiring relatively more poses / frames) is relatively smaller than that in the middle of the updated segment, where the velocity is relatively greater (thus requiring relatively fewer poses / frames). Therefore, the shrunken velocity curve function 1410 indicates that more poses / frames should be removed towards the middle of the selected segment, and in contrast, fewer poses / frames should be removed towards the beginning and end of the selected segment to achieve the desired updated segment.

[0111] At step 1290, as an optional step, the animation editing application 120 modifies the visual appearance (such as color) of the curve representation of each activity trajectory dataset 220 displayed in the 3D timeline UI 130 based on the expansion or contraction of the selected segment. In these embodiments, the representation of the activity trajectory datasets 820 (such as 820a and 820b) may initially be displayed in a first color (such as yellow) before the user selects the initial segment. After the selected initial segment is retimed to generate an updated segment, the representation of the activity trajectory datasets 820 may be updated to be displayed in a second color different from the first color to indicate the retime of the selected segment. In some embodiments, the color change to the second color may be limited to the portion of the curve representation of the activity trajectory datasets 820 included within the updated segment, whereby the remaining portion of the curve representation of the activity trajectory datasets 820 for the rest of the character animation 150 retains the first color. In some embodiments, for the expanded, updated segment, the second color is red to indicate a slowdown in motion within the selected segment. In some embodiments, for the shrunken, updated segment, the second color is green to indicate an acceleration of motion in the selected segment.

[0112] In some embodiments, the second color can have different appearances (such as different levels of hue, tint, shade, etc.) based on the amount / degree of expansion or contraction of the selected segment. In these embodiments, the second color can have different appearances based on a retiming ratio that indicates the amount / degree of expansion or contraction of the selected segment. For example, for an expanded updated segment, a smaller retiming ratio (which would be less than 1) indicates a greater amount / degree of expansion of the selected segment compared to a larger retiming ratio. Therefore, for example, for a relatively lower retiming ratio of an expanded updated segment, the second color can have a relatively larger level of red shading. For example, for a contracted updated segment, a larger retiming ratio (which would be greater than 1) indicates a smaller amount / degree of contraction of the selected segment compared to a smaller retiming ratio. Therefore, for example, for a relatively larger retiming ratio of a contracted updated segment, the second color can have a relatively larger level of green shading.

[0113] At step 1292, the animation editing application 120 stores the character animation 150 with the updated segment (the retimed segment) as the updated character animation 150. Method 1200 then ends.

[0114] In summary, the disclosed technology enables the integration of spatial and temporal controls for editing character animations within a 3D timeline UI in animation editing applications. Character animation includes a set of frames containing motion capture data of a 3D model representing a set of poses for the specified character, with each frame including a specific pose of the character. The character also includes a set of joints incorporated into the 3D model of the character's pose set. The pose set includes a set of optional poses and a set of key poses representing the character's movement within the animation. The 3D timeline UI displays a timeline comprising a time axis with frame numbers as time units. The 3D timeline UI also displays 3D models of the key pose sets overlaid / overlaid on the timeline, with each 3D model of a specific key pose overlaid / overlaid at the frame number corresponding to the frame containing that specific key pose. The 3D timeline UI further displays a set of trajectories overlaid / overlaid on the key pose sets based on the character's joint sets. Each displayed trajectory includes a 3D curve traversing a specific joint of the character across the key pose set, helping the user visualize the passage of time and the character's movement between poses.

[0115] Users can directly edit the 3D model of the current key pose displayed in the 3D timeline UI by modifying the position and / or rotation of at least one joint of the 3D model in the current key pose. In response, the 3D timeline UI automatically propagates the corresponding changes to the 3D models of one or more poses near the current key pose and automatically updates the trajectory set based on the changes to the current key pose and nearby poses. The 3D timeline UI also automatically displays the updated trajectory set to indicate the changes to the current key pose and nearby poses, thereby helping users visualize the updated motion of the character that occurs between key poses. Changes to the current key pose and nearby poses generate modified character animations, which can be stored and further modified based on user edits.

[0116] The disclosed technology has at least one technical advantage over existing technologies in that it allows spatial and temporal controls for editing character animation to be integrated and housed together within a 3D timeline user interface (UI) of the animation editing application. The 3D timeline UI simultaneously displays a set of key poses and a set of trajectories overlaid on the animation timeline. Each displayed trajectory crosses specific joints of the character across the set of key poses, helping the user visualize the character's movement in different poses between key poses. The user can modify the current key pose displayed along the animation timeline, and in response, the 3D timeline UI automatically propagates the corresponding change to poses near the key pose and indicates these changes via an updated set of trajectories that helps the user visualize the modified movement of the character along the animation timeline. Utilizing this functionality, the 3D timeline UI simplifies animation editing for non-professional users compared to existing methods because it eliminates the need for users to constantly switch between spatial controls / views of poses and temporal controls / views of the animation timeline to modify the character's animation movement. Furthermore, when a change is made to the current key pose, the 3D timeline UI automatically indicates the resulting changes to nearby poses via an updated trajectory, eliminating the need for users to accurately predict how a change in one pose will affect nearby poses, as required by existing methods. In this way, the disclosed technology provides an accessible and intuitive interface that allows non-professional users to edit character animations more easily and efficiently compared to conventional editing applications. These technological advantages offer one or more technological improvements over existing methods.

[0117] In summary, the disclosed technology enables animation editing applications to automatically extract a set of key poses from character animation, representing the character's movement within the animation. This set of key poses is automatically extracted from the character animation based on one or more joint trajectories associated with the character animation. The animation editing application initially generates a trajectory for each joint of the character. The trajectory for a specific joint includes a dataset comprising a sorted set of positions for all poses of that joint across all frames of the character animation. The set of positions is sorted based on the frame number (time) corresponding to this set of positions. For example, a first trajectory might include a first set of positions for the character's right ankle joint, specifying the 3D spatial coordinates of the right ankle joint for each pose in each frame of the character animation. This first set of positions can be ordered sequentially based on the frame number (time). The animation editing application can generate and store trajectories for each joint of the character, such as 10 trajectories corresponding to 10 joints of the character.

[0118] The animation editing application then receives user selections for multiple active joints of the character and retrieves multiple activity trajectories corresponding to the multiple active joints. In other embodiments, the animation editing application selects a default set of multiple active joints. The animation editing application then extracts a set of key poses from the character animation based on the multiple activity trajectories. Specifically, each activity trajectory includes a set of local extreme positions / points, and the set of key poses for the character animation is identified based on the set of local extreme positions / points associated with the multiple activity trajectories. The set of local extreme positions is a set of local target positions that the animation editing application 120 aims to identify and utilize to extract a set of key poses 162 from the character animation 150. The user can also select new multiple active joints of the character, causing the animation editing application to dynamically identify a new set of key poses for the character animation based on the new multiple active joints. The animation editing application displays the set of key poses, allowing the user to select and directly edit key poses within the set to modify the character animation.

[0119] At least one technical advantage of the disclosed technology over existing technologies is that it enables animation editing applications to automatically and accurately extract key pose sets from character animations, which is impossible using existing methods. Therefore, the disclosed technology avoids non-professional users inaccurately selecting key pose sets from character animations. Specifically, the disclosed technology enables animation editing applications to extract key pose sets that accurately represent the motion of a character animation based on the trajectories of multiple joints in various poses of the character. Automatically identifying and extracting accurate key pose sets that represent character animations also improves the overall accuracy and quality of editing work performed on character animations. These technical advantages provide one or more technical improvements over existing methods.

[0120] In summary, the disclosed technology implements a retiming function in animation editing applications, which is applied to selected segments of character animation to slow down or speed up the character's movement in the selected segment. The user selects a segment by choosing a pair of key poses displayed along the animation timeline 210 in the 3D timeline UI 130 of the character animation 150. The selected pair of key poses includes a start key pose and an end key pose defining the selected segment. The start key pose is included in a "start frame" within the character animation 150 with an initial start frame number, and the end key pose is included in an "end frame" within the character animation 150 with an initial end frame number higher than the initial start frame number. Note that in this context, "start frame" and "end frame" do not refer to the start and end frames of the character animation 150, but rather to the start and end frames of the selected segment. The selected initial segment has an initial frame distance equal to the difference between the initial end frame number and the initial start frame number (i.e., the initial end frame number minus the initial start frame number).

[0121] The user can then expand (increase) or contract (decrease) the selected initial segment along the animation timeline 210 to slow down or speed up the character's movement within the selected initial segment, respectively. The user can do this by expanding (increasing) or contracting (decreaseing) the frame distance between the selected pair of key poses along the timeline. Expanding or contracting the selected initial segment along the timeline generates an updated segment defined by a start key pose with an updated start frame number and an end key pose with an updated end frame number, the updated end frame number being higher than the updated start frame number. This updated segment has an updated frame distance equal to the difference between the updated end frame number and the updated start frame number (i.e., the updated end frame number minus the updated start frame number).

[0122] In response to receiving an updated clip, the animation editing application 120 automatically performs one or more retiming operations on the selected initial clip based on the degree of expansion or contraction along the animation timeline 210. Specifically, the animation editing application 120 automatically generates a retiming ratio based on the amount of expansion or contraction, and then generates a velocity curve function based on the retiming ratio. The retiming ratio is equal to the ratio between the initial frame distance and the updated frame distance, whereby a retiming ratio less than 1 indicates expansion (slowing down), and a retiming ratio greater than 1 indicates contraction (speeding up). The velocity curve function is automatically applied to all poses / frames of the selected clip to provide smooth and natural movement for the characters in the selected clip.

[0123] The disclosed technology has at least one technical advantage over existing technologies in that it implements a retiming function in animation editing applications. This retiming function can be automatically applied to selected segments of character animation to slow down or speed up the character's movement in the selected segments, which is impossible to achieve using existing methods. In operation, the user expands or contracts the selected segment to specify the updated segment. In response, the retiming function automatically generates a velocity curve function based on the amount of expansion or contraction and applies the velocity curve function to the selected segment to add new frames or remove frames from the selected segment, thereby automatically generating an updated segment in which the character has smooth and natural movement. In this way, the disclosed technology provides a retiming function that allows both professional and non-professional users to retiming given segments of character animation more accurately and efficiently, thereby improving the quality of character animation compared to what can be achieved using existing methods. These technical advantages provide one or more technical improvements over existing methods.

[0124] The various aspects of the subject matter described herein are listed in the following numbered clauses.

[0125] 1. In some embodiments, a computer-implemented method for modifying character animation via a user interface includes: displaying a timeline including frame numbers as units of time; displaying a set of key poses along the timeline based on a set of frame numbers associated with a set of key poses of the character; displaying a set of trajectories superimposed on the set of key poses, wherein each trajectory in the set of trajectories is displayed as a curve passing through a specific joint of the character across the set of key poses; and receiving a modification to a first key pose included in the set of key poses via the user interface.

[0126] 2. The computer-implemented method as described in Clause 1, further comprising: in response to receiving the modification, modifying a set of neighboring poses of the first key pose in the character animation based on the modification of the first key pose, wherein the set of neighboring poses is defined by a pair of key poses in the set of key poses.

[0127] 3. The computer-implemented method as described in Clause 1 or 2, wherein one key pose included in the pair of key poses is adjacent to the first key pose on a first side of the first key pose, and the other key pose included in the pair of key poses is adjusted to the first key pose on a second side of the first key pose.

[0128] 4. The computer-implemented method as described in any one of Clauses 1 to 3, wherein each key pose in the pair of key poses is selected by the user.

[0129] 5. The computer-implemented method of any one of Clauses 1 to 4, further comprising: receiving, via the user interface, a selection of a first key pose in the set of key poses; and, in response, displaying a selectable element on a first joint associated with the first key pose in the set of key poses, wherein the selectable element is selectable for modifying the position or rotation of the first joint.

[0130] 6. The computer-implemented method as described in any one of Clauses 1 to 5, further comprising: receiving, via the user interface, a modification of a first key pose in the set of key poses; modifying the character animation based on the modification of the first key pose to generate a modified character animation; and displaying the updated set of key poses and the updated set of trajectories based on the modified character animation.

[0131] 7. A computer-implemented method as described in any one of Clauses 1 to 6, wherein receiving the modification of the first key pose includes receiving a modification of the position or rotation of a first joint associated with the first key pose.

[0132] 8. A computer-implemented method as described in any one of Clauses 1 to 7, wherein displaying the set of key poses of the character includes displaying a three-dimensional (3D) model of the character for each key pose in the set of key poses along the timeline.

[0133] 9. A computer-implemented method as described in any one of Clauses 1 to 8, wherein the user interface includes a three-dimensional immersive environment interface.

[0134] 10. The computer-implemented method as described in any one of Clauses 1 to 9, wherein the immersive environment interface includes a virtual reality interface or an augmented reality interface.

[0135] 11. In some embodiments, one or more non-transitory computer-readable media include instructions that, when executed by one or more processors, cause the one or more processors to modify a character animation via a user interface by performing the following steps: displaying a timeline including frame numbers as units of time; displaying a set of key poses along the timeline based on a set of frame numbers associated with a set of key poses of the character; displaying a set of trajectories superimposed on the set of key poses, wherein each trajectory in the set of trajectories is displayed as a curve passing through a specific joint of the character across the set of key poses; and receiving a modification to a first key pose included in the set of key poses via the user interface.

[0136] 12. One or more non-transitory computer-readable media as described in Clause 11, further comprising: in response to receiving the modification, modifying a set of neighboring poses of the first key pose in the character animation based on the modification of the first key pose, wherein the set of neighboring poses is defined by a pair of key poses in the set of key poses.

[0137] 13. One or more non-transitory computer-readable media as described in Clause 11 or 12, wherein one key pose included in the pair of key poses is adjacent to the first key pose on a first side of the first key pose, and the other key pose included in the pair of key poses is adjusted to the first key pose on a second side of the first key pose.

[0138] 14. One or more non-transitory computer-readable media as described in any one of Clauses 11 to 13, wherein each of the key poses in the pair of key poses is selected by the user.

[0139] 15. One or more non-transitory computer-readable media as described in any one of Clauses 11 to 14, further comprising: receiving, via the user interface, a selection of a first key pose in the set of key poses; and, in response, displaying a selectable element on a first joint associated with the first key pose in the set of key poses, wherein the selectable element is selectable for modifying the position or rotation of the first joint.

[0140] 16. One or more non-transitory computer-readable media as described in any one of Clauses 11 to 15, further comprising: receiving, via the user interface, a modification of a first key pose in the set of key poses; modifying the character animation based on the modification of the first key pose to generate a modified character animation; and displaying the updated set of key poses and the updated set of trajectories based on the modified character animation.

[0141] 17. One or more non-transitory computer-readable media as described in any one of Clauses 11 to 16, wherein the character animation includes the set of key poses and the set of non-key poses, wherein the timeline displays the frame number of each key pose in the set of key poses in the character animation and does not display the frame number of any non-key pose in the set of non-key poses.

[0142] 18. One or more non-transitory computer-readable media as described in any one of Clauses 11 to 17, wherein the timeline displays an optional frame number for each key pose in the set of key poses.

[0143] 19. One or more non-transitory computer-readable media as described in any one of Clauses 11 to 18, further comprising: receiving a selection of a first selectable frame number on a timeline for a first key pose in the set of key poses; and, in response, guiding the first key pose in the set of key poses.

[0144] 20. In some embodiments, a computer system includes: a memory including instructions; and at least one processor coupled to the memory, which, when executing the instructions, modifies a character animation via a user interface by performing the following steps: displaying a timeline including frame numbers as units of time; displaying a set of key poses along the timeline based on a set of frame numbers associated with a set of key poses of the character; displaying a set of trajectories superimposed on the set of key poses, wherein each trajectory in the set of trajectories is displayed as a curve passing through a specific joint of the character across the set of key poses; and receiving, via the user interface, a modification of a first key pose included in the set of key poses.

[0145] Any and all combinations of any element of the claim set forth in any claim and / or any element described in this application, in any manner, fall within the scope of this embodiment and the intent of protection.

[0146] Various embodiments have been described for illustrative purposes, but these descriptions are not intended to be exhaustive or limited to the disclosed embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described embodiments.

[0147] Various aspects of the present invention may be embodied as systems, methods, or computer program products. Therefore, aspects of this disclosure may take the form of entirely hardware embodiments, entirely software embodiments (including firmware, resident software, microcode, etc.), or embodiments combining software and hardware aspects, all of which are generally referred to herein as “modules” or “systems.” Furthermore, any hardware and / or software techniques, processes, functions, components, engines, modules, or systems described in this disclosure may be implemented as circuits or sets of circuits. Additionally, aspects of this disclosure may take the form of computer program products embodied on one or more computer-readable media having computer-readable program code embodied thereon. Software constructs and entities (e.g., engines, modules, GUIs, etc.) are stored in one or more memories shown in the relevant system diagrams in various embodiments and executed by processors shown in those same system diagrams.

[0148] Any combination of one or more computer-readable media may be used. A computer-readable medium can be a computer-readable signal medium or a computer-readable storage medium. A computer-readable storage medium can be, for example, but not limited to, electronic, non-transitory, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices, or apparatuses, or any suitable combination thereof. More specific examples (not an exhaustive list) of computer-readable storage media will include: electrical connections having one or more wires, portable computer floppy disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable optical disc read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. In the context of this document, a computer-readable storage medium can be any tangible medium that can contain or store programs for use with or in connection with an instruction execution system, device, or apparatus.

[0149] The foregoing description of aspects of this disclosure is based on flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this disclosure. It should be understood that each block in the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, or other programmable data processing apparatus to produce a machine. When executed via a processor of a computer or other programmable data processing apparatus, these instructions enable the implementation of the functions / actions specified in one or more blocks of the flowchart illustrations and / or block diagrams. Such processors can be, but are not limited to, general-purpose processors, special-purpose processors, special-purpose processors, or field-programmable gate arrays.

[0150] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code comprising one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may not occur in the order shown in the drawings. For example, two blocks shown consecutively may actually be executed substantially concurrently, or these blocks may sometimes be executed in reverse order, depending on the functionality involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, may be implemented by a system based on special-purpose hardware or a combination of special-purpose hardware and computer instructions that performs the specified function or action.

[0151] While the foregoing describes embodiments of this disclosure, other and additional embodiments of this disclosure may be devised without departing from the basic scope of this disclosure, the scope of which is defined by the appended claims.

Claims

1. A computer-implemented method for modifying character animations via a user interface, the method comprising: Displays a timeline including frame numbers as time units; The key pose set is displayed along the timeline based on the set of frame numbers associated with the key pose set of the character; Displays a set of trajectories superimposed on the set of key poses, wherein each trajectory in the set of trajectories is displayed as a curve passing through a specific joint of the character across the set of key poses; and Modifications to the first key pose included in the key pose set are received via the user interface.

2. The computer-implemented method as described in claim 1, further comprising: In response to receiving the modification, the set of neighboring poses of the first key pose in the character animation is modified based on the modification of the first key pose, wherein the set of neighboring poses is defined by a pair of key poses in the set of key poses.

3. The computer-implemented method as described in claim 2, wherein, One of the key poses in the pair of key poses is adjacent to the first key pose on a first side of the first key pose, and the other key pose in the pair of key poses is adjusted to the first key pose on a second side of the first key pose.

4. The computer-implemented method as described in claim 2, wherein, Each of the pair of key poses is selected by the user.

5. The computer-implemented method as described in claim 1, further comprising: The user interface receives the selection of a first key pose from the set of key poses. as well as In response, selectable elements are displayed on a first joint associated with the first key pose in the set of key poses, wherein the selectable elements can be selected to modify the position or rotation of the first joint.

6. The computer-implemented method of claim 1, further comprising: Modifications to the first key pose in the key pose set are received via the user interface; The character animation is modified based on the modification of the first key pose to generate a modified character animation; as well as The updated set of key poses and the updated set of trajectories are displayed based on the modified character animation.

7. The computer-implemented method as described in claim 6, wherein, Receiving modifications to the first key pose includes receiving modifications to the position or rotation of the first joint associated with the first key pose.

8. The computer-implemented method as described in claim 1, wherein, The set of key poses for the character includes displaying a 3D model of the character for each key pose in the set of key poses along the timeline.

9. The computer-implemented method as described in claim 1, wherein, The user interface includes a three-dimensional immersive environment interface.

10. The computer-implemented method as described in claim 9, wherein, The immersive environment interface includes a virtual reality interface or an augmented reality interface.

11. One or more non-transitory computer-readable media, including instructions that, when executed by one or more processors, cause the one or more processors to modify character animation via a user interface by performing the following steps: Displays a timeline including frame numbers as time units; The key pose set is displayed along the timeline based on the set of frame numbers associated with the key pose set of the character; Displays the set of trajectories superimposed on the set of key attitudes, wherein... Each trajectory in the trajectory set is displayed as a curve traversing a specific joint of the character across the set of key poses; and Modifications to the first key pose included in the key pose set are received via the user interface.

12. The one or more non-transitory computer-readable media of claim 11, further comprising: In response to receiving the modification, the set of neighboring poses of the first key pose in the character animation is modified based on the modification of the first key pose, wherein the set of neighboring poses is defined by a pair of key poses in the set of key poses.

13. One or more non-transitory computer-readable media as claimed in claim 12, wherein, One of the key poses in the pair of key poses is adjacent to the first key pose on a first side of the first key pose, and the other key pose in the pair of key poses is adjusted to the first key pose on a second side of the first key pose.

14. One or more non-transitory computer-readable media as claimed in claim 12, wherein, Each of the pair of key poses is selected by the user.

15. The one or more non-transitory computer-readable media of claim 11, further comprising: The user interface receives the selection of a first key pose from the set of key poses. as well as In response, selectable elements are displayed on a first joint associated with the first key pose in the set of key poses, wherein the selectable elements can be selected to modify the position or rotation of the first joint.

16. The one or more non-transitory computer-readable media of claim 11, further comprising: Modifications to the first key pose in the key pose set are received via the user interface; The character animation is modified based on the modification of the first key pose to generate a modified character animation; as well as The updated set of key poses and the updated set of trajectories are displayed based on the modified character animation.

17. One or more non-transitory computer-readable media as claimed in claim 11, wherein, The character animation includes the set of key poses and the set of non-key poses, wherein the timeline displays the frame number of each key pose in the set of key poses in the character animation and does not display the frame number of any non-key pose in the set of non-key poses.

18. One or more non-transitory computer-readable media as claimed in claim 11, wherein, The timeline displays the selectable frame number for each key pose in the set of key poses.

19. The one or more non-transitory computer-readable media of claim 18, further comprising: Receive the selection of a first selectable frame number on the timeline for a first key pose in the set of key poses; as well as In response, the system navigates to the first key pose in the set of key poses.

20. A computer system, comprising: The memory stores instructions; as well as At least one processor, coupled to the memory, modifies the character animation via a user interface by performing the following steps when executing these instructions: Displays a timeline including frame numbers as time units; The key pose set is displayed along the timeline based on the set of frame numbers associated with the key pose set of the character; Displays a set of trajectories superimposed on the set of key poses, wherein each trajectory in the set of trajectories is displayed as a curve passing through a specific joint of the character across the set of key poses; and Modifications to the first key pose included in the key pose set are received via the user interface.

21. A computer-implemented method for automatically extracting a set of key poses from character animation, the method comprising: Determine multiple trajectories for multiple joints associated with a character included in a character animation, wherein each trajectory includes a set of positions of the corresponding joints included in the multiple joints across a set of poses of the character in the character animation; Extracting the key pose set from the character animation based on the multiple trajectories; and The set of key poses is displayed within the user interface for further processing.

22. The computer-implemented method as described in claim 21, wherein, Each of the plurality of trajectories includes a set of local target positions, wherein the key pose set extracted from the character animation is further based on the set of local target positions of each of the plurality of trajectories.

23. The computer-implemented method as described in claim 21, wherein, Extracting the key pose set from the character animation includes calculating the difference between the trajectory and the smoothed form of the trajectory for each of the plurality of trajectories.

24. The computer-implemented method as described in claim 21, wherein, Extracting the key pose set from the character animation includes calculating weights for each of the plurality of trajectories that indicate the amount of motion of the corresponding joints included in the plurality of joints.

25. The computer-implemented method of claim 21, further comprising: Receive selections for multiple different joints associated with the role; In response, different trajectories are determined for the different joints associated with the character; as well as Different sets of key poses are extracted from the character animation based on the different multiple trajectories.

26. The computer-implemented method of claim 25, wherein the different plurality of joints associated with the role are selected via a joint diagram displayed in the user interface.

27. The computer-implemented method of claim 21, further comprising: The selection of different joints for the character is received via a joint diagram displayed in the user interface, wherein the joint diagram displays selectable joint sets representing the character's set of joints, the selectable joint sets being arranged in a tree-like hierarchical arrangement showing the relationships between the character's set of joints.

28. The computer-implemented method as described in claim 21, wherein, The set of key poses represents the first movement of the character in the character animation.

29. The computer-implemented method as described in claim 21, wherein, The set of positions of the corresponding joints included in each trajectory includes a time-ordered set of three-dimensional 3D positions of the corresponding joints across the set of frames included in the character animation.

30. The computer-implemented method as described in claim 21, wherein, The plurality of joints associated with the character comprises a subset of all joints of the character.

31. One or more non-transitory computer-readable media, including instructions that, when executed by one or more processors, cause the one or more processors to automatically extract a set of key poses from a character animation by performing the following steps: Determine multiple trajectories for multiple joints associated with the character included in the character animation, where, Each trajectory includes a set of positions of the corresponding joints included in the plurality of joints across the set of poses of the character in the character animation; The key pose set is extracted from the character animation based on the multiple trajectories; as well as The set of key poses is displayed within the user interface for further processing.

32. One or more non-transitory computer-readable media as claimed in claim 31, wherein, Each of the plurality of trajectories includes a set of local target positions, wherein the key pose set extracted from the character animation is further based on the set of local target positions of each of the plurality of trajectories.

33. One or more non-transitory computer-readable media as claimed in claim 31, wherein, Extracting the key pose set from the character animation includes calculating the difference between the trajectory and the smoothed form of the trajectory for each of the plurality of trajectories.

34. One or more non-transitory computer-readable media as claimed in claim 31, wherein, Extracting the key pose set from the character animation includes calculating weights for each of the plurality of trajectories that indicate the amount of motion of the corresponding joints included in the plurality of joints.

35. The one or more non-transitory computer-readable media of claim 31, further comprising: Receive selections for multiple different joints associated with the role; In response, different trajectories are determined for the different joints associated with the character; as well as Different sets of key poses are extracted from the character animation based on the different multiple trajectories.

36. In one or more non-transitory computer-readable media as claimed in claim 35, the different plurality of joints associated with the role are selected via a joint diagram displayed in the user interface.

37. The one or more non-transitory computer-readable media of claim 31, further comprising: Modifications to the first key pose in the key pose set are received via the user interface; as well as In response, the character animation is modified based on the modification of the first key pose.

38. One or more non-transitory computer-readable media as claimed in claim 31, wherein, The character animation includes a set of non-critical poses and a set of critical poses, wherein the set of critical poses represents the movement of the character in the character animation.

39. One or more non-transitory computer-readable media as claimed in claim 31, wherein, Displaying the set of key poses in the user interface includes displaying the set of key poses along a timeline with frame numbers as the time unit.

40. A computer system, comprising: The memory stores instructions; as well as At least one processor, coupled to the memory, automatically extracts a set of key poses from the character animation by means of the following steps when executing these instructions: Determine multiple trajectories for multiple joints associated with a character included in a character animation, wherein each trajectory includes a set of positions of the corresponding joints included in the multiple joints across a set of poses of the character in the character animation; Extracting the key pose set from the character animation based on the multiple trajectories; and The set of key poses is displayed within the user interface for further processing.

41. A computer-implemented method for retiming segments included in a character animation, the method comprising: Display the set of key poses associated with the character's animation along the animation timeline; Receive selection of multiple key poses included in the pose set, the multiple key poses defining the selected segment included in the character animation; Receive a command for expanding or contracting a selected segment along the animation timeline, wherein the command specifies the updated segment; as well as Perform one or more retiming operations on the selected segment to generate the updated segment.

42. The computer-implemented method as described in claim 41, wherein, The one or more retiming operations performed on the selected segment are based on the extent to which the selected segment expands or contracts along the animation timeline.

43. The computer-implemented method as described in claim 41, wherein, Performing one or more retiming operations on the selected segment includes: Calculate the extent to which the selected segment expands or contracts along the animation timeline; and The velocity curve function is calculated based on the degree of expansion or contraction of the selected segment.

44. The computer-implemented method as described in claim 43, wherein, Performing the one or more retiming operations on the selected segment further includes modifying the selected segment based on the velocity curve function.

45. The computer-implemented method as described in claim 43, wherein, Performing the one or more retiming operations on the selected segment further includes removing one or more poses included in the selected segment based on the velocity curve function.

46. ​​The computer-implemented method as described in claim 43, wherein, Performing the one or more retiming operations on the selected segment further includes adding one or more attitudes to the selected segment based on the velocity curve function.

47. The computer-implemented method as described in claim 41, wherein, The received commands include commands to expand the selected segment to slow down the movement of the character within the selected segment.

48. The computer-implemented method as described in claim 41, wherein, The received commands include commands for shrinking the selected segment to accelerate the movement of the character within the selected segment.

49. The computer-implemented method as described in claim 41, wherein, The set of key poses and the animation timeline are displayed via a 3D immersive environment interface.

50. The computer-implemented method as described in claim 49, wherein, The multiple key attitudes included in the set of key attitudes are selected by one or more immersive environment controllers.

51. One or more non-transitory computer-readable media, including instructions that, when executed by one or more processors, cause the one or more processors to retime segments included in a character animation by performing the following steps: Display the set of key poses associated with the character's animation along the animation timeline; Receive selection of multiple key poses included in the pose set, the multiple key poses defining the selected segment included in the character animation; Receive commands for expanding or contracting the selected segment along the animation timeline, wherein, The command specifies the segment to be updated; as well as Perform one or more retiming operations on the selected segment to generate the updated segment.

52. One or more non-transitory computer-readable media as claimed in claim 51, wherein, The one or more retiming operations performed on the selected segment are based on the extent to which the selected segment expands or contracts along the animation timeline.

53. One or more non-transitory computer-readable media as claimed in claim 51, wherein, Performing one or more retiming operations on the selected segment includes: Calculate the extent to which the selected segment expands or contracts along the animation timeline; and The velocity curve function is calculated based on the degree of expansion or contraction of the selected segment.

54. One or more non-transitory computer-readable media as claimed in claim 53, wherein, Performing the one or more retiming operations on the selected segment further includes modifying the selected segment based on the velocity curve function.

55. One or more non-transitory computer-readable media as claimed in claim 53, wherein, Performing the one or more retiming operations on the selected segment further includes removing one or more poses included in the selected segment based on the velocity curve function.

56. One or more non-transitory computer-readable media as claimed in claim 53, wherein, Performing the one or more retiming operations on the selected segment further includes adding one or more attitudes to the selected segment based on the velocity curve function.

57. One or more non-transitory computer-readable media as claimed in claim 51, wherein, Performing one or more retiming operations on the selected segment includes: Calculate the initial frame distance associated with the selected segment; and Calculate the updated frame distance associated with the updated segment.

58. One or more non-transitory computer-readable media as claimed in claim 57, wherein, Performing the one or more retiming operations on the selected segment further includes: The retiming ratio is calculated based on the initial frame distance and the updated frame distance; and The selected segment is modified based on the retiming ratio.

59. One or more non-transitory computer-readable media as claimed in claim 57, wherein, Performing the one or more retiming operations on the selected segment further includes: The velocity curve function is calculated based on the initial frame distance and the updated frame distance; The selected segment is modified based on the velocity curve function.

60. A computer system comprising: The memory stores instructions; as well as At least one processor, coupled to the memory, retiming segments included in the character animation by performing the following steps when executing these instructions: Display the set of key poses associated with the character's animation along the animation timeline; Receive selection of multiple key poses included in the pose set, the multiple key poses defining the selected segment included in the character animation; Receive a command for expanding or contracting a selected segment along the animation timeline, wherein the command specifies the updated segment; as well as Perform one or more retiming operations on the selected segment to generate the updated segment.

Images