Animation point position or animation shape simulation method, device, system, medium

Abstract

Provided are an animation point position or animation shape simulation method, device, system and medium. The method comprises: receiving a time point of an animation point position or an animation shape expected to be simulated; outputting an expected animation point position or an expected animation shape based on the time point through a trained neural network, wherein the trained neural network is trained in the following manner: generating a differential equation based on motion physics simulation constraints of the animation point position or the animation shape, the differential equation representing a relationship between the animation point position or the animation shape and the time point under the constraint of the motion physics simulation constraints; generating a loss function of the neural network based on the differential equation, wherein the loss function comprises a loss related to the differential equation; and training the neural network based on the loss function to output the expected animation point position or the expected animation shape based on an input time point. Animation interpolation is performed using the neural network, and changes of the animation meet natural physical constraints, and are more smooth, natural and real.

Description

Technical Field

[0001] This application relates to the field of image processing, and more specifically to methods, devices, systems, and media for simulating the position or shape of animation points. Background Technology

[0002] In the field of animation, such as when editing or processing animation, or when creating video effects, to achieve smooth and natural movement or change, at least two distinct key states (e.g., key positions or key shapes) must be provided at the beginning and end. The transitions and connections between these intermediate states can be automatically achieved by the computer through interpolation techniques. A frame is the smallest unit of image in animation; a frame representing a key state is called a keyframe. Interpolated frames are called interpolation frames. Therefore, smooth animation can often be created using keyframes at both ends and interpolation frames in the middle, reducing the workload of creating each interpolation frame individually. Keyframe technology is the most fundamental and widely used animation simulation method in computer animation.

[0003] Existing technologies offer various methods for automatically interpolating animation between keyframes, but the interpolated animation still lacks naturalness. A smoother, more natural animation interpolation technique is needed. Summary of the Invention

[0004] To address the aforementioned problems, according to one aspect of this application, a method for simulating animation point positions or animation shapes based on motion physics simulation constraints using a neural network is provided. The method includes: receiving one or more time points of one or more animation point positions or animation shapes to be simulated; outputting the expected animation point positions or expected animation shapes based on the one or more time points using a trained neural network, wherein the trained neural network is trained by: generating differential equations based on motion physics simulation constraints on the animation point positions or animation shapes, the differential equations representing the relationship between the animation point positions or animation shapes and the time points, constrained by the motion physics simulation constraints; generating a loss function for the neural network based on the differential equations, wherein the loss function includes a loss associated with the differential equations; and training the neural network based on the loss function to output the expected animation point positions or expected animation shapes based on the input time points.

[0005] In one embodiment, the motion physics simulation constraint is a constraint that can be defined by differential equations.

[0006] In one embodiment, the motion physics simulation constraints include force-related motion physics simulation constraints based on Newtonian mechanics.

[0007] In one embodiment, the loss associated with the differential equation includes the difference between the relationship between the expected animation point location or expected animation shape output by the neural network and the time point and the differential equation.

[0008] In one embodiment, the loss associated with the differential equation is achieved by designing a loss function to represent whether the expected animation point position or expected animation shape output by the neural network satisfies the differential equation with respect to the time point, wherein the loss function is differentiable.

[0009] In one embodiment, the system receives user-defined animation point positions or animation shapes for one or more second time points. The loss function further includes the difference between the expected animation point positions or expected animation shapes output by the neural network for the one or more second time points and the defined animation point positions or animation shapes.

[0010] In one embodiment, setting the animation point position or setting the animation shape includes at least one of the following: the start and end points of the animation curve, the key points of the animation curve, the start shape of the animation, the end shape of the animation, and the key shape of the animation shape.

[0011] In one embodiment, the differential equation includes one or more of ordinary differential equations and partial differential equations, and the differential equation has one or more parameters or no parameters.

[0012] In one embodiment, the backpropagation method is used to train the neural network so that the loss function is minimized or less than a preset threshold.

[0013] According to another aspect of this application, a device for simulating animation point positions or animation shapes based on motion physics simulation constraints using a neural network is provided, comprising: a receiver configured to receive one or more time points of one or more animation point positions or animation shapes to be simulated; and a simulator configured to output the expected animation point positions or expected animation shapes based on the one or more time points using a trained neural network, wherein the trained neural network is trained by: generating differential equations based on motion physics simulation constraints on the animation point positions or animation shapes, the differential equations representing the relationship between the animation point positions or animation shapes and the time points constrained by the motion physics simulation constraints; generating a loss function for the neural network based on the differential equations, wherein the loss function includes a loss associated with the differential equations; and training the neural network based on the loss function to output the expected animation point positions or expected animation shapes based on the input time points.

[0014] In one embodiment, the motion physics simulation constraint is a constraint that can be defined by differential equations.

[0015] In one embodiment, the motion physics simulation constraints include force-related motion physics simulation constraints based on Newtonian mechanics.

[0016] In one embodiment, the loss associated with the differential equation includes the difference between the relationship between the expected animation point location or expected animation shape output by the neural network and the time point and the differential equation.

[0017] In one embodiment, the loss associated with the differential equation is achieved by designing a loss function to represent whether the expected animation point position or expected animation shape output by the neural network satisfies the differential equation with respect to the time point, wherein the loss function is differentiable.

[0018] In one embodiment, the system receives user-defined animation point positions or animation shapes for one or more second time points. The loss function further includes the difference between the expected animation point positions or expected animation shapes output by the neural network for the one or more second time points and the defined animation point positions or animation shapes.

[0019] In one embodiment, setting the animation point position or setting the animation shape includes at least one of the following: the start and end points of the animation curve, the key points of the animation curve, the start shape of the animation, the end shape of the animation, and the key shape of the animation shape.

[0020] In one embodiment, the differential equation includes one or more of ordinary differential equations and partial differential equations, and the differential equation has one or more parameters or no parameters.

[0021] In one embodiment, the backpropagation method is used to train the neural network so that the loss function is minimized or less than a preset threshold.

[0022] According to another aspect of this application, a simulation system for the position or shape of animated points based on motion physics simulation constraints using neural networks is provided, comprising: a processor; and a memory storing one or more computer-executable instructions which, when executed by the processor, perform the method of this application.

[0023] According to another aspect of this application, a non-transitory computer-readable medium is provided, storing one or more computer-executable instructions which, when executed by a processor, perform the method of this application.

[0024] In this way, the trained neural network is used for animation interpolation, so that the curves or shapes of the animation meet the natural physical constraints, making the interpolated animation smooth, natural and realistic. Attached Figure Description

[0025] To more clearly illustrate the technical solutions in the embodiments of this disclosure or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this disclosure. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0026] Figure 1 An exemplary diagram illustrates how three types of curves are used to interpolate between keyframes in the prior art.

[0027] Figure 2 A schematic flowchart of an animation point position or animation shape simulation method based on motion physics simulation constraints using a neural network, according to an embodiment of this application, is shown.

[0028] Figure 3A An example diagram shows the result of interpolating between the start and end positions of an animation point according to a prior art method.

[0029] Figure 3B An example diagram is shown showing the result of interpolating between the start and end positions of an animation point, taking into account physical constraints, according to an embodiment of this application.

[0030] Figure 4A A flowchart illustrating a neural network training method for interpolating between the start and end positions of animation points, according to an embodiment of this application, further considers the user's editing or customization needs.

[0031] Figure 4B An example diagram is shown illustrating the result of interpolating between the start and end positions of an animation point, taking into account the user's editing or customization needs, according to an embodiment of this application.

[0032] Figure 5 An example diagram is shown illustrating the result of interpolation between the start and end positions of an animation point applied to a partial differential equation according to an embodiment of this application.

[0033] Figure 6 A block diagram of an animation point position or animation shape simulation device based on motion physics simulation constraints using a neural network, according to an embodiment of this application, is shown.

[0034] Figure 7 A block diagram of an exemplary computer system suitable for implementing embodiments of the present application is shown.

[0035] Figure 8 A schematic diagram of a non-transitory computer-readable storage medium according to an embodiment of the present disclosure is shown. Detailed Implementation

[0036] Specific embodiments of this application will now be described in detail, with examples of the application illustrated in the accompanying drawings. Although this application will be described in conjunction with specific embodiments, it will be understood that it is not intended to limit this application to the described embodiments. Rather, it is intended to cover variations, modifications, and equivalents included within the spirit and scope of this application as defined by the appended claims. It should be noted that the method steps described herein can be implemented by any functional block or functional arrangement, and any functional block or functional arrangement can be implemented as a physical entity or a logical entity, or a combination of both.

[0037] Existing techniques typically use specific curves to interpolate intermediate states in animation. To put it more vividly, a keyframe in animation includes, for example, the position of an animation point at a key time point. Interpolation in the middle of the animation involves, for example, the position of an animation point at an intermediate time point. The curves between keyframes are formed using a certain interpolation method. These curves can be categorized into three types: Bézier curves, linear curves, and constant curves.

[0038] Figure 1 An exemplary diagram illustrates how three types of curves are used to interpolate between keyframes in the prior art.

[0039] Bézier curves are mathematical curves used in 2D graphics applications. A curve is defined by a start point, an end point (also called an anchor point), and control points. By adjusting the control points, the shape of the Bézier curve changes. Moving the endpoints changes the curvature (degree of bending) of the curve; moving the midpoint (i.e., moving the virtual control line) causes the Bézier curve to move uniformly while the start and end points are locked. Figure 1 As shown, assuming the animation point of the keyframe moves from position A at time 1 second to position B at another keyframe at time 4 seconds, the intermediate interpolation curve is a Bezier curve.

[0040] A linear curve refers to an animation curve where the position of an animation point changes linearly over time between the interpolation and the keyframe. For example... Figure 1 As shown, assuming the animation point of the keyframe moves from position B at time 4 seconds to position C at another keyframe at time 6 seconds, the interpolation curve used is a linear curve.

[0041] Constant curves directly use the keyframe values ​​for constant interpolation. For example, the position of an animation point in the keyframe and the intermediate interpolation frames remains constant; that is, the position of the animation point does not change. Figure 1 As shown, assuming the animation point of the keyframe moves from position C at time 6 seconds to position D at time 7 seconds of another keyframe, the interpolation curve used is a constant curve.

[0042] certainly, Figure 1 The examples shown are merely illustrative.

[0043] However, the aforementioned existing technology has the following drawbacks:

[0044] The values ​​obtained by interpolating keyframes using the above method do not conform to the corresponding physical laws; the extrapolated results of the interpolation do not conform to the actual motion laws, meaning that the motion of the animated points does not conform to the objective physical laws of nature. Therefore, the animation will not appear realistic or natural.

[0045] If the motion trajectory of animated points is strictly followed according to the laws of physics, the simulation results, which conform to the laws of physics, are not easy to use for animation creation. This is because users cannot easily integrate the simulation results into the animation for secondary editing and adjustments, as the data after secondary editing is unlikely to meet physical constraints again. In complex motion scenarios, users need to perform tedious data editing even more.

[0046] There is currently no good technical solution to solve the above-mentioned technical problems.

[0047] This application can integrate the physical simulation process with the animation curve interpolation method through the Physics-informed neural network method to help perform animation curve interpolation, so that the motion of the animation conforms to the laws of physical motion, and further makes it easy for users to edit and adjust, as well as to customize the constraints of the animation motion.

[0048] The starting point of this application is to integrate numerical constraints from at least one of the following two aspects: constraints of real physical simulation laws on animation curves or shapes, and numerical constraints generated by user-edited (or customized) key states of animation curves or shapes.

[0049] The solution proposed in this application achieves the purpose of interpolating animation states (animation point position curves or animation shapes) through neural network training. The main principle is to use physical constraints that satisfy the laws of natural (force) motion as the regularization term of the neural network training loss function, or further use user-edited data points as training data for the neural network. Finally, the neural network is trained to generate an optimized neural network. The trained neural network is then used to interpolate animation states, thereby ensuring that the curve or shape changes of the animation states satisfy the natural physical constraints and further reflect the user's editing intentions, making the interpolated animation smooth, natural, and realistic.

[0050] Figure 2 A schematic flowchart of an animation point position or animation shape simulation method 200 based on motion physics simulation constraints using a neural network, according to an embodiment of this application, is shown.

[0051] The method 200 includes: step 210, receiving one or more time points of one or more animation point positions or animation shapes to be simulated; step 220, outputting the expected animation point positions or expected animation shapes based on the one or more time points using a trained neural network. The trained neural network is trained as follows: generating differential equations based on the motion physics simulation constraints of the animation point positions or animation shapes, the differential equations representing the relationship between the animation point positions or animation shapes and the time points, constrained by the motion physics simulation constraints; generating a loss function for the neural network based on the differential equations, wherein the loss function includes a loss associated with the differential equations; and training the neural network based on the loss function to output the expected animation point positions or expected animation shapes based on the input time points.

[0052] In this way, by using the loss function of the differential equation generated based on the motion physics simulation constraints of the animation point position or animation shape to train the neural network, and inputting the time point to be interpolated into the trained neural network, the neural network can output the animation point position or shape interpolation that conforms to the motion physics simulation constraints, making the interpolated animation smooth, natural and realistic.

[0053] Note that in this article, "animation point" can refer to an object in the animation, a point on an object, or a region on an object. If the object is rigid and its posture remains unchanged, then any animation point on the object can represent the object's position; determining the position of the animation point means determining the position of the object. In other cases, the object's center point can also be used as the animation point to determine the object's position. The animation point can also be any point on the object, regardless of the object's properties, posture, or movement, as long as the movement of the animation point can be represented by some curve, equation, or law.

[0054] In one embodiment, the motion physics simulation constraint is a constraint that can be defined by differential equations.

[0055] Motion physics simulation constraints typically include force-based motion physics simulation constraints conforming to Newtonian mechanics. Newtonian mechanics describes the relationship between force and motion, where force includes gravity and other forces. It applies not only to the motion of solids (rigid bodies) but also to the mechanics of systems of particles, fluids, vibrations, and waves. Motion physics simulation constraints include force-based motion physics simulation constraints related to Newtonian mechanics, such as oscillatory motion physics simulation constraints, force-based motion physics simulation constraints, acceleration or deceleration motion physics simulation constraints, etc. A motion physics simulation constraint that can be defined by differential equations means that the constraint can be expressed by differential equations. For example, under force-based motion physics simulation constraints of Newtonian mechanics, the relationship between the position or shape of an animated point or object and the derivative or differential of time, or the relationship between the force acting on an object and the derivative or differential of the object's velocity and position, can all be used as differential equations.

[0056] A differential equation is a relation between an unknown function and its derivative or differential. Differential equations can be further divided into ordinary differential equations (ODEs) and partial differential equations (PDEs). An ODE is a relation between a single unknown function and its derivative or differential, while a PDE is a relation between multiple unknown functions and their partial derivatives or partial differentials.

[0057] For example, in the vibrational motion of an object, the physical constraints of the object's motion, where the force acting on it decreases continuously with velocity and position, can be described by the following ordinary differential equation:

[0058]

[0059] Where y represents the object's position, t represents time, and α and β are parameters. Here, the variable y is an unknown function, for example, it could represent the object's position on the y-axis in a three-dimensional xyz coordinate system.

[0060] For example, in the vortex motion of an object, the physical constraints of the object's position on the x and y axes, which change with time due to the forces acting on it, can be described by the following partial differential equations:

[0061]

[0062] Here, x and y are two unknown functions, which can represent, for example, the positions on the x-axis and y-axis in a three-dimensional xyz-axis coordinate system, and together determine the position of the object. λ is a parameter.

[0063] The following is a schematic diagram illustrating the results of interpolation using prior art and various embodiments of the present application to demonstrate the fluency, naturalness, and realism of the interpolation results according to the various embodiments of the present application.

[0064] Figure 3AAn example diagram shows the result of interpolating between the start and end positions of an animation point according to a prior art method.

[0065] like Figure 3A As shown, the starting point of the animation point at time 0s or the point of a keyframe is shown in A, and the ending point of the animation point at time 2500s or another keyframe is shown in B. If interpolation is performed using, for example, a Bézier curve according to an existing interpolation scheme, the interpolation result may be as follows: Figure 3A The curve is shown. If we want the animated points to appear to be vibrating, we use some Bézier curves to simulate vibration.

[0066] The movement of the animated point displayed after such interpolation looks unnatural, as if the animated point is being dragged at a constant speed rather than like real vibration.

[0067] Figure 3B An example diagram is shown showing the result of interpolating between the start and end positions of an animation point, taking into account the physical constraints described above, according to an embodiment of this application.

[0068] In this example, a neural network trained according to an embodiment of this application is used to interpolate between the start and end positions of the animation point.

[0069] The trained neural network is trained as follows: differential equations are generated based on the motion physics simulation constraints on the position or shape of the animation point, which represent the relationship between the position or shape of the animation point and the time point, constrained by the motion physics simulation; a loss function for the neural network is generated based on the differential equations, wherein the loss function includes a loss associated with the differential equations; and the neural network is trained based on the loss function to output the expected position or shape of the animation point based on the input time point.

[0070] Here, motion physics simulation constraints can limit the relationship between the position of animation points and time points, such as the force vibration of an object, parabolic motion, etc., and can also limit the relationship between the shape of animation and time points, such as the elastic deformation of a ball, the shape change of a fluid, etc.

[0071] Differential equations include one or more of ordinary differential equations and partial differential equations, and differential equations may have one or more parameters or no parameters.

[0072] The following example, using the vibration constraint—the relationship between the position of an animation point and a time point—as a motion physics simulation constraint, describes the training process of a neural network.

[0073] First, as mentioned above, based on the motion physics simulation constraints of the animation point position (or animation shape), such as vibration motion constraints, the following ordinary differential equation is generated to represent the physical motion constraint that the force on the object will continuously decrease with velocity and position, similar to the scenario of an object fixed on a spring vibrating up and down with the spring:

[0074]

[0075] That is, the second derivative of y with respect to t is equal to -α multiplied by y, minus β multiplied by y with respect to t. Here, y represents the object's position, t represents time, and α and β are parameters. In this context, the variable y is an unknown function, for example, it could represent the object's position on the y-axis in a three-dimensional xyz coordinate system.

[0076] Secondly, a loss function for the neural network is generated based on the differential equation, wherein the loss function includes a loss associated with the differential equation. This loss associated with the differential equation can include the difference between the relationship between the expected animation point position (or expected animation shape) output by the neural network and the time point and the differential equation.

[0077] For example, the following loss can be used to represent the expected animation point position of the neural network output. With time point t i The relationship between the differential equations and the difference between the equations are used as the loss function of the neural network.

[0078]

[0079] The loss function described above is derived from Equation 1.

[0080] For example, a loss function can be designed to represent whether the expected animation point position or expected animation shape of the neural network output satisfies the differential equation with respect to the time point, thereby realizing a loss related to the differential equation, where the loss function is differentiable.

[0081] Formula 1 above can be transformed to obtain:

[0082]

[0083] In other words, if the expected animation point position output by the neural network satisfies Formula 4 above, that is, the position of the neural network output at time t... i Expected animation point position and time t i If substituting the left-hand side of Formula 4 above results in 0, it means that the animation point position output by the neural network satisfies the above differential equation, or in other words, it satisfies the physical motion law of force vibration. Conversely, if the neural network output at time t... i Expected animation point position and time ti Substituting the left-hand side of Formula 4 above, the less equal to 0 or the further away from 0 it is, whether positive or negative, indicates that the position of the animation point output by the neural network does not satisfy the differential equation, or in other words, does not satisfy the physical motion law of force vibration.

[0084] Therefore, the following loss function can be designed to represent whether the animation point position of the output of the neural network satisfies the differential equation, or satisfies the differential equation to a certain extent, or the degree to which the differential equation is satisfied.

[0085]

[0086] The reason for using the square root is to eliminate the influence of whether the calculation result within the square brackets is positive or negative, since the focus is on whether it equals 0, or how close or far it is from 0. Regardless of whether the calculation result within the square brackets is positive or negative, the further it is from 0, the larger the calculated result of the loss function in Formula 3; conversely, the closer it is to 0 or equal to 0, the smaller the calculated result of the loss function.

[0087] Backpropagation can be used to train a neural network to minimize the loss function or make it less than a preset threshold. The parameters α and β can also be determined during the training process. Therefore, designing the loss function to minimize it or make it less than the preset threshold allows the animation point positions output by the trained neural network to better satisfy the differential equation, or in other words, to satisfy the physical laws of force and vibration. This results in smoother, more natural, and more realistic animation interpolation, conforming to the laws of natural motion.

[0088] A neural network consists of multiple layers of neurons. The computation process of a neural network involves feeding the input vector to each neuron in the first layer, weighting and summing the inputs using their respective weights to obtain the activation level, and then applying an activation function to that level. These results are then fed to the next layer of neurons. This process continues until the last layer (the output layer) calculates the output vector of the neural network. The training process of a neural network involves forward propagation to obtain the loss function value, then backward propagation of the loss function value to update the weights of the neural network. Forward propagation calculates the weighted sum of the inputs to each neuron from the input layer to the hidden layers, and then from the hidden layers to the output layer to obtain the output of each neuron. Backpropagation calculates the total loss function value, and then calculates the loss function value for each neuron individually. The total loss function value is the sum of the error values ​​of the loss function for each neuron. Then, the weights are updated from the hidden layer to the output layer. To determine the impact of a particular weight on the overall loss function's error, the partial derivative of that weight with respect to the overall loss function's error can be used (i.e., using the chain rule—the derivative of a composite function is the product of the derivatives of the finite number of functions constituting the composite at the corresponding points). This reveals how the loss function's error propagates backward. Based on this error, for example, reducing it, the weight is updated. Then, the weights are updated from the hidden layer to the hidden layer. This process is repeated to update all weights. Finally, the updated weights are used for forward and backward propagation again, iterating continuously until the error of the loss function is reduced to a minimum, zero, or less than a preset threshold.

[0089] For example, the samples in the training set include input and output target values. During training, samples in the training set are submitted to the neural network one by one. The neural network calculates the output for each sample input, and then calculates the loss function between the sample target value and the neural network's output. For each sample submitted to the neural network, all weight values ​​are updated once, until the loss function value of all samples is minimized or less than a preset threshold, at which point training is complete.

[0090] This application does not impose any restrictions on the number of layers, the setting of hidden layers, the setting of activation functions, or the setting of parameters in the neural network, as long as the loss function designed according to the principles of this application is used.

[0091] Then, after the neural network is trained, the user inputs interpolation points (i.e., one or more time points at the locations of one or more animation points to be simulated, which can be uniform interpolation, i.e., the intervals between each time point are the same): T1, T2, T3, T4..., and the trained neural network will output the interpolation results: That is, the position of the object at these points in time.

[0092] like Figure 3B As shown, this is the interpolation curve of the animation points between the starting and ending points, output by the neural network trained as described above. It is evident that the result of this animation interpolation is smoother, more natural, and more realistic, conforming to the natural motion laws of vibration.

[0093] The above example uses the position of animation points, and the same principle applies to the shape of animation. That is, the change in the shape of an animation over time can be represented by the position of certain animation points at certain times. For example, the shape change of water spreading on a table can be constructed by the positions of the various animation points that make up the water moving outwards from the center of the water (in accordance with natural laws) over time. This change in position over time can also be expressed by one or more differential equations. Then, a loss function is designed based on these differential equations, such that the smaller the loss function, the more naturally the animation shape change conforms to natural laws. Specific examples will not be discussed further here.

[0094] Furthermore, sometimes when editing animations, users may want the animation to not only conform to realistic motion patterns but also to be in the positions or shapes they desire at certain key moments. In other words, users want to edit their own intentions into the animation.

[0095] For example, when an object vibrates, the user might want it to be in one or more specific positions at certain times, such as when the object hits another object or a leaf at a certain location, causing a bird on the leaf to fly away. Therefore, this application can also consider the user's editing intentions and incorporate them into the training of the neural network to obtain a neural network that considers both natural motion laws and the user's editing intentions, so that the output conforms to both natural motion laws and the user's editing intentions.

[0096] Figure 4A A flowchart illustrating a neural network training method for interpolating between the start and end positions of animation points, according to an embodiment of this application, further considers the user's editing or customization needs.

[0097] The main task of this scheme is to train a neural network that can perform interpolation, satisfying the laws of physical simulation. This neural network is denoted as: Once the neural network is trained, the user inputs the desired time points for interpolation: T1, T2, T3, T4..., and the network will output the interpolation results (e.g., the positions of the animation points). The following are the specific steps for neural network training: 410-440

[0098] Step 410: Express the physical constraints that conform to the laws of nature as differential equations: For example, in vibrational motion, the force on an object is related to the object's velocity and its position, and can be expressed as the following ordinary differential equation: The physical meaning of this equation is that the force acting on an object decreases continuously with velocity and position. This implementation scheme may include trainable parameters α and β, or it may not.

[0099] Step 420: Convert the user's editing intent into data points:

[0100] a. Users can set the start and end points of the curve according to their creative intentions:

[0101] (t start y start ), (t end y end )

[0102] b. Users can edit the data of keyframe points according to their creative intent:

[0103] (t1, y1) (t2, y2), (t3; y3),…

[0104] c. Users can select the interpolation points they want to interpolate in the interpolation region based on their creative intent (for example, they can choose uniform interpolation), that is, they can choose which time points or points to interpolate at:

[0105] T1, T2, T3, T4...

[0106] Step 430: Constructing the neural network training process

[0107] a. Write data points with user intent into the data error loss term of the loss function:

[0108]

[0109] b. Incorporate the physical simulation constraints into the physical constraint loss term of the loss function:

[0110]

[0111] c. Finally, the following loss function is obtained:

[0112]

[0113] Step 440: Train the neural network using the conventional backpropagation method.

[0114] Next, the trained neural network will predict the data at the user's desired interpolation time points T1, T2, T3, T4... to obtain the output results.

[0115] Figure 4B An example diagram is shown illustrating the result of interpolating between the start and end positions of an animation point, taking into account the user's editing or customization needs, according to an embodiment of this application.

[0116] like Figure 4B As shown, users can also specify certain key points as the positions of their desired animation points at certain times, for example... Figure 4B Points C, D, and E in the simulation, and even the starting point A and ending point B of animation points, can be edited or customized by the user. Suppose the user edits points C, D, and E; these three points are drawn by the user and represent the positions the user expects the animation points to be in at certain times, for example, to achieve a specific animation effect. In this case, if only physical motion simulation constraints are considered, the animation points might not be in these positions at those times.

[0117] Therefore, in order to ensure that the positions of animation points in the final animation effect both satisfy the physical motion simulation constraints as much as possible, and also meet the user's special editing or customization needs, this application further improves the loss function of the neural network to reflect the user's special editing or customization requirements.

[0118] Specifically, the system receives user-defined animation point positions or animation shapes for one or more second time points (e.g., T1, T2, T3, T4, etc.). The loss function also includes the difference between the expected animation point positions or expected animation shapes output by the neural network for one or more second time points and the defined animation point positions or animation shapes. The defined animation point positions or animation shapes may include at least one of the following: the start and end points of an animation curve, key points of an animation curve, the start shape of the animation, the end shape of the animation, and the key shape of the animation shape.

[0119] For example, Figure 4B As shown, it receives the start and end points set by the user: (t start y start ), (t end y end It receives the y-coordinate position y1 and the corresponding time point t1 of point C edited by the user, the y-coordinate position y2 and the corresponding time point t2 of point D edited by the user, and the y-coordinate position y3 and the corresponding time point t3 of point E edited by the user, for example (t1, y1), (t2, y2), (t3, y3), ...

[0120] In addition to the loss function term generated by the differential equation of the physical simulation constraint of the vibration mentioned above, the loss function of the neural network also includes a loss function term representing the difference between the expected animation point position or expected animation shape output by the neural network for one or more second time points and the set animation point position or set animation shape.

[0121] For example, the loss function term representing the user's editing intent can be... This means that at time point t i The output of the neural network at points t1, t2, and t3 (as mentioned above) (For example, ) and user-edited y i The difference between (e.g., y1, y2, y3 above) is also designed to eliminate the difference between positive and negative numbers obtained by subtracting the two, so that the larger the difference, the larger the calculated result of the loss function term, and the smaller the difference, the smaller the calculated result of the loss function term.

[0122] Therefore, the loss function L of a neural network can be designed as follows:

[0123]

[0124] Where n is the number of time points.

[0125] Backpropagation can be used to train a neural network to minimize the loss function. The parameters α and β can also be determined during the training process of the neural network. For example, ... Figure 4B As shown, the training yielded parameters α≈6.232 and β≈0.417. Therefore, designing the above loss function to minimize it or make it less than the preset threshold ensures that the animation point positions output by the trained neural network satisfy both the differential equation and the physical laws of force vibration, making the animation interpolation results smoother, more natural, and more realistic, conforming to the laws of natural motion, while also satisfying the user's editing or customization intentions.

[0126] Next, after obtaining the trained neural network, you can input the various time points you want to interpolate, such as T1, T2, T3, T4, etc. The trained neural network will then output the interpolation results: That is, the location data of objects at these points in time.

[0127] Here, the interpolation between user editing points C, D, and E is called interpolation, while the interpolation outside of user editing points C, D, and E, such as to the starting point A and the ending point B, is called extrapolation. Each time point T1, T2, T3, T4... can be located between user editing points C, D, and E, or outside of user editing points C, D, and E, such as to the starting point A and the ending point B.

[0128] In summary, this application can: 1) integrate physical simulation with user editing intent, ensuring that the interpolated effect matches the user's intention; 2) utilize neural networks as a tool for calculating interpolation data; 3) construct the following features using a neural network loss function: user editing intent is used as training data and fed into the loss function, while physical simulation constraints are also used as a constraint term in the loss function; 4) use the trained neural network to interpolate the animation curve. For users, only a few points need to be edited to design complex motion curves that conform to natural laws, such as... Figure 4B The interpolation curve shown exhibits a physical oscillation process. This allows interpolated points within the edit point to automatically satisfy the constraints of physical motion laws, eliminating the need for users to tediously add numerous edit points and automatically predicting the symmetrical oscillation position. Extrapolated points from the edit point to the start or end point also automatically satisfy the physical motion law constraints because they conform to the trained neural network, eliminating the need for users to worry about extrapolated points from the edit point not conforming to these constraints. If the differential equation representing the physical motion law constraints contains trainable parameters, this method can also automatically train specific parameter values ​​through neural network training.

[0129] The examples above illustrate various methods according to embodiments of this application for ordinary differential equations, i.e., those with only one variable y. However, this application is also applicable to partial differential equations.

[0130] Figure 5 An example diagram is shown illustrating the result of interpolation between the start and end positions of an animation point applied to a partial differential equation according to an embodiment of this application.

[0131] For example, as shown above, in the vortex motion of an object, the physical motion constraints of the object's position on the x and y axes, which change with time due to the forces acting on it, can be described by the following partial differential equations:

[0132]

[0133] Here, x and y are two unknown functions, which can represent, for example, the positions on the x-axis and y-axis in a three-dimensional xyz-axis coordinate system, and together determine the position of the object. λ is a parameter. The specific relationship between x and t can be seen as follows: Figure 5 As shown in the upper left, the specific relationship between y and t can be seen as follows: Figure 5 As shown in the upper right corner.

[0134] For example, the loss function term representing the physical motion simulation law can be:

[0135]

[0136] This aims to ensure that the movement of the positions on the x-axis and y-axis over time conforms to the laws of physical motion simulation, thereby minimizing or reducing the loss function term to a preset threshold.

[0137] For example, the loss function term representing the user's editing intent can still be...

[0138] Therefore, the entire loss function L can be designed as follows:

[0139]

[0140] While training the neural network to minimize the loss function L or make it less than the preset threshold, the parameter value λ≈-1.232 can also be obtained.

[0141] Thus, when using the trained neural network to interpolate at each time point t, the interpolated motion trajectory of that animation point can be obtained as follows: Figure 5 As shown in the lower right corner.

[0142] The above example illustrates a partial differential equation with two unknown functions. However, the loss function term can be constructed in a similar manner for partial differential equations with more than two unknowns. These will not be elaborated upon here.

[0143] Of course, the physical simulation motion forms and differential equations exemplified above are merely examples. In reality, other physical simulation motion forms and differential equations can also be applied in this application as loss function terms to solve the animation point interpolation problem for various motion forms.

[0144] Furthermore, the construction of the loss function term mentioned above, such as using a quadratic mathematical expression, is merely an example. In reality, any form of loss function term that conforms to the laws of nature can be obtained by minimizing the loss function term, such as using a cubic expression. Not all feasible cases will be listed here.

[0145] Figure 6 A block diagram is shown of an animation point position or animation shape simulation device 600 based on motion physics simulation constraints using a neural network, according to an embodiment of this application.

[0146] The device 600 includes: a receiver 610 configured to receive one or more time points of one or more animation point positions or animation shapes to be simulated; and a simulator 620 configured to output the expected animation point positions or animation shapes based on one or more time points using a trained neural network. The trained neural network is trained by: generating differential equations based on motion physics simulation constraints on the animation point positions or animation shapes, the differential equations representing the relationship between the animation point positions or animation shapes and time points, constrained by the motion physics simulation constraints; generating a loss function for the neural network based on the differential equations, wherein the loss function includes a loss associated with the differential equations; and training the neural network based on the loss function to output the expected animation point positions or expected animation shapes based on the input time points.

[0147] In this way, by using the loss function of the differential equation generated based on the motion physics simulation constraints of the animation point position or animation shape to train the neural network, and inputting the time point to be interpolated into the trained neural network, the neural network can output the animation point position or shape interpolation that conforms to the motion physics simulation constraints, making the interpolated animation smooth, natural and realistic.

[0148] In one embodiment, the motion physics simulation constraint is a constraint that can be defined by differential equations.

[0149] Motion physics simulation constraints typically include force-based motion physics simulation constraints conforming to Newtonian mechanics. Newtonian mechanics describes the relationship between force and motion, where force includes gravity and other forces. It applies not only to the motion of solids (rigid bodies) but also to the mechanics of systems of particles, fluids, vibrations, and waves. Motion physics simulation constraints include force-based motion physics simulation constraints related to Newtonian mechanics, such as oscillatory motion physics simulation constraints, force-based motion physics simulation constraints, acceleration or deceleration motion physics simulation constraints, etc. A motion physics simulation constraint that can be defined by differential equations means that the constraint can be expressed by differential equations. For example, under force-based motion physics simulation constraints of Newtonian mechanics, the relationship between the position or shape of an animated point or object and the derivative or differential of time, or the relationship between the force acting on an object and the derivative or differential of the object's velocity and position, can all be used as differential equations.

[0150] A differential equation is a relation between an unknown function and its derivative or differential. Differential equations can be further divided into ordinary differential equations (ODEs) and partial differential equations (PDEs). An ODE is a relation between a single unknown function and its derivative or differential, while a PDE is a relation between multiple unknown functions and their partial derivatives or partial differentials.

[0151] In one embodiment, the loss associated with the differential equation includes the difference between the relationship between the expected animation point location or expected animation shape of the neural network output and the time point and the differential equation.

[0152] In one embodiment, a loss related to a differential equation is achieved by designing a loss function to represent whether the expected animation point position or expected animation shape output by the neural network satisfies a differential equation with respect to a time point, wherein the loss function is differentiable.

[0153] Thus, the animation interpolation result obtained by the neural network trained using the above loss function is smoother, more natural, and more realistic, conforming to the natural motion law of vibration.

[0154] Furthermore, sometimes when editing animations, users may want the animation to not only conform to realistic motion patterns but also to be in the positions or shapes they desire at certain key moments. In other words, users want to edit their own intentions into the animation.

[0155] In one embodiment, the system receives user-defined animation point positions or animation shapes for one or more second time points, and the loss function further includes the difference between the expected animation point positions or animation shapes output by the neural network for one or more second time points and the set animation point positions or animation shapes.

[0156] In one embodiment, setting the animation point position or setting the animation shape includes at least one of the following: the start and end points of the animation curve, the key points of the animation curve, the start shape of the animation, the end shape of the animation, and the key shape of the animation shape.

[0157] Thus, by designing the above loss function to minimize or be less than the preset threshold, the animation point positions output by the trained neural network can satisfy both the differential equation and the physical motion law of force vibration, making the animation interpolation results smoother, more natural, and more realistic, conforming to the laws of natural motion, and also satisfying the user's editing or customization intentions.

[0158] In one embodiment, the differential equation includes one or more of ordinary differential equations and partial differential equations, with one or more parameters or without parameters.

[0159] In one embodiment, the backpropagation method is used to train the neural network so that the loss function is minimized or less than a preset threshold.

[0160] In summary, this application can: 1) integrate physical simulation with user editing intent, ensuring that the interpolated effect conforms to the user's intention; 2) utilize neural networks as a tool for calculating interpolation data; 3) construct the following features using a neural network loss function: user editing intent is used as training data and fed into the loss function, while physical simulation constraints are also used as a constraint term in the loss function; 4) use the trained neural network to interpolate the animation curve. For users, only a few points need to be edited to design complex motion curves that conform to natural laws. The interpolated points within the edited points automatically satisfy the constraints of physical motion laws, eliminating the need for users to tediously add many edited points, and automatically predicting the symmetrical oscillation position. The extrapolated points from the edited points to the starting or ending points also automatically satisfy the constraints of physical motion laws because they conform to the trained neural network, eliminating the need for users to worry about extrapolated points not conforming to the constraints. If the differential equation representing the constraints of physical motion laws contains trainable parameters, this method can also automatically train specific parameter values ​​through neural network training.

[0161] Figure 7 A block diagram of an exemplary computer system suitable for implementing embodiments of the present application is shown.

[0162] The computer system may include a processor (H1); and a memory (H2) coupled to the processor (H1) and storing computer-executable instructions therein for performing the steps of the various methods of the embodiments of this application when executed by the processor.

[0163] The processor (H1) may include, but is not limited to, one or more processors or microprocessors.

[0164] The memory (H2) may include, but is not limited to, random access memory (RAM), read-only memory (ROM), flash memory, EPROM memory, EEPROM memory, registers, computer storage media (e.g., hard disk, floppy disk, solid-state drive, removable disk, CD-ROM, DVD-ROM, Blu-ray disc, etc.).

[0165] In addition, the computer system may also include a data bus (H3), an input / output (I / O) bus (H4), a display (H5), and input / output devices (H6) (e.g., keyboard, mouse, speakers, etc.).

[0166] The processor (H1) can communicate with external devices (H5, H6, etc.) via the I / O bus (H4) through a wired or wireless network (not shown).

[0167] The memory (H2) may also store at least one computer-executable instruction for performing steps of various functions and / or methods in the embodiments described in this technology when executed by the processor (H1).

[0168] In one embodiment, the at least one computer-executable instruction may also be compiled into or comprise a software product, wherein one or more computer-executable instructions are executed by a processor to perform the steps of the various functions and / or methods in the embodiments described herein.

[0169] Figure 8 A schematic diagram of a non-transitory computer-readable storage medium according to an embodiment of the present disclosure is shown.

[0170] like Figure 8 As shown, the computer-readable storage medium 820 stores instructions, such as computer-readable instruction 810. When the computer-readable instruction 810 is executed by a processor, the various methods described above can be performed. The computer-readable storage medium includes, but is not limited to, volatile memory and / or non-volatile memory. Volatile memory may include, for example, random access memory (RAM) and / or cache memory. Non-volatile memory may include, for example, read-only memory (ROM), hard disk, flash memory, etc. For example, the computer-readable storage medium 820 can be connected to a computing device such as a computer, and then, when the computing device executes the computer-readable instruction 810 stored on the computer-readable storage medium 820, the various methods described above can be performed.

[0171] In summary, this application can: 1) integrate physical simulation with user editing intent, ensuring that the interpolated effect conforms to the user's intention; 2) utilize neural networks as a tool for calculating interpolation data; 3) construct the following features using a neural network loss function: user editing intent is used as training data and fed into the loss function, while physical simulation constraints are also used as a constraint term in the loss function; 4) use the trained neural network to interpolate the animation curve. For users, only a few points need to be edited to design complex motion curves that conform to natural laws. The interpolated points within the edited points automatically satisfy the constraints of physical motion laws, eliminating the need for users to tediously add many edited points, and automatically predicting the symmetrical oscillation position. The extrapolated points from the edited points to the starting or ending points also automatically satisfy the constraints of physical motion laws because they conform to the trained neural network, eliminating the need for users to worry about extrapolated points not conforming to the constraints. If the differential equation representing the constraints of physical motion laws contains trainable parameters, this method can also automatically train specific parameter values ​​through neural network training.

[0172] Of course, the specific embodiments described above are merely examples and not limitations. Those skilled in the art can combine and integrate some steps and devices from the various embodiments described separately above to achieve the effects of this application based on the concept of this application. Such combined and integrated embodiments are also included in this application, but will not be described one by one here.

[0173] Note that the advantages, benefits, and effects mentioned in this disclosure are merely examples and not limitations, and should not be considered as essential features of the various embodiments of this application. Furthermore, the specific details disclosed above are for illustrative and facilitative purposes only, and are not limitations; these details do not restrict this application from being implemented using the aforementioned specific details.

[0174] The block diagrams of devices, apparatuses, devices, and systems disclosed herein are merely illustrative examples and are not intended to require or imply that they must be connected, arranged, or configured in the manner shown in the block diagrams. As those skilled in the art will recognize, these devices, apparatuses, devices, and systems can be connected, arranged, and configured in any manner. Words such as “comprising,” “including,” “having,” etc., are open-ended terms meaning “including but not limited to,” and are used interchangeably with them. The terms “or” and “and” as used herein refer to the terms “and / or,” and are used interchangeably with them unless the context clearly indicates otherwise. The term “such as” as used herein refers to the phrase “such as but not limited to,” and is used interchangeably with it.

[0175] The flowcharts and method descriptions in this disclosure are merely illustrative examples and are not intended to require or imply that the steps of the various embodiments must be performed in the given order. As those skilled in the art will recognize, the steps in the above embodiments can be performed in any order. Words such as "then," "next," etc., are not intended to limit the order of the steps; these words are only used to guide the reader through the description of these methods. Furthermore, any reference to a singular element, such as the use of the articles "a," "one," or "the," is not to be construed as limiting that element to the singular.

[0176] Furthermore, the steps and apparatus in the various embodiments herein are not limited to any one embodiment. In fact, new embodiments can be conceived by combining relevant steps and apparatus in the various embodiments herein based on the concepts of this application, and these new embodiments are also included within the scope of this application.

[0177] The various operations of the methods described above can be performed by any suitable means capable of performing the corresponding functions. Such means may include various hardware and / or software components and / or modules, including but not limited to hardware circuits, application-specific integrated circuits (ASICs), or processors.

[0178] The various exemplified logic blocks, modules, and circuits described herein can be implemented or performed using a general-purpose processor, digital signal processor (DSP), ASIC, field-programmable gate array (FPGA) or other programmable logic device (PLD), discrete gate or transistor logic, discrete hardware components, or any combination thereof. The general-purpose processor can be a microprocessor, but alternatively, it can be any commercially available processor, controller, microcontroller, or state machine. The processor can also be implemented as a combination of computing devices, such as a combination of a DSP and a microprocessor, multiple microprocessors, a microprocessor cooperating with a DSP core, or any other such configuration.

[0179] The steps of the methods or algorithms described in this disclosure can be directly embedded in hardware, in a software module executed by a processor, or a combination of both. The software module can reside in any form of tangible storage medium. Some examples of storage media that can be used include random access memory (RAM), read-only memory (ROM), flash memory, EPROM memory, EEPROM memory, registers, hard disks, removable disks, CD-ROMs, etc. The storage medium can be coupled to the processor so that the processor can read information from and write information to the storage medium. Alternatively, the storage medium can be integral with the processor. The software module can be a single instruction or many instructions, and can be distributed across several different code segments, different programs, and across multiple storage media.

[0180] The methods disclosed herein include actions for implementing the described methods. The methods and / or actions may be interchanged without departing from the scope of the claims. In other words, unless a specific order of actions is specified, the order and / or use of specific actions may be modified without departing from the scope of the claims.

[0181] The above functions can be implemented in hardware, software, firmware, or any combination thereof. If implemented in software, the functions can be stored as instructions on a tangible computer-readable medium. The storage medium can be any available tangible medium that can be accessed by a computer. By way of example and not limitation, such a computer-readable medium can include RAM, ROM, EEPROM, CD-ROM or other optical disc storage, magnetic disk storage or other magnetic storage devices, or any other tangible medium that can be used to carry or store desired program code in the form of instructions or data structures and that can be accessed by a computer. As used herein, disk and disc include compact disc (CD), laser disc, optical disc, digital universal disc (DVD), floppy disk, and Blu-ray disc, wherein a disc typically magnetically reproduces data, while a disc optically reproduces data using lasers.

[0182] Therefore, a computer program product can perform the operations given herein. For example, such a computer program product can be a computer-readable tangible medium having instructions tangibly stored (and / or encoded) thereon, which can be executed by a processor to perform the operations described herein. The computer program product may include packaging materials.

[0183] Software or instructions can also be transmitted via a transmission medium. For example, software can be transmitted from a website, server, or other remote source using transmission media such as coaxial cable, fiber optic cable, twisted pair, digital subscriber line (DSL), or wireless technologies such as infrared, radio, or microwave.

[0184] Furthermore, modules and / or other suitable means for carrying out the methods and techniques described herein can be downloaded and / or obtained by user terminals and / or base stations as appropriate. For example, such a device can be coupled to a server to facilitate the transmission of means for carrying out the methods described herein. Alternatively, the various methods described herein can be provided via storage components (e.g., RAM, ROM, physical storage media such as CDs or floppy disks) so that user terminals and / or base stations can obtain the various methods when coupled to the device or when storage components are provided to the device. Furthermore, any other suitable techniques for providing the methods and techniques described herein to the device can be utilized.

[0185] Other examples and implementations are within the scope and spirit of this disclosure and the appended claims. For example, due to the nature of software, the functions described above can be implemented using software executed by a processor, hardware, firmware, hardwired, or any combination thereof. Features implementing the functions can also be physically located in various places, including being distributed so that parts of the functions are implemented at different physical locations. Moreover, as used herein, including as used in the claims, the "or" used in a list of items beginning with "at least one" indicates a separate list, such that a list of, for example, "at least one of A, B, or C" means A or B or C, or AB or AC or BC, or ABC (i.e., A and B and C). Furthermore, the word "exemplary" does not mean that the described examples are preferred or better than other examples.

[0186] Various changes, substitutions, and modifications can be made to the technology described herein without departing from the teachings defined by the appended claims. Furthermore, the scope of the claims of this disclosure is not limited to the specific aspects of the processes, machines, manufactures, events, means, methods, and actions described above. Currently existing or later-developed processes, machines, manufactures, events, means, methods, or actions that perform substantially the same function or achieve substantially the same result as the corresponding aspects described herein can be utilized. Therefore, the appended claims include such processes, machines, manufactures, events, means, methods, or actions within their scope.

[0187] The above description of the disclosed aspects is provided to enable any person skilled in the art to make or use this application. Various modifications to these aspects will be readily apparent to those skilled in the art, and the general principles defined herein can be applied to other aspects without departing from the scope of this application. Therefore, this application is not intended to be limited to the aspects shown herein, but rather to be accorded the widest scope consistent with the principles and novel features disclosed herein.

[0188] The above description has been given for purposes of illustration and description. Furthermore, this description is not intended to limit the embodiments of this application to the forms disclosed herein. Although numerous exemplary aspects and embodiments have been discussed above, those skilled in the art will recognize certain variations, modifications, alterations, additions, and sub-combinations thereof.

Claims

1. A method for simulating an animation point position or an animation shape based on motion physics constraints using a neural network, comprising: receiving one or more time points of one or more animation point positions or animation shapes of a desired simulation; outputting, by a trained neural network, an expected animation point position or an expected animation shape based on the one or more time points, wherein the trained neural network is trained by: generating a differential equation based on motion physics constraints on the animation point position or the animation shape, the differential equation representing a relationship between the animation point position or the animation shape and the time points subject to the motion physics constraints; generating a loss function of the neural network based on the differential equation, wherein the loss function comprises a loss related to the differential equation; training the neural network to output the expected animation point position or the expected animation shape based on an input time point based on the loss function, wherein receiving a set animation point position or a set animation shape for one or more second time points set by a user, the loss function further comprises a loss function term representing a difference between the expected animation point position or the expected animation shape output by the neural network and the set animation point position or the set animation shape for the one or more second time points, the loss function term representing an editing intention of the user.

2. The method of claim 1, wherein, the motion physics constraints are constraints that can be defined by a differential equation.

3. The method of claim 2, wherein, the motion physics constraints comprise force-based motion physics constraints related to Newtonian mechanics.

4. The method of claim 1, wherein, the loss related to the differential equation comprises a difference between the relationship between the expected animation point position or the expected animation shape output by the neural network and the time points and the differential equation.

5. The method of claim 4, wherein, the loss related to the differential equation is achieved by designing a loss function to represent whether the expected animation point position or the expected animation shape output by the neural network satisfies the differential equation, wherein the loss function is differentiable.

6. The method of claim 1, wherein, the set animation point position or the set animation shape comprises at least one of a start point and an end point of an animation curve, a key point of an animation curve, a start shape of an animation, an end shape of an animation, a key shape of an animation shape.

7. The method of claim 1, wherein, the differential equation comprises one or more of an ordinary differential equation and a partial differential equation, the differential equation with one or more parameters or without parameters.

8. The method of any one of claims 1-7, wherein, the neural network is trained using a backpropagation method to minimize the loss function or to be less than a preset threshold.

9. An apparatus for simulating an animation point position or an animation shape based on motion physics constraints using a neural network, comprising: a receiver configured to receive one or more time points of one or more animation point positions or animation shapes of a desired simulation; a simulator configured to output, by a trained neural network, an expected animation point position or an expected animation shape based on the one or more time points, wherein the trained neural network is trained by: generating a differential equation based on motion physics constraints on the animation point position or the animation shape, the differential equation representing a relationship between the animation point position or the animation shape and the time points subject to the motion physics constraints; generate a loss function of the neural network based on the differential equation, wherein the loss function comprises a loss related to the differential equation; train the neural network based on the loss function to output an expected animation point position or an expected animation shape based on an input time point, wherein a user sets a set animation point position or a set animation shape for one or more second time points, and the loss function further comprises a loss function term representing a difference between the expected animation point position or the expected animation shape output by the neural network and the set animation point position or the set animation shape for the one or more second time points, and the loss function term represents a user editing intention.

10. The apparatus of claim 9, wherein, The motion physics simulation constraint is a constraint that can be defined by a differential equation.

11. The apparatus of claim 10, wherein, The motion physics simulation constraint comprises a force-based motion physics simulation constraint related to Newtonian mechanics.

12. The apparatus of claim 9, wherein, The loss related to the differential equation comprises a difference between a relationship between the expected animation point position or the expected animation shape output by the neural network and the time point and the differential equation.

13. The apparatus of claim 12, wherein, The loss related to the differential equation is achieved by designing a loss function to represent whether the expected animation point position or the expected animation shape output by the neural network satisfies the differential equation, wherein the loss function is differentiable.

14. The apparatus of claim 9, wherein, The set animation point position or the set animation shape comprises at least one of a start point and an end point of an animation curve, a key point of an animation curve, a start shape of an animation, an end shape of an animation, and a key shape of an animation shape.

15. The apparatus of claim 9, wherein, The differential equation comprises one or more of an ordinary differential equation and a partial differential equation, with one or more parameters or without parameters.

16. The apparatus of any one of claims 9-15, wherein, The neural network is trained by using a back propagation method to minimize the loss function or to be less than a preset threshold.

17. An animation point position or animation shape simulation system based on a motion physics simulation constraint using a neural network, comprising: a processor; a memory storing one or more computer executable instructions which, when executed by the processor, perform the method of any one of claims 1-8.

18. A non-transitory computer readable medium storing one or more computer executable instructions which, when executed by a processor, perform the method of any one of claims 1-8.

Images