Program, motion control method, and motion control device

Abstract

To eliminate the unnaturalness that occurs when transitioning between motions that move between two points with different elevations. [Solution] A program according to one embodiment of the present invention causes the computer to perform the following steps: the root position setting unit 120 acquires root position information indicating the height of the root setting position which serves as a reference when switching the motion of an object in the virtual space; based on the root position information, it acquires motion data from a motion database which defines the motion including vertical movement of the object; and if it is determined that one of the multiple assumed ground contact points of the object in the motion data is in contact with the ground, it sets the root position based on the ground contact point where the ground contact point on which the object's center of gravity is located is in contact with the terrain information.

Description

Technical Field

[0001] The present invention relates to a program, a motion control method, and a motion control device.

Background Art

[0002] Conventionally, motion matching is known as a technique for naturally moving a virtual character in a virtual space. Motion matching is an exhaustive motion synthesis method that searches an enormous animation database for animation clips such as a character's walking or running that are most suitable for the current situation. Motion matching is an easy-to-understand method for obtaining an appropriate animation clip that meets the motion conditions, and the found motion can be blended into the animation being executed.

[0003] For example, Patent Document 1 discloses a game system including a sub-feature quantity extraction unit that extracts a predetermined type of sub-feature quantity corresponding to the position or orientation of a virtual object included in a game state generated by the game system from feature quantities related to the virtual object included in the game state generated by the game system, and a motion sequence determination unit that determines one motion sequence from a motion database by comparing each of the predetermined types of sub-feature quantities extracted by the sub-feature quantity extraction unit with each of the predetermined types of sub-feature quantities of each motion sequence stored in the motion database. According to the technique described in Patent Document 1, it is possible to realize a more flexible motion matching that can cope with various contexts.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0005] Incidentally, in conventional motion matching, the height of the root point (hereinafter also referred to as the "root position") of the object before and after the motion switch is matched before the motion is switched. This allows multiple motions to be played seamlessly in sequence, resulting in natural animation.

[0006] The root position is often set to the height of the object's waist. However, when the root position is set to waist height, a phenomenon called "slipping" can occur when moving between two points with a difference in elevation. Slipping is a phenomenon in which the object's motion does not match the terrain, resulting in unnatural animation.

[0007] Furthermore, the visual unnaturalness in animation resulting from the adoption of motion matching technology has traditionally been addressed by using animation blending techniques. However, the visual unnaturalness that occurs when an object with a root position set at waist height moves through areas with clear differences in height, such as stairs or rocky terrain, cannot always be resolved even with animation blending techniques.

[0008] The object of this invention has been made in view of the above circumstances, and is to eliminate the unnaturalness that occurs in the connection between motions moving between two points with a difference in elevation. [Means for solving the problem]

[0009] The present invention provides a program to cause a computer to execute the following steps: a root position setting unit acquires root position information indicating the height of the root setting position which serves as a reference when switching the motion of an object in a virtual space; a motion database storing motion data that defines the motion including vertical movement of an object, based on the root position information; a procedure to determine whether one of the multiple assumed ground contact points of the object in the motion data is in contact with the terrain information of the scene in which the object is played; and, if it is determined that one of the multiple ground contact points of the object is in contact with the terrain information, a procedure to set the root position based on the ground contact position where the ground contact point on which the object's center of gravity is located is in contact with the terrain information. [Effects of the Invention]

[0010] According to the present invention, it is possible to eliminate unnatural movements that occur in the transition between motions when an object moves between two points with a difference in elevation. Other issues, configurations, and effects not mentioned above will be clarified by the following description of the embodiments. [Brief explanation of the drawing]

[0011] [Figure 1] This is an overall configuration diagram showing an overview of a motion control system according to one embodiment of the present invention. [Figure 2] This block diagram shows examples of the hardware configurations of a motion control server and a terminal device that constitute a motion control system according to one embodiment of the present invention. [Figure 3] This is a block diagram showing an example of the functional configuration of a motion control server according to one embodiment of the present invention. [Figure 4] This figure shows an overview of static root position correction by the root position setting unit according to one embodiment of the present invention. [Figure 5]It is a diagram showing a configuration example of a motion DB according to an embodiment of the present invention. [Figure 6] It is a diagram showing an example of setting a route to an object after the execution of static route position correction according to an embodiment of the present invention. [Figure 7] It is a diagram showing an example of the hierarchical structure of each joint of an object when the concept of a conventional route is not used. [Figure 8] It is a diagram showing an example of the hierarchical structure of each joint of an object when a conventional route is used. [Figure 9] It is a diagram showing an example of a route position that changes based on a conventional foot contact. [Figure 10] It is a diagram showing a correspondence example between the coordinate position of a route indicated by a conventional global coordinate, the coordinate position of a lumbar vertebra indicated by a global coordinate, and the coordinate position of a lumbar vertebra indicated by a relative coordinate from the coordinate position of the route. [Figure 11] It is a diagram showing an outline of a coordinate system change process showing the coordinate position of a lumbar vertebra before and after the execution of inertial blending by a motion correction unit according to an embodiment of the present invention. [Figure 12] It is a diagram showing an outline of mutual feedback by static route position correction and dynamic motion correction according to an embodiment of the present invention. [Figure 13] It is a flowchart showing an example of a motion control process procedure by a motion control system according to an embodiment of the present invention. [Figure 14] It is a flowchart showing an example of a procedure of static route position correction processing according to an embodiment of the present invention. [Figure 15] It is a flowchart showing an example of a procedure of dynamic motion correction processing according to an embodiment of the present invention. [Figure 16] It is a flowchart showing an example of a procedure of correction processing for the position of a foot according to an embodiment of the present invention.

Modes for Carrying Out the Invention

[0012] Hereinafter, embodiments for implementing the present invention will be described with reference to the accompanying drawings. In this specification and the drawings, for components having substantially the same function or configuration, the same reference numerals are given and duplicate descriptions are omitted.

[0013] [One Embodiment] <Overview of Motion Control System> FIG. 1 is an overall configuration diagram showing an overview of a motion control system 100 according to an embodiment of the present invention. The motion control system 100 analyzes a motion clip during game development and determines whether the motion of an object in the motion clip to be analyzed is a motion that moves between two points with a height difference. When it is determined that the motion is a motion that moves between two points with a height difference, the motion control system 100 performs correction of the route position and correction of the motion. A motion clip refers to a bundle of three-dimensional coordinates of about 15 seconds to 90 seconds obtained by recording an animation by an object. In the following description, the data of the motion clip is also referred to as motion data.

[0014] An object is a general term for display objects displayed in a virtual space in a game. The object includes, for example, a two-dimensional image or a three-dimensional image of a game medium, an image of a UI (User Interface) such as a button, etc. In the present embodiment, a case where the object is a game character having a part such as a foot (an example of a grounding-assumed part) whose installation with the ground in the virtual section is assumed will be described as an example. Details of the correction of the route position and the correction of the motion by the motion control system 100 will be described later.

[0015] Also, the motion control system 100 searches the motion DB (Database) 151 (see FIG. 2) using the corrected motion as a query. Then, the motion control system 100 blends the corrected motion and the motion in the search result.

[0016] As shown in Figure 1, the motion control system 100 includes a motion control server 1, a smartphone 2A, and a PC (Personal Computer) 2B.

[0017] Motion control server 1 is an example of a motion control device and performs the correction of the root position and motion described above. Smartphone 2A and PC 2B can connect to motion control server 1 via a network N such as the internet. In the following description, smartphone 2A and PC 2B will be collectively referred to as terminal device 2. A game-dedicated terminal may also be used as terminal device 2.

[0018] Terminal device 2 sends instructions to motion control server 1 to search for motion matching based on user instructions, etc., and displays motion as a result of motion matching sent from motion control server.

[0019] The input device 23 of terminal device 2 generates an operation signal corresponding to the operation input by the user and supplies the operation signal to the control unit 20 (see Figure 2). The display device 24 displays motion and other data as search results for motion matching based on control by the control unit 20.

[0020] <Example of hardware configuration for a motion control system> Next, with reference to Figure 2, an example of the hardware configuration of a motion control system 100 according to one embodiment of the present invention will be described. Figure 2 is a block diagram showing an example of the hardware configuration of the motion control server 1 and terminal device 2 that constitute the motion control system 100.

[0021] (Example configuration of a motion control server) Motion control server 1 is an example of a computer that operates as a computer. This motion control server 1 includes a control unit 10, a non-volatile storage 11, and a network interface 12, each connected to bus B1.

[0022] The control unit 10 includes a CPU (Central Processing Unit) 101, a GPU (Graphics Processing Unit) 102, a ROM (Read Only Memory) 103, and a RAM (Random Access Memory) 104. The control unit 10 may also include an MPU (Micro-Processing Unit) instead of a CPU, and may be configured as an FPGA (Field Programmable Gate Array).

[0023] The CPU 101 reads the program code of the software that implements each function according to this embodiment from the ROM 103, loads it into the RAM 104, and executes it. Variables and parameters that occur during the calculation process of the CPU 101 are temporarily written to the RAM 104. These variables and parameters written to the RAM 104 are read out by the CPU 101 as appropriate. GPU102 performs the calculations necessary for rendering images.

[0024] Examples of non-volatile storage 11 include HDDs (Hard Disk Drives), SSDs (Solid State Drives), Blu-ray® discs, flexible disks, optical discs, magneto-optical discs, CD-ROMs, CD-Rs, magnetic tapes, or non-volatile memory. This non-volatile storage 11 stores the OS (Operating System), various parameters, and a program for operating the motion control server 1. The program for operating the motion control server 1 may also be stored in ROM 103. In other words, ROM 103 and the non-volatile storage 11 are used as examples of computer-readable, non-transient recording media that store programs executed by the motion control server 1.

[0025] For example, a NIC (Network Interface Card) can be used for the network interface 12. The network interface 12 transmits and receives various data to and from the terminal device 2 via the network N through a dedicated line connected to the terminals of the NIC.

[0026] (Example of terminal device configuration) Terminal device 2 is an example of a computer that operates as a computer capable of executing various programs. This terminal device 2 includes a control unit 20, a network interface 21, an input / output interface 22, an input device 23, a display device 24, and a non-volatile storage 25, all connected to bus B2.

[0027] The control unit 20 includes a CPU 201, a GPU 202, a ROM 203, and a RAM 204. The control unit 20 may also be provided with an MPU instead of the CPU 201, and the control unit 20 may be configured as an FPGA.

[0028] The CPU 201 reads the program code of the software that implements each function according to this embodiment from the ROM 203, loads it into the RAM 204, and executes it. Variables and parameters that occur during the calculation process of the CPU 201 are temporarily written to the RAM 204. These variables and parameters written to the RAM 204 are read out by the CPU 201 as needed. The CPU 201 performs processing such as OS processing in the terminal device 2 and management of data input and output performed in various parts of the terminal device 2. GPU202 performs the calculations necessary for rendering images.

[0029] For example, a network interface (NIC) can be used for the network interface 21. The network interface 21 can acquire various data from the motion control server 1 and communicate with other terminal devices 2 via a LAN, dedicated line, etc., connected to the terminals of the NIC.

[0030] The input / output interface 22 converts the operation signals received from the input device 23 into data in a predetermined format and passes them to the CPU 201. The input / output interface 22 also passes images such as motions output from the control unit 20 to the display device 24. The display device 24 may be equipped with a speaker (not shown), in which case the input / output interface 22 can also output audio signals to the speaker.

[0031] In the case of a smartphone 2A (see Figure 1), the input device 23 consists of touch sensors on a touch panel, etc., while in the case of a PC 2B, it consists of a mouse, keyboard, etc.

[0032] The display device 24 is composed of an LCD (Liquid Crystal Display) or an organic EL (Electro Luminescence) display, etc.

[0033] Examples of non-volatile storage 25 include HDDs, SSDs, Blu-ray discs, flexible disks, optical discs, magneto-optical discs, CD-ROMs, CD-Rs, or non-volatile memory. This non-volatile storage 25 stores the OS, various parameters, and programs necessary for the operation of the terminal device 2.

[0034] The program for operating the motion control server 1 may be stored in the ROM 203. In other words, the ROM 203 and the non-volatile storage 25 are used as examples of computer-readable, non-transient recording media that store programs executed by the terminal device 2.

[0035] If the non-volatile storage 25 is an external recording medium consisting of an optical disc or the like, an optical drive (not shown) reads the non-volatile storage 25 and retrieves the data. The data retrieved by the optical drive is used for predetermined arithmetic processing by the CPU 201.

[0036] <Functional Configuration of Motion Control Server> Next, with reference to Figure 3, an example of the functional configuration of the motion control server 1 that constitutes the motion control system 100 will be described. Figure 3 is a block diagram showing an example of the functional configuration of the motion control server 1.

[0037] As shown in Figure 3, the motion control server 1 includes a communication unit 110, a route position setting unit 120, a motion search unit 130, a motion correction unit 140, and a storage unit 150.

[0038] The communication unit 110 controls communication with the terminal device 2 (see Figure 1). For example, when the communication unit 110 receives an analysis instruction or the like transmitted from the terminal device 2, it transmits an instruction to the route position setting unit 120 or the motion correction unit 140. Also, when the communication unit 110 receives a motion search instruction or the like transmitted from the terminal device 2, it transmits an instruction to the motion search unit 130. Furthermore, when the communication unit 110 receives a search result from the motion search unit 130, it transmits the received search result to the terminal device 2. The functions of the communication unit 110 are realized by the network interface 12 shown in Figure 2, etc.

[0039] The root position setting unit 120 corrects the root position of an object when the object moves between two points with a difference in elevation. Initially, the root position of an object is set to the height of the object's hip bone (waist). For motion data in which one of the object's feet is in contact with the ground, the root position setting unit 120 corrects the root position to the contact point (ground surface) of the foot on which the center of gravity is located.

[0040] The foot on which the center of gravity is located is, for example, the foot whose coordinate position is lower among the feet that are in contact with the ground, or the foot that is perceived to have a greater weight by the weight sensor used in motion capture to create motion data. Note that the sensor used in motion capture is not limited to a weight sensor, but may also be a contact sensor, proximity sensor, or other type of sensor.

[0041] Information about the contact points between the object's feet and the terrain is managed in the motion database 151, where the motion data is stored, and is associated with each frame that makes up the motion data. The configuration of the motion database 151 will be described in detail later with reference to Figure 5.

[0042] Furthermore, for motion data in which none of the object's feet are touching the ground, the root position setting unit 120 performs a correction, setting the root position of the most recent past frame in which the object's feet were touching the ground to the root position of the motion. In the following description, the above-mentioned root position correction by the motion control system 100 according to this embodiment will also be referred to as "static root position correction".

[0043] Here, with reference to Figure 4, an overview of static root position correction by the root position setting unit 120 will be explained. Figure 4 is a diagram illustrating an overview of static root position correction by the root position setting unit 120.

[0044] Figure 4 shows an animated character, Object Ob, walking across a rocky area in a virtual space. In Figure 4, Object Ob is climbing a ledge made up of rocks, with its weight shifted to its right foot, that is, the foot on the upper ledge. For motion data of such Object Ob, the route position setting unit 120 sets the route position at the point where the right foot makes contact with the terrain (rock).

[0045] If the distance between the root position set for an object and the terrain surface differs significantly between the transition and the motion, a "slipping" phenomenon occurs in the motion. The difference in the distance between the root position and the terrain surface between the transition and the motion can be large in motions such as climbing or descending stairs, but it becomes more pronounced in rocky areas where the height of each rock is uneven, as shown in Figure 4. As a result, the movement of an object in an animation that plays such a motion will appear unnatural to the game player. In this embodiment, the root position setting unit 120 performs static root position correction, which reduces the difference in the distance between the root position and the terrain surface between the transition and the motion.

[0046] Returning to Figure 3, the explanation continues. At the timing of motion switching, the motion search unit 130 searches for the destination motion using motion matching, with the source motion as a query. Motion matching by the motion search unit 130 is performed on the motion DB 151, which reflects the contents of the static root position correction. The motion search unit 130 outputs the motion obtained by motion matching to the motion correction unit 140.

[0047] The motion correction unit 140 blends the motion input to the motion search unit 130 as a query with the motion obtained as a search result by the motion search unit 130. For blending techniques, for example, Inertial Blend can be used. Inertial Blend is a method that instantly switches the animation motion from the source to the destination, leaving only the inertia of the movement when the source motion was being played.

[0048] Furthermore, the motion correction unit 140 performs corrections on the motion data after inertial blending based on the correspondence between the root position set for the object and the terrain information. Terrain information is environmental information, including terrain information within the virtual space of the game, managed in the game engine Ge (see Figure 12), etc. The motion corrected by the motion correction unit 140 according to the terrain information and root position will also be referred to as "dynamic motion correction" in the following description. Details of the dynamic motion correction by the motion correction unit 140 will be described later.

[0049] The memory unit 150 stores the motion database 151. The functions of the memory unit 150 are realized by the non-volatile storage 11 shown in Figure 2.

[0050] <Example of MotionDB configuration> Next, the configuration of the motion DB151 will be described with reference to Figure 5. Figure 5 is a diagram showing an example of the configuration of the motion DB151. Motion DB151 manages motion data consisting of several frames, associating it with a motion ID. Figure 5 shows only the items included in Motion DB151. Note that the number of frames constituting the motion data can be two or more, and the upper limit of the number of frames is not limited to a few frames.

[0051] As shown in Figure 5, the motion DB151 contains the following items: "motion ID", "motion data", "route trajectory information", "object joint information", "foot grounding / non-grounding information", and "information about the next frame in which grounding occurs".

[0052] The Motion ID field stores information about the Motion ID, which serves as a unique identifier for identifying motion data. The motion data section manages motion data where frame numbers are associated with the data. The frame number is a frame identifier assigned to each frame that makes up the motion data.

[0053] Each item following the route trajectory information corresponds to each frame of the motion data. The "Route Trajectory Information" section stores trajectory (path) information for each route in the "Past," "Present," and "Future." The trajectory information for each route in the past and future is shown using relative coordinates to the current trajectory's coordinate position.

[0054] The "Object Joint Information" section stores the position information of each joint that makes up the object, as well as the velocity information when the joint moves. The joint position information is shown in relative coordinates with respect to the current root coordinate position, which is indicated by global coordinates.

[0055] The "Foot Grounding / Non-Grounding Information" field stores information about whether the current object's right and left feet are grounded or not. The "Information about the next frame in which ground contact occurs" field stores the position information of the terrain directly beneath the toes when the object's feet touch the ground in the next frame in which ground contact occurs. This terrain position information is shown in relative coordinates to the root coordinates.

[0056] The item "Information regarding the next frame in which ground contact occurs" stores the position information of the terrain directly beneath the toes of the foot making contact, and the coordinate position information of the toes and heel of the foot making contact. The "position information of the terrain directly beneath the toes of the foot making contact" is information that the motion search unit 130 (see Figure 2) refers to when selecting a footstep motion that matches the terrain in the actual scene in which the motion is played back. When the motion search unit 130 performs a search, it compares the position information shown in "position information of the terrain directly beneath the toes of the foot making contact" with the terrain information in the actual scene.

[0057] The coordinate position information of the toes and heels of the grounding foot is indicated by relative coordinates to the coordinate position of the route. The coordinate position information of the toes and heels of the grounding foot is referenced by the motion correction unit 140 when it determines whether the area around the heel or toe of the object is embedded in the terrain. When performing a search, the motion correction unit 140 compares the coordinate position information indicated in "coordinate position information of the toes and heels of the grounding foot" with the terrain information in the actual scene.

[0058] Note that the items included in the motion DB151 shown in Figure 5 are just an example, and the motion DB151 of the present invention may include items other than those shown in Figure 5. For example, the motion DB151 may include "the number of frames until a new ground contact occurs," "information on which foot (right or left) makes contact when a new ground contact occurs," and "the coordinates of the toe and heel from the current frame to N future frames." "The coordinates of the toe and heel from the current frame to N future frames" means that the coordinates of the toe and heel are expected to be drawn a number of frames in the future up to N, with the current frame as the starting point in time. By including these items in the motion DB151, the motion search unit 130 can select a motion that does not collide with the terrain.

[0059] <Dynamic Motion Correction> Next, we will explain the dynamic motion correction performed by the motion correction unit 140. Here, correction refers to calculating a correction amount to bridge the gap with the ideal position using a calculation formula such as linear interpolation, and setting the new coordinate position corrected based on the calculated correction amount to the object in the motion. Specifically, the motion correction unit 140 performs six corrections, numbered from the 1st to the 6th, as dynamic motion correction.

[0060] (First amendment) As the first correction, the motion correction unit 140 performs inertia blending as a correction to smooth the connections between motions. In performing inertia blending, the motion correction unit 140 converts the coordinate system indicating the coordinate position of the object's hip bone (waist) from relative coordinates from the root coordinate position indicated by global coordinates to global coordinates. Then, the motion correction unit 140 performs inertia blending on the hip bone indicated by global coordinates and other joints (parts) indicated by relative coordinates from the coordinate position of the hip bone. After performing inertia blending, the motion correction unit 140 reverse-converts the coordinate system indicating the coordinate position of the object's hip bone from global coordinates to relative coordinates from the root coordinate position (indicated by global coordinates).

[0061] The reason why such processing is necessary will now be explained with reference to Figures 6 to 8. Figure 6 shows an example of setting a route to an object after static root position correction has been performed.

[0062] As shown in Figure 6, after static root position correction is performed, the root Rt is set to the point of contact with the foot on which the object Ob's center of gravity rests (the right foot in the figure). The horizontal coordinate position of the root is set to the same position as the hip bone Bh, and the height of the root Rt, i.e., the root position, is set to the height of the foot's contact point, i.e., the position on the ground.

[0063] Figure 7 shows an example of the hierarchical structure of each joint of object Ob without using the concept of root Rt, and Figure 8 shows an example of the hierarchical structure of each joint of object Ob when root Rt is used.

[0064] In the case where the concept of root Rt is not used, as shown in Figure 7, the coordinate position of the hip bone of object Ob is shown in global coordinates, and the right and left hip joints, which are connected to the lower layer of the hip bone, are shown in relative coordinates from the coordinate position of the hip bone. Each of these joints corresponds to a child when the hip bone is the parent. Similarly, the coordinate positions of the right and left knees, which correspond to grandchildren when the hip bone is the parent, are shown in relative coordinates from the coordinate positions of the right and left hip joints, respectively. In this embodiment as well, for motion data of motions that do not involve vertical movement by object Ob, the hierarchical structure shown in Figure 7, in which the coordinate position of the hip bone of object Ob is shown in global coordinates, is adopted.

[0065] On the other hand, when the concept of root Rt is used, as shown in Figure 8, the coordinate position of root Rt, i.e., the coordinate position of the grounding point, is shown in global coordinates, and the coordinate position of the hip bone Bh, which is a child of root Rt and connected to the lower layer of root Rt, is shown in relative coordinates from the coordinate position of root Rt. The coordinate positions of each joint in the lower layer of hip bone Bh are the same as in the example shown in Figure 7, so the explanation is omitted. In this embodiment, the root position setting unit 120 performs static root position correction, so the hierarchical structure of each joint of object Ob becomes the example shown in Figure 8.

[0066] Figure 9 shows an example of a root position that changes based on foot contact. In the diagram on the left of Figure 9, the foot of object Ob is in contact with the lower step St1 at contact point Pg1, and the root Rt is set at this contact point Pg1. The coordinate position of the hip bone Bh is then set using the relative coordinates from the coordinate position of the root Rt. In other words, the root position Rp is the height from the lower step St1 to the hip bone Bh.

[0067] On the other hand, in the diagram shown on the right side of Figure 9, the foot of object Ob is in contact with the ground point Pg2 of the higher step St2 in the step difference St, and the root Rt is set at this ground point Pg2. Then, the coordinate position of the hip bone Bh is set using the relative coordinate position from the coordinate position of root Rt. In other words, the root position Rp is the height from the higher step St2 to the hip bone Bh.

[0068] Due to static root position correction, the position of root Rt is set to the point where the object Ob's center of gravity rests on the ground. As a result, each time the object Ob ascends or descends a step St, the height of root Rt, and thus the root position, changes significantly, as shown in the left and right diagrams of Figure 9.

[0069] Figure 10 shows an example of the correspondence between the coordinate position of root Rt, expressed in global coordinates, the coordinate position of hip bone Bh, expressed in global coordinates, and the coordinate position of hip bone Bh, expressed in relative coordinates from the coordinate position of root Rt. In Figure 10, the vertical axis represents height at the coordinate position, and the horizontal axis represents time.

[0070] Time t1 on the time axis in Figure 10 represents the time when the foot is in contact with the lower step St1 of the step difference St, as shown on the left side of Figure 9. Time t2 on the time axis in Figure 10 represents the time when the foot is in contact with the higher step St2 of the step difference St, as shown on the right side of Figure 9. As shown in the top row of Figure 10, the coordinate position (height) of the root Rt, shown in global coordinates, changes significantly when switching from time t1 to time t2. The coordinate position of the hip bone Bh, shown in relative coordinates from the coordinate position of the root Rt, also changes significantly, as shown in the bottom row of Figure 10. On the other hand, the coordinate position of the hip bone Bh, shown in global coordinates, as shown in the middle row of Figure 10, changes gradually when switching from time t1 to time t2, and no abrupt change in height is observed.

[0071] Figure 11 is a diagram illustrating the overview of the coordinate system modification process performed by the motion correction unit 140, which shows the coordinate position of the hip bone Rt before and after inertia blending. Figure 11A shows an example of the hierarchical structure of each joint of the object after static root positioning is performed. Figure 11B shows an example of the hierarchical structure of each joint of the object before inertia blending is performed. Figure 11C shows an example of the hierarchical structure of each joint of the object after inertia blending is performed.

[0072] After static root positioning is performed, as shown in Figure 11A, the coordinate position of the top-level root is shown in global coordinates, and the coordinate positions of the joints in each lower level below the hip bone are shown in relative coordinates from the coordinate positions of the joints in the higher level. Before inertia blending is performed, as shown in Figure 11B, the motion correction unit 140 converts the coordinate system of the hierarchical structure of each joint of the object to a coordinate system that shows the coordinate position of the hip bone in global coordinates.

[0073] By performing this processing by the motion correction unit 140 before inertial blending is executed, it becomes possible to minimize fluctuations in the coordinate positions of each joint in the hierarchy below the hip bone that are handled in inertial blending.

[0074] After inertia blending is performed, the motion correction unit 140 reverse-transforms the coordinate system of the hierarchical structure of each joint of the object to a coordinate system that indicates the root coordinate position using global coordinates, as shown in Figure 11C. The motion correction unit 140 then returns each joint in the hierarchy below the hip bone to below the root. The motion correction unit 140 also excludes objects at the root coordinate position from the inertia blending process. This is because the coordinate position of the hip bone was originally indicated by relative coordinates from the root coordinate position, and therefore the positional relationship between the root and the hip bone is considered not to change before and after the coordinate system transformation. Consequently, the motion correction unit 140 uses the objects at the root coordinate position as they are without performing inertia blending. The motion correction unit 140 may also perform processing such as recreating the objects at the root coordinate position after the transformation and reverse transformation of the coordinate system of the hierarchical structure of each joint of the object.

[0075] (Second amendment) As a second correction, the motion correction unit 140 corrects the position of the root when a collision occurs between the object and the no-entry object, but only in the lateral (horizontal) direction. In performing the second correction, the motion correction unit 140 pre-calculates the position of the object's feet from the time of motion detection to the frame in which the next ground contact occurs. Note that the motion correction unit 140 only corrects the position of the root in the lateral direction, and corrections in the height direction are performed in subsequent corrections.

[0076] (Third amendment) The motion correction unit 140 calculates the correction amount necessary for the sixth correction, which will be described later, as the third correction. The sixth correction is a correction to eliminate the discrepancy between the foot position and the terrain. In the third correction, the motion correction unit 140 calculates the correction amount for the sixth correction, which is used when an unnatural feeling occurs, such as the foot sinking into the side of stairs or rocks while not in contact with the ground, and shifts the foot position to a position above the surface of the terrain. Specifically, in the third correction, the motion correction unit 140 first calculates and determines whether or not the foot will sink into the terrain in the next ground contact frame when a motion switch occurs. If it determines that sinking will occur, the motion correction unit 140 calculates an offset value to the position where the sinking is eliminated. Then, the motion correction unit 140 uses technology such as SpringDamper to shift the foot position a little each frame and calculates the amount of shift (offset value) for each frame. The motion correction unit 140 calculates the amount of shift for a range of, for example, several frames to several tens of frames.

[0077] (Fourth amendment) The motion correction unit 140 calculates the correction amount required for the sixth correction as the fourth correction. More specifically, in the fourth correction, the motion correction unit 140 calculates the correction amount required for the sixth correction, which is to move the foot to a desirable position when the foot sinks in or overhangs at the next ground contact timing on stairs, etc. The calculation of the correction amount in the fourth correction is also performed in the same way as in the third correction. In other words, an offset value to a desirable position where the misalignment can be eliminated is calculated for each frame.

[0078] (Fifth amendment) As a fifth correction, the motion correction unit 140 corrects the root position to match the point of contact between the foot and the terrain in the actual scene in which the motion is played. If either of the object's feet is in a grounded state in the motion, the root position is set to the point of contact between the foot bearing the center of gravity and the terrain. Therefore, in the fifth correction, the motion correction unit 140 corrects the root position to the point of contact with the terrain in the actual scene.

[0079] (Sixth amendment) The motion correction unit 140 performs a sixth correction over multiple frames (frame by frame) to eliminate the discrepancy between the foot position and the terrain, based on the correction amounts calculated in the third and fourth corrections. The motion correction unit 140 can perform the foot position correction using techniques such as IK (Inverse Kinematics). IK is a method in which the position and posture of a specific part are specified, and the posture of all intermediate joints leading up to that point is determined by calculation.

[0080] More specifically, the motion correction unit 140 first applies the correction amounts calculated by the third and fourth corrections to the toes of each of the object's feet to correct the position of the feet. Then, if the feet, after correction based on the correction amounts, are in a ground-contact state and sinking into the terrain is confirmed, the unit uses a technique such as foot locking to correct the position of the feet that are embedded below the surface of the terrain, raising them above the terrain. Foot locking is a technique that fixes the position of the feet at the point of contact with the terrain from the time the feet make contact until they leave the terrain, and adjusts the object's pose using inverse kinematics so that the feet remain in the fixed position.

[0081] By performing the first to sixth dynamic motion corrections described above, it becomes possible to reproduce movements such as objects ascending and descending stairs or actions on rocky terrain as extremely natural animations without any visual inconsistencies.

[0082] Furthermore, the new motion synthesized by the dynamic motion correction described above will be used as a query when the motion search unit 130 performs the next motion matching. Therefore, according to this embodiment, the correction process can indirectly return feedback to the search process.

[0083] Furthermore, since the root position is selected (set) both during dynamic motion correction and when generating queries for motion matching, this embodiment creates mutual feedback between query generation and dynamic motion correction. As motion search and dynamic motion correction repeatedly provide mutual feedback, it becomes possible to converge the motion into a natural motion without any visual inconsistencies in an extremely short time, such as a few frames to tens of frames.

[0084] <Mutual feedback through static root position correction and dynamic motion correction> Next, with reference to Figure 12, the mutual feedback between static root position correction and dynamic motion correction by the motion control system 100 according to this embodiment will be described. Figure 12 is a diagram showing an overview of the mutual feedback between static root position correction and dynamic motion correction.

[0085] In this embodiment, static root position correction is performed by the root position setting unit 120 (step S1), so that the root position is set at the point where the foot on which the center of gravity is placed touches the ground for motion data in the motion DB 151 that includes the vertical movement of an object. Then, the motion search unit 130 performs motion search (motion matching) on ​​the motion DB 151 that has been statically corrected by the root position setting unit 120 (step S2). Therefore, in the motion search results, motions that are similar to the motion of an object that performs vertical movement are ranked higher in the search results. As a result, the motion correction unit 140 that synthesizes motions using the search results can synthesize motions that actually fit the terrain with a small-scale correction process.

[0086] The motion search unit 130 passes the motion obtained as a result of motion matching to the motion correction unit 140 (step S3). Next, the motion correction unit 140 performs dynamic motion correction based on the correspondence between the root position set for the object in the motion data and the terrain information obtained from the game engine Ge (step S4). Finally, the motion search unit 130, having received the result of the dynamic motion correction from the motion correction unit 140, saves the result to the motion DB 151.

[0087] The motion correction performed by the motion correction unit 140 corrects the motion according to the terrain (terrain information), so terrain information is embedded in the query. In other words, the dynamic motion correction in step S4 makes it possible to import terrain information that changes according to the game content into the motion DB 151 when the game is executed.

[0088] In this way, when the motion search unit 130 searches for route information, it uses dynamic terrain information within the game as a query, enabling it to search for desirable motion data (motion clips) that result in natural motion with less correction distance. In other words, according to this embodiment, since static route position setting and dynamic motion correction are continuously fed back over several to tens of frames, highly accurate motion matching can be achieved.

[0089] <Procedure for motion control processing> Next, a motion control method using the motion control system 100 according to this embodiment will be described. Figure 13 is a flowchart showing an example of the procedure for motion control processing by the motion control system 100. First, the root position setting unit 120 of the motion control server 1 of the motion control system 100 performs static root position correction processing (step S11). The procedure for the static root position correction processing in step S11 will be described in detail with reference to Figure 14.

[0090] Next, the motion search unit 130 performs motion matching (search for the target motion) on the motion DB 151, which reflects the contents of the static root position correction process performed in step S11 (step S12).

[0091] Next, the motion correction unit 140 of the motion control server 1 performs dynamic motion correction processing on the motion data obtained as a result of the motion matching in step S12 (step S13). The procedure for the dynamic motion correction processing in step S13 will be described in detail later with reference to Figure 15.

[0092] [Static root position correction process] Next, with reference to Figure 14, the static root position correction process performed by the root position setting unit 120 in step S11 of Figure 13 will be described. Figure 14 is a flowchart showing an example of the procedure for the static root position correction process.

[0093] First, the root position setting unit 120 determines whether or not one of the object's feet is touching the ground in the motion data to be analyzed (step S21). The root position setting unit 120 can determine whether or not one of the object's feet is touching the ground based on the information stored in the motion DB 151 (see Figure 5).

[0094] If it is determined in step S21 that one of the object's feet is touching the ground (step S21 is YES), the root position setting unit 120 sets the height of the contact point of the foot on which the center of gravity is located as the root position of the object (step S22). On the other hand, if it is determined in step S21 that neither of the object's feet is touching the ground (step S21 is NO), the root position setting unit 120 sets the root position in the most recent (near past) frame in which one of the feet was touching the ground as the root position of the object in that motion (step S23). After processing in step S22 or step S23, the static root position correction processing performed by the root position setting unit 120 is completed. In this embodiment, an example is given in which the root position setting unit 120 adopts the root position in the most recent frame in which one of the feet was touching the ground as the root position to be set for the object, but the present invention is not limited to this. The root position setting unit 120 may adopt any root position that is included from the end of the previous motion to the most recent frame.

[0095] The static route position correction process performed by the route position setting unit 120 reflects the corrected route position information in the motion DB 151. As a result, when the motion search unit 130 performs motion matching, motions that are closer to the ideal motion of an object performing vertical movement are output as the top-ranked search motions. In other words, unlike the conventional method of setting the route at the position of the hip bone, it becomes possible to obtain motions in motion matching searches that require little to no height correction. Therefore, according to this embodiment, it becomes possible to synthesize motions that actually fit the terrain without performing large-scale correction processing.

[0096] [Dynamic motion correction processing] Next, with reference to Figure 15, the dynamic motion correction process performed by the motion correction unit 140 in step S13 of Figure 13 will be described. Figure 15 is a flowchart showing an example of the procedure for the dynamic motion correction process.

[0097] First, the motion correction unit 140 converts the coordinate system of the hip bone coordinate position of the object in the motion data as a result of motion matching from relative coordinates from the root coordinate position to global coordinates (step S31). Next, the motion correction unit 140 performs inertia blending on the hip bone and each part connected to the lower layer of the hip bone, as indicated by the global coordinates (step S32).

[0098] Next, the motion correction unit 140 reverse-transforms the coordinate system of the object's hip bone coordinate position after inertial blending from global coordinates to relative coordinates from the root coordinate position (step S33). Next, the motion correction unit 140 calculates the position of the object's feet from the motion search point to the frame in which the next ground contact occurs (step S34). Next, the motion correction unit 140 compares the calculated foot position with the terrain in the actual scene in which the motion is played (step S35).

[0099] Next, the motion correction unit 140 determines whether or not a collision occurs between the object and the no-entry object (step S36). The information on the no-entry object is defined in the terrain information, which includes terrain information. If it is determined in step S36 that a collision occurs between the object and the no-entry object (step S36 is determined to be YES), the motion correction unit 140 corrects the position of the route set for the object only in the lateral (horizontal) direction (second correction) (step S37).

[0100] On the other hand, if it is determined in step S36 that no collision will occur (step S36 is NO), or after processing in step S37, the motion correction unit 140 calculates the amount of correction for each of the multiple frames to eliminate the discrepancy between the terrain and the foot position that will occur at the next ground contact timing (third and fourth corrections) (step S38).

[0101] Next, the motion correction unit 140 performs a correction (referred to as root height correction processing) to match the root position set for the object to the point of contact with the terrain in the actual scene where the motion is played (fifth correction) (step S39). Next, the motion correction unit 140 performs a correction process for the foot position based on the correction amount calculated in step S38 (sixth correction) (step S40). The foot position correction process in step S40 will be described in detail with reference to Figure 16 below.

[0102] [Foot position correction process] Next, the foot position correction process performed in step S40 of Figure 15 will be explained with reference to Figure 16. Figure 16 is a flowchart showing an example of the foot position correction process.

[0103] First, the motion correction unit 140 applies a correction based on the correction amount calculated in step S38 of Figure 15 to the toes of each of the object's feet (step S51). Next, the motion correction unit 140 determines whether the feet are touching the ground after the correction has been applied (step S52). If it is determined in step S52 that the feet are touching the ground (step S52 is YES), the motion correction unit 140 performs foot locking processing over multiple frames, targeting the feet that are touching the ground (step S53).

[0104] Next, the motion correction unit 140 determines whether the position of the object's toes is below the ground (surface of the terrain) (step S54). If it is determined in step S54 that the toes are below the surface of the terrain (step S54 is YES), the motion correction unit 140 performs a correction to raise the position of the toes above the surface of the terrain over multiple frames, based on the correction amount calculated in step S38 of Figure 15 (step S55). If step S52 is determined to be NO, if step S54 is determined to be NO, or after the processing in step S55, the motion correction unit 140 finishes correcting the position of the feet.

[0105] In the embodiment described above, the root position setting unit 120 acquires root position information indicating the height of the root setting position which serves as the reference when switching the motion of an object in the virtual space. Based on the root position information, it acquires motion data from the motion database that defines the motion including the vertical movement of the object. The root position setting unit 120 also determines whether any of the multiple assumed ground contact points of the object in the motion data are in contact with the terrain information of the scene in which the object is played. If it is determined that any of the multiple ground contact points of the object are in contact with the terrain, the root position setting unit 120 sets the root position based on the ground contact position where the ground contact point where the object's center of gravity is located is in contact with the terrain information. Therefore, according to this embodiment, the difference in distance between the root position and the terrain surface in the motion before and after switching can be reduced. In other words, according to this embodiment, the unnaturalness that occurs in the connection between motions in which an object moves between two points with a difference in elevation can be eliminated, so that movements such as climbing stairs or the behavior of an object on a rocky area can be played back as animation very naturally.

[0106] Furthermore, in the above-described embodiment, the root position setting unit 120 performs static analysis of motion data as a process during development (when constructing the motion DB 151) and corrects the root position, thereby improving the quality of the animation at runtime. This reduces the computational cost incurred when executing motion matching. Therefore, according to this embodiment, the quality of the animation can be improved with a computational cost at runtime comparable to that of conventional motion matching.

[0107] Furthermore, in this embodiment described above, the motion correction unit 140 dynamically corrects highly versatile motion data when motion matching is performed, so that motion according to the terrain is synthesized and an animation is generated. Therefore, according to this embodiment, it is not necessary to shoot motion according to the terrain each time, so the cost of motion shooting can be greatly reduced.

[0108] Furthermore, in the above-described embodiment, natural movement of the object in animation can be achieved through static correction of the root position and dynamic correction of the root position during motion matching. In other words, the motion control processing according to this embodiment does not depend on the gender, equipment, or personality of the object (character) and the individuality or style of the motion. Therefore, according to this embodiment, the motion that has undergone motion control processing can be integrated seamlessly with various effects and styles.

[0109] Furthermore, in the embodiment described above, since the motion DB151, which has undergone static root position correction, is the target of motion search, the motion control processing according to this embodiment can be seamlessly integrated with the technology for automatically generating motion data using a generation AI.

[0110] In the embodiments described above, the assumed ground contact area of ​​the object was the feet, but the present invention is not limited to this. The assumed ground contact area of ​​the object may be a part other than the feet, such as the hands of a quadruped character or a non-human character such as an alien.

[0111] It should be noted that the present invention is not limited to the embodiments described above, and various other applications and modifications can be taken as long as they do not deviate from the gist of the present invention as described in the claims. For example, the embodiments described above are intended to explain the system and apparatus configuration in detail and specifically in order to make the present invention easier to understand, and are not necessarily limited to having all the configurations described. Furthermore, it is possible to add, delete, or replace some of the configurations in these embodiments with other configurations. Furthermore, the control lines and information lines shown are those deemed necessary for explanatory purposes, and not all control lines and information lines are necessarily shown in the actual product. In reality, it is safe to assume that almost all components are interconnected. [Explanation of Symbols]

[0112] 1…Motion control server, 100…Motion control system, 120…Route position setting unit, 130…Motion search unit, 140…Motion correction unit, 151…Motion DB

Claims

1. The root position setting unit acquires root position information indicating the height of the root setting position which serves as the reference when switching the motion of an object in the virtual space, and based on the root position information, acquires motion data that defines the motion, including vertical movement of the object, from a motion database which stores motion data including coordinate positions associated with each of the multiple parts of the object defined by each motion of the object, and A procedure for determining the contact between multiple assumed contact points of the object in the motion data and the terrain information of the scene in which the object is played back, If it is determined that one of the multiple ground contact points of the object is in contact with the ground, the procedure for setting the root position is based on the ground contact position where the ground contact point on which the object's center of gravity is located is in contact with the terrain information, A program designed to be executed by a computer.

2. If the root position setting unit determines that none of the object's ground contact points are in contact with the terrain information, it sets the root position for the motion data to the root position set in the most recent frame in which any of the object's ground contact points were in contact. The program according to claim 1.

3. The assumed ground contact area where the center of gravity is located is the assumed ground contact area where the coordinate position is lower, or the assumed ground contact area where the sensor used for motion capture to create the motion data detects a greater weight. The program according to claim 2.

4. When playing back the motion of the aforementioned object, the motion search unit outputs the motion data obtained from the motion database as a query for the motion data in which the root position has been set, and motion data obtained by correcting the acquired motion data. The program according to claim 3.

5. If the motion data output from the motion search unit represents the ground contact of any of the assumed ground contact points of the object, the motion correction unit corrects the root position to the height of the ground contact point where the terrain information and the assumed ground contact point in the scene in which the animation is played make contact. The program according to claim 4.

6. The motion correction unit performs the correction of the root position over several frames. The program according to claim 5.

7. The motion correction unit performs inertia blending as a correction to smooth the connections between motions reproduced based on the motion data output from the motion search unit. The program according to claim 6.

8. Each part of the aforementioned object consists of a top-level node indicated by global coordinates, and lower-level nodes connected to the layers below the top-level node. The motion correction unit, before executing the inertia blend, changes the top-level node from the root to the waist, performs the inertia blend on the waist and each part of the lower-level nodes of the waist, which are indicated by the global coordinates, and after executing the inertia blend, changes the top-level node back from the waist to the root, and converts the coordinate system of the waist's coordinate position from the global coordinates to coordinates relative to the root's coordinate position. The program according to claim 7.

9. The motion correction unit calculates the position of the assumed ground contact point of the object from the time the motion search unit searches the motion data until the next frame in which the assumed ground contact point of the object occurs, and performs a correction over multiple frames to eliminate the discrepancy between the calculated position of the assumed ground contact point and the terrain information. The program according to claim 8.

10. If the motion correction unit determines that a collision occurs between the calculated position of the assumed ground contact point and the no-entry object in the scene, it corrects the position of the assumed ground contact point to a position where no collision occurs in the horizontal direction. The program according to claim 9.

11. If the motion correction unit determines that the calculated position of the assumed ground contact point is embedded in the side surface of the terrain which has height, it performs a correction to shift the position of the assumed ground contact point to a position above the top surface of the terrain. The program according to claim 10.

12. The motion correction unit applies the calculation result of the correction to eliminate the misalignment to the toe of the assumed ground contact area. If it determines that the toe is in a ground contact state after applying the calculation result, it performs a foot locking process to lock the position of the assumed ground contact area in a predetermined position until the ground contact state ends. If it determines that the position of the toe is below the terrain after applying the calculation result, it performs a process to raise the position of the toe above the terrain. The program according to claim 11.

13. The root position setting unit acquires root position information indicating the height of the root setting position which serves as the reference when switching the motion of an object in the virtual space, and based on the root position information, acquires motion data that defines the motion, including vertical movement of the object, from a motion database which stores motion data including coordinate positions associated with each of the multiple parts of the object defined by each motion of the object, and A procedure for determining the contact between multiple assumed contact points of the object in the motion data and the terrain information of the scene in which the object is played back, The procedure includes, if it is determined that one of the multiple ground contact points of the object is in contact with the ground, setting the root position based on the ground contact point where the ground contact point on which the object's center of gravity is located is in contact with the terrain information. Motion control method.

14. The system includes a root position setting unit that acquires root position information indicating the height of the root setting position which serves as a reference when switching the motion of an object in a virtual space; retrieves motion data that defines the motion, including the vertical movement of the object, from a motion database that stores motion data including coordinate positions associated with each of the multiple parts of the object, as defined by each motion of the object, based on the root position information; determines whether the multiple assumed ground contact points of the object in the motion data are in contact with the terrain information of the scene in which the object is played; and if it is determined that any of the multiple ground contact points of the object are in contact, sets the root position based on the ground contact position where the ground contact point where the center of gravity of the object is located is in contact with the terrain information. Motion control device.

Images