Information processing device, information processing method, and program

Abstract

To more naturally connect a plurality of pieces of motion data.SOLUTION: An information processing device includes: an acquisition unit that acquires interpolation motion data that interpolates between first motion data and second motion data which are temporally and spatially independent; and a calculation unit that calculates an index relating to a motion speed of each piece of motion data, based on the first motion data, the second motion data, and the interpolation motion data acquired by the acquisition unit.SELECTED DRAWING: Figure 3

Description

[Technical field]

[0001] The present disclosure relates to an information processing device, an information processing method, and a program. [Background technology]

[0002] In recent years, animation production and distribution using motion capture to obtain motion information indicating a user's motion has become popular. For example, motion data that imitates a user's motion is generated using the motion information obtained by motion capture, and an avatar video based on the motion data is distributed.

[0003] Against this background, the amount of motion data is increasing year by year, and techniques for reusing previously generated motion data are being developed. For example, Patent Document 1 discloses a technique for blending the movements of multiple pieces of motion data and reproducing the movements obtained by blending in real time using an avatar or the like in a virtual space. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] International Publication No. 2019 / 203190 Summary of the Invention [Problem to be solved by the invention]

[0005] When connecting multiple pieces of motion data, there are cases where interpolation editing is required to join the multiple pieces of motion data more naturally. When performing such interpolation editing, parameters such as boundary conditions may be specified in detail, but it may be difficult for the user to set all of the parameters.

[0006] Therefore, the present disclosure proposes a new and improved information processing device, information processing method, and program that are capable of connecting multiple pieces of motion data more naturally. [Means for solving the problem]

[0007] According to the present disclosure, an information processing device is provided that includes an acquisition unit that acquires interpolated motion data that interpolates between first motion data and second motion data that are independent in time and space, and a calculation unit that calculates an index related to the motion speed of each motion data based on the first motion data, the second motion data, and the interpolated motion data acquired by the acquisition unit.

[0008] In addition, according to the present disclosure, there is provided an information processing method executed by a computer, the method including obtaining interpolated motion data that interpolates between first and second motion data that are independent in time and space, and calculating an index related to the motion speed of each motion data based on the first motion data, the second motion data, and the interpolated motion data.

[0009] In addition, according to the present disclosure, a program is provided that causes a computer to realize an acquisition function for acquiring interpolated motion data that interpolates between first motion data and second motion data that are independent in time and space, and a calculation function for calculating an index related to the motion speed of each motion data based on the first motion data, the second motion data, and the interpolated motion data acquired by the acquisition function. [Brief description of the drawings]

[0010] [Figure 1] FIG. 1 is an explanatory diagram illustrating an information processing system according to an embodiment of the present disclosure. [Diagram 2] 2 is an explanatory diagram for explaining an example of a functional configuration of a server 10 according to the present disclosure. FIG. [Diagram 3] 2 is an explanatory diagram for explaining an example of a functional configuration of a PC 20 according to the present disclosure. FIG. [Figure 4] FIG. 11 is an explanatory diagram for explaining an example of a linking process of motion data. [Diagram 5] FIG. 11 is an explanatory diagram for explaining an overview of interpolation editing for interpolating between a plurality of motion data; [Figure 6A] FIG. 11 is an explanatory diagram illustrating motion blending, which is an example of interpolation editing. [Figure 6B] FIG. 13 is an explanatory diagram illustrating generation of interpolated motion data C, which is another example of interpolation editing. [Figure 7] 13 is an explanatory diagram for explaining a specific example of a process in which the generating unit 241 according to the present disclosure generates interpolated motion data. FIG. [Figure 8] FIG. 11 is an explanatory diagram for explaining a specific example of an interpolation parameter. [Figure 9] FIG. 2 is an explanatory diagram for explaining an example of a GUI according to the present disclosure. [Figure 10] 10 is a flowchart for explaining an example of an operation process of the PC 20 according to the present disclosure. [Figure 11] 2 is a block diagram showing a hardware configuration of a PC 20 according to the present disclosure. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0011] Hereinafter, preferred embodiments of the present disclosure will be described in detail with reference to the accompanying drawings. In this specification and the drawings, components having substantially the same functional configurations are denoted by the same reference numerals, and redundant description will be omitted.

[0012] The "Mode for Carrying Out the Invention" will be described in the following item order. 1. Overview of the information processing system 2. Functional configuration example 2.1. Example of functional configuration of server 10 2.2. PC20 Functional Configuration Example 3.Details 3.1. Linking Motion Data 3.2. Interpolation of motion data 3.3.Generating Interpolated Motion Data Scoring 3.5. GUI (Graphical User Interface) 4. Example of operation processing 5. Examples of effects 6. Variations 7.Hardware Configuration 8. Supplementary Information

[0013] <<1. Overview of the information processing system>> In order to visualize information on the movement of a moving body such as a human or an animal, for example, skeleton data expressed by a skeleton structure showing the structure of the body is used as the motion data. The skeleton data includes information such as the position and posture of a part. More specifically, the skeleton data includes various information such as the global position of the root joint and the relative posture of each joint. Note that the part in the skeleton structure corresponds to, for example, an end part or a joint part of the body. The skeleton data may also include bones, which are line segments connecting parts. The bones in the skeleton structure may correspond to, for example, human bones, but the positions and number of bones do not necessarily have to match the actual human skeleton. The motion data may further include various information such as skeletal information showing the length of each bone and the connection relationship with other bones, and time-series data of the ground contact information of both feet.

[0014] The position and orientation of each part in the skeleton data can be obtained by various motion capture techniques, such as a camera-based technique in which markers are attached to each part of the body and the positions of the markers are obtained using an external camera, and a sensor-based technique in which motion sensors are attached to the parts of the body and position information of the motion sensors is obtained based on time-series data obtained by the motion sensors.

[0015] In addition, skeleton data has a variety of uses. For example, motion data, which is time-series data of skeleton data, is used to improve form in dance or sports, or in applications such as VR (Virtual Reality) or AR (Augmented Reality). In addition, the motion data is used to generate an avatar image that imitates the user's movements, and the avatar image is distributed.

[0016] The information processing system according to the present disclosure makes it possible to link a plurality of pieces of motion data more naturally. First, an overview of the information processing system will be described with reference to FIG.

[0017] 1 is an explanatory diagram for explaining an information processing system according to an embodiment of the present disclosure. As shown in FIG. 1, the information processing system according to the present disclosure includes a network 1, a server 10, and a PC (Personal Computer) 20.

[0018] (Network 1) The network 1 according to the present disclosure is a wired or wireless transmission path for information transmitted from devices connected to the network 1. For example, the network 1 may include public line networks such as the Internet, telephone line networks, and satellite communication networks, various LANs (Local Area Networks) including Ethernet (registered trademark), and WANs (Wide Area Networks). The network 1 may also include a dedicated line network such as an IP-VPN (Internet Protocol-Virtual Private Network). The server 10 and the PC 20 are connected via the network 1.

[0019] (Server 10) The server 10 according to the present disclosure is a device that holds multiple pieces of motion data. The server 10 also learns the relationship between two pieces of motion data and parameters related to movements (hereinafter, sometimes referred to as interpolation parameters), and the interpolated motion data, and obtains a generation model that generates the interpolated motion data.

[0020] (PC20) The PC 20 according to the present disclosure is an example of an information processing device that obtains interpolated motion data that interpolates between two motion data, and calculates an index related to the motion speed of each motion data based on the two motion data and the interpolated motion data.

[0021] The overview of the information processing system according to the present disclosure has been described above. Next, the functional configurations of the server 10 and the PC 20 will be described in detail with reference to Figs.

[0022] <<2. Example of functional configuration>> <2.1. Example of functional configuration of server 10>> 2 is an explanatory diagram for explaining an example of a functional configuration of the server 10 according to the present disclosure. As shown in FIG. 2, the server 10 according to the present disclosure includes a storage unit 110, a learning unit 120, and a communication unit 130.

[0023] (Storage unit 110) The storage unit 110 according to the present disclosure stores software and various data. The storage unit 110 stores, for example, a plurality of pieces of motion data. The storage unit 110 may also store a generative model obtained by the learning unit 120.

[0024] (Learning Section 120) The learning unit 120 according to the present disclosure learns the relationship between two pieces of temporally and spatially independent motion data, the interpolation parameters, and the interpolated motion data, to generate a generative model. Details related to learning will be described later.

[0025] (Communication unit 130) The communication unit 130 according to the present disclosure transmits and receives various information to and from the PC 20 via the network 1. The communication unit 130 receives, for example, request information requesting motion data from the PC 20. The communication unit 130 also transmits motion data to the PC 20 in response to the request information received from the PC 20.

[0026] In addition, the communication unit 130 may transmit the generative model held in the storage unit 110 to the PC 20.

[0027] An example of the functional configuration of the server 10 according to the present disclosure has been described above. Next, an example of the functional configuration of the PC 20 according to the present disclosure will be described with reference to FIG.

[0028] <2.2. PC20 functional configuration example>> 3 is an explanatory diagram for explaining an example of a functional configuration of the PC 20 according to the present disclosure. As shown in FIG. 3, the PC 20 according to the present disclosure includes a communication unit 210, an operation display unit 220, a storage unit 230, and a control unit 240.

[0029] (Communication unit 210) The communication unit 210 according to the present disclosure transmits and receives various information to and from the server 10 via the network 1. The communication unit 210 transmits, for example, request information requesting motion data to the server 10. Then, the communication unit 210 receives the motion data transmitted in response to the request information from the server 10.

[0030] Furthermore, the communication unit 210 may receive from the server 10 a generative model for generating the interpolated motion data.

[0031] (Operation display section 220) The operation display unit 220 according to the present disclosure functions as a display unit that displays motion data received from the server 10 and various display information (e.g., linked motion data, etc.) generated by a generating unit 241 described later. The operation display unit 220 also functions as an input unit for the user to select motion data. When certain motion data is selected by the user, the communication unit 210 transmits request information requesting the motion data to the server 10.

[0032] The function of the display unit is realized by, for example, a CRT (Cathode Ray Tube) display device, a liquid crystal display (LCD) device, or an OLED (Organic Light Emitting Diode) device.

[0033] The function of the input unit is realized by, for example, a touch panel, a keyboard, or a mouse.

[0034] In FIG. 1, the PC 20 has a configuration in which the functions of the display unit and the operation unit are integrated, but the functions of the display unit and the input unit may be separate.

[0035] (Storage unit 230) The storage unit 230 according to the present disclosure stores software and various data. The storage unit 230 stores a generative model received from the server 10. The storage unit 230 may also store interpolated motion data generated by a generating unit 241 described below. The storage unit 230 may also store linked motion data that combines the interpolated motion data, first motion data, and second motion data generated by the generating unit 241.

[0036] (Control unit 240) The control unit 240 according to the present disclosure controls the overall operation of the PC 20. The PC 20 includes a generating unit 241 and a calculating unit 243, as shown in FIG.

[0037] The generating unit 241 according to the present disclosure is an example of an acquiring unit, and generates interpolated motion data that interpolates between first motion data and second motion data that are independent in time and space. The generation of the interpolated motion data will be described in detail later.

[0038] For example, the generating unit 241 may input the first motion data, the second motion data, and at least one or more interpolation parameters related to the movement to a generative model to generate the interpolated motion data. The generating unit 241 may also modify the interpolation parameters input to the generative model to generate the interpolated motion data multiple times.

[0039] Furthermore, the generation section 241 may generate linked motion data by linking the first motion data, the interpolated motion data, and the second motion data.

[0040] The calculation unit 243 according to the present disclosure calculates an index related to the motion speed of each piece of motion data based on the first motion data and the second motion data received by the communication unit 210 and the interpolated motion data generated by the generation unit 241. The calculation of the index related to the motion speed will be described in detail later.

[0041] The details of the PC 20 according to the present disclosure have been described above. Next, the details of the information processing system according to the present disclosure will be sequentially described with reference to FIGS.

[0042] <<3.Details>> Nowadays, motion data obtained by motion capture or manually is registered in a database, and applications exist that search for or edit the motion data from the database. For example, in editing the motion data, there are cases where a linking edit is required to link multiple motion data. First, a specific example of linking edit of multiple motion data will be described with reference to FIG. 4.

[0043] <3.1. Linking Motion Data> 4 is an explanatory diagram for explaining an example of a motion data linking process. In some cases, a section included in some motion data (hereinafter referred to as first motion data A) is modified by replacing it with other motion data (hereinafter referred to as second motion data B).

[0044] For example, a user who uses an application searches for a certain first motion data A, and selects, from among a plurality of sections A1 to A3 contained in the first motion data A, section A2 as a correction section.

[0045] Then, the user selects the second motion data B as the correction target for section A2. As a result, section A2 of the first motion data A may be corrected by replacing it with the correction target, the second motion data B. Note that the second motion data B here may be any motion data selected by the user on an application, or may be motion data automatically selected by the application.

[0046] For example, as shown in Fig. 4, when there are two second motion data B as candidates for correction of section A2 of first motion data A, the user selects one of the second motion data B. For example, when the user selects the left diagram of second motion data B shown in Fig. 4, section A2 of the first motion data A is corrected by replacing it with the left diagram of motion data B shown in Fig. 4.

[0047] However, if section A2 of the first motion data A is simply replaced with the second motion data B, the motion will suddenly switch at the boundary between the first motion data A and the second motion data B, and the movements contained in the motion data may appear unnatural.

[0048] Note that the boundary portion here includes the portion switching from the end position of section A1 of the first motion data A to the start position of the second motion data B. The boundary portion also includes the portion switching from the end position of the second motion data B to the start position of section A3 of the first motion data A.

[0049] Therefore, when linking and editing such a plurality of motion data, it is desirable to perform an interpolation edit that interpolates between the first motion data A and the second motion data B (that is, the boundary portion).

[0050] A specific example of joint editing of a plurality of motion data has been described above. Next, a specific example of interpolation editing for interpolating between a plurality of motion data will be described.

[0051] <3.2. Motion data interpolation editing> 5 is an explanatory diagram for explaining an outline of the interpolation editing for interpolating between a plurality of motion data. For example, it is assumed that the first motion data A includes a walking motion, and the second motion data B includes a kicking motion.

[0052] Here, as shown in Fig. 5, when the first motion data A is suddenly switched to the second motion data B, the motion of the motion data may become unnatural because the motion of the motion changes from walking to kicking instantaneously. For example, the joint positions of each part included in the first motion data A transition to the joint positions of each part included in the second motion data B in one frame, which may cause a situation in which the position of a certain part moves (warps) instantaneously. Thus, there is an interpolation editing method as a method for suppressing the unnaturalness of the motion that may occur when connecting multiple motion data. Here, a specific example of the interpolation editing will be described with reference to Figs. 6A and 6B.

[0053] 6A is an explanatory diagram for explaining a motion blend, which is an example of an interpolation edit. For example, the motion blend is an interpolation edit that linearly and geometrically combines a first motion data A and a second motion data B.

[0054] Interpolation editing using motion blending may perform interpolation in an interpolation section BQ such that the movement included in the first motion data A gradually switches to the movement included in the second motion data B. Note that motion data in which movement gradually switches in the interpolation section BQ is an example of interpolated motion data C.

[0055] Motion blending has few parameters related to interpolation and can be used for interpolation editing suitable for situations where real-time performance is required even if some unnaturalness of movement remains, whereas motion blending can be an interpolation editing that is not suitable for situations such as animation editing where more natural movement is required and many parameters related to interpolation are required.

[0056] 6B is an explanatory diagram for describing the generation of interpolated motion data C, which is another example of interpolation editing. The generation of the interpolated motion data C includes a process of generating new motion data using machine learning technology such as Deep Learning. Here, the newly generated motion data is an example of the interpolated motion data C.

[0057] Although generating the interpolated motion data C involves more parameters related to interpolation than motion blending, it is possible to generate interpolated motion data C that can interpolate between the first motion data A and the second motion data B with a more natural movement.

[0058] However, since there are many parameters related to interpolation, it is difficult for the user to manually set all the parameters. In addition, the parameters set by the user do not necessarily generate the interpolated motion data C that naturally interpolates between the motion data. Furthermore, when the parameters are set automatically, the type of movement desired by the user also depends on the user's preferences, so it is difficult to uniquely determine the desired movement when setting the parameters.

[0059] Next, the process in which the generation unit 241 according to the present disclosure generates the interpolated motion data C will be described in detail.

[0060] <3.3. Generation of Interpolated Motion Data> 7 is an explanatory diagram for explaining a specific example of a process in which the generating unit 241 according to the present disclosure generates interpolated motion data. First, as a preliminary step, the learning unit 120 included in the server 10 generates a generative model for generating the interpolated motion data C. For example, the learning unit 120 generates the generative model by machine learning using a set of two temporally and spatially independent motion data and parameters related to interpolation (hereinafter referred to as interpolation parameters), and the interpolated motion data as teacher data.

[0061] 7, the learning unit 120 divides the motion data into three parts of arbitrary length, which are the first motion data A, the second motion data B, and the third motion data L. The lengths of the first motion data A, the second motion data B, and the third motion data L may be changed for each use case, or may be variable during learning of the generative model.

[0062] Next, the learning unit 120 inputs the first motion data A, the second motion data B, and the interpolation parameters to the generative model. At this time, the third motion data L is regarded as lost (i.e., the third motion data L is not input to the generative model). In this way, the learning unit 120 generates a provisional generative model.

[0063] Next, the learning unit 120 uses the generated generative model to calculate the loss between the interpolated motion data C generated from the first motion data A and the second motion data B and the third motion data (i.e., the correct motion).

[0064] The learning unit 120 may then modify parameters (e.g., joint positions and joint orientations) of the interpolated motion data to reduce loss and repeat learning to update the generative model. The communication unit 130 included in the server 10 may then transmit the generative model generated by the learning unit 120 to the PC 20.

[0065] The learning unit 120 may generate a generative model from any motion data (a plurality of motion data including various movements), or may generate a generative model for each specific motion data such as walking or running.

[0066] The generation unit 241 included in the PC 20 according to the present disclosure may generate the interpolated motion data C using the generation model G obtained from the server 10. More specifically, the generation unit 241 may generate the interpolated motion data x C may be generated.

[0067]

number

[0068] In addition, in formula (1), x C is the interpolated motion data, and x A is the first motion data, and x B is the second motion data, and c 0 ~c n is the interpolation parameter.

[0069] The interpolation parameters are various parameters related to the movement of the motion data, such as boundary conditions or constraint conditions, and may be partially set by the user or automatically set based on the first motion data A and the second motion data B.

[0070] Here, a specific example of the interpolation parameters will be described with reference to FIG.

[0071] (Interpolation parameters) Fig. 8 is an explanatory diagram for explaining a specific example of the interpolation parameters. Fig. 8 shows a situation where the first motion data A and the second motion data B are independent in time and space. Here, spatial independence refers to a state where the first motion data A and the second motion data B are spatially separated, and temporal independence refers to a state where the time from the start time to the end time of the first motion data A does not overlap with the time from the start time to the end time of the second motion data B.

[0072] For example, the position of the motion data in the three-dimensional space is expressed by a position vector P. For example, at time t A The position of the first motion data A in A =(x A ,y A ,z A ) and at time t B The position of the second motion data B in B =(x B ,y B ,z B )

[0073] The position of the motion data in three-dimensional space may be, for example, the position of the character's root joint (waist joint), the position of another joint (for example, the toe joint), or a position calculated from the positions of multiple joints.

[0074] In addition, the posture of the motion data in the three-dimensional space is expressed as a total joint posture S, which corresponds to the relative postures of all the joints. For example, at time tA The posture of the first motion data A in A (t A ) and at time t B The posture of the second motion data B in B (t B )

[0075] There may be various types of interpolation parameters. In this specification, eight types of parameters (A) to (H) are introduced, but other parameters may be included. In addition, some of the parameters (A) to (H) may be manually specified by the user. In addition, all of the parameters (A) to (H) may not be used to generate the interpolated motion data. In other words, some of the parameters (A) to (H) may be input to the generative model.

[0076] (A) Spatial position The interpolation parameters may include parameters related to the spatial positions of the motion data. For example, the parameters related to the spatial positions are the distances in three-dimensional space between the first motion data A and the second motion data B. In other words, the parameters related to the spatial positions are the differences between the spatial positions of the first motion data A and the second motion data B, and ||p A -p B Represented by ||.

[0077] Here, the position p of the first motion data A or the position p of the second motion data B may be set manually by a user, or may be set automatically based on the first motion data A and the second motion data B.

[0078] For example, the spatial position difference may be automatically set based on the estimation result by estimating the trajectory of a part or joint of the interpolated motion data based on the velocity of each root joint of the first motion data A and the second motion data B.

[0079] (B) Interpolation time The interpolation parameters may include parameters related to an interpolation time to be interpolated by the interpolated motion data. The parameters related to the interpolation time are a time difference on the timeline between the first motion data A and the second motion data B, and are represented as t B -t A It is expressed as:

[0080] Furthermore, the end time of the first motion data A or the start time of the second motion data B may be set manually by the user or automatically.

[0081] For example, the interpolation time may be automatically set based on the difference between the velocity or spatial position of the root joint between the first motion data A and the second motion data B.

[0082] (C) Posture of the first motion data A (interpolation start posture, boundary conditions) The interpolation parameters may include parameters related to the posture of the first motion data A. The parameters related to the posture of the first motion data A are set at the interpolation start time t A By trimming the following motion data A, the interpolation start time t A is a parameter that adjusts the attitude of S A (t A ) By adjusting the parameters relating to the posture of the first motion data A, it may be possible to make the posture of the interpolated motion data C at the start of the interpolation more natural.

[0083] Furthermore, the parameters relating to the posture of the first motion data A may be automatically set based on the most stable posture as the interpolation start posture estimated from the velocity of the root joint of the first motion data A.

[0084] (D) Posture of the second motion data B (interpolation end posture, boundary conditions) The interpolation parameters may include parameters related to the posture of the second motion data B. The parameters related to the posture of the second motion data B may include parameters related to the posture of the second motion data B at the interpolation end time t B By trimming the previous motion data B, the interpolation end time t B is a parameter that adjusts the attitude of S B (t B ) By adjusting the parameters relating to the posture of the second motion data B, it may be possible to make the posture of the interpolated motion data C at the end of the interpolation more natural.

[0085] Furthermore, the parameters relating to the posture of the second motion data B may be automatically set based on the most stable posture as the interpolation end posture estimated from the velocity of the root joint of the second motion data B.

[0086] (E) Contact Conditions The interpolation parameters may include parameters related to a contact condition. The parameters related to the contact condition include an interpolation start time t A to the end of the interpolation time t B It may also be a parameter related to contact with the ground, such as the number of steps taken between the time when the user walks and the time when the user walks.

[0087] For example, the parameters related to the contact condition are the interpolation start time t A to the end of the interpolation time t B For example, if a parameter for the contact condition is set to "5 steps" throughout the entire interpolation time, the interpolation start time t A to the end of the interpolation time t B Then, the interpolated motion data C is generated, which takes five steps.

[0088] In addition, if the state in which the toes are in contact with the ground is set to "1" and the state in which the toes are not in contact with the ground (i.e., the toes are off the ground) is set to "0", the parameters related to the contact condition are t A ~t BFor example, c(t) may be specified sequentially at each time t between the two points in time, such as c(t)={0,1}. 1 If the contact condition parameter is set as "{0,1}", then at time t 1 In this case, interpolated motion data C is generated in which the toe of the left foot is not in contact with the ground and the toe of the right foot is in contact with the ground.

[0089] In addition, the parameters related to the contact condition may be automatically set based on a more appropriate number of steps estimated from the difference in spatial positions of the first motion data A and the second motion data B or the interpolated time, and the parameters may be automatically set based on the estimation result.

[0090] (F) Location constraints The interpolation parameters may include parameters related to position constraint conditions. The parameters related to position constraint conditions are parameters related to fixing the posture of the interpolated motion data C during the process. The parameters related to the position constraint conditions are t A ~t B At a certain time t between the time instants, S is a parameter that fixes the posture of all joints in the interpolated motion data C. C It is expressed as (t).

[0091] For example, t A ~t B If a user is looking for interpolated motion data in which the knee touches the ground at a certain time t between time t and time t, the user may set the parameters of all joint postures that will cause the knee to touch the ground at that time t as parameters related to the position constraint conditions.

[0092] (G) Semantic Conditions The interpolation parameters may include parameters related to semantic conditions. The parameters related to semantic conditions include parameters related to a style of the interpolated motion data C.

[0093] The style here is defined by labels expressed as the movement characteristics and genre of the motion data, and the learning unit 120 can assign labels to the motion data and train a generative model to set parameters related to semantic conditions.

[0094] Specific examples of styles include "dynamically," "dance-like," "jumping," "masculine (feminine)," etc. Parameters related to semantic conditions may be added for each application, or the generating unit 241 may generate the interpolated motion data C without providing parameters related to such semantic conditions.

[0095] (H) Random numbers or noise The interpolation parameters may include parameters related to random numbers or noise. For example, a random number may be input as a probabilistic generation model to a generation model that generates the interpolated motion data C. By using parameters related to random numbers or noise, different interpolated motion data C can be generated even if the other interpolation parameters described above are fixed.

[0096] The generation unit 241 may generate various variations of the interpolated motion data C by inputting a random number as a seed, or may generate the interpolated motion data C without providing a random number (i.e., by setting the random number to a fixed value).

[0097] Specific examples of the interpolation parameters have been described above. As described above, the generation unit 241 generates the interpolation motion data C based on the first motion data A, the second motion data B, and the interpolation parameters.

[0098] The generation unit 241 then generates linked motion data by linking the first motion data A, the interpolated motion data C, and the second motion data. However, depending on the setting values ​​of the interpolation parameters, there is a risk that the motion data included in the generated linked motion data may not be linked naturally (smoothly).

[0099] Therefore, the information processing system according to the present disclosure has a mechanism that enables a more natural linking of the first motion data A, the interpolated motion data C, and the second motion data B. More specifically, the calculation unit 243 calculates an index related to the motion speed of each piece of motion data based on the first motion data A, the second motion data B, and the interpolated motion data C generated by the generation unit 241.

[0100] Next, a specific example of the index calculated by the calculation unit 243 and a specific example of the process of presenting candidates for interpolated motion data based on the index to the user will be described.

[0101] <3.4. Scoring> First, the generation unit 241 uses the above-mentioned formula (1) to generate the first motion data x A , the second motion data x B and the interpolation parameter c 0 ~c n Based on the interpolated motion data x C Then, the calculation unit 243 may calculate a total index s including at least one index S by using the following formula (2).

[0102]

number

[0103] For example, the calculation unit 243 may calculate a total index s including five indexes S by using the following formula (3).

[0104]

number

[0105] In formula (3), w p、 w v、 w j、 w f and w m are weighting coefficients for each index, and may be set automatically or may be specified for each use case.

[0106] Furthermore, the total index s in formula (3) is defined as a combination of a physical index and a heuristic index. However, the index S included in the total index s is not limited to the physical index and the heuristic index described below. Furthermore, the index S in the total index s does not necessarily include all of the indexes S described below. For example, in formula (3), w m S meta( The term meta-indicator may not be included. First, the four physical indices in Equation (3) will be explained.

[0107] (Physical index) For example, the physical indices include indices related to the motion speeds of the first motion data A, the interpolated motion data C, and the second motion data B (hereinafter, sometimes referred to as each motion data). The motion speed here includes, for example, the speed of the root joint, the speed of other joints (e.g., the speed of the toes), or a speed obtained by combining the speeds of a plurality of joints. Indices related to the motion speed may also include indices related to the speed, acceleration, or position of the motion.

[0108] For example, the physical index is the continuous position index S, which indicates the degree to which each motion data is continuous. position The continuous position index S position may be an index based on the difference between the maximum value of the velocity of the interpolated motion data C and the average velocity of the first motion data A and the second motion data B in multiple frames before and after the interpolated motion data C.

[0109] In the section where the first motion data A transitions to the interpolated motion data C and the section where the interpolated motion data C transitions to the second motion data B, if the joint positions of the motion data move instantaneously, unnatural behavior may occur. The continuous position index S is used as an index to reduce such unnatural behavior. position is defined, and the smaller the value obtained by the following formula (4), the smaller the continuous position index S position is calculated to be large.

[0110]

number

[0111] According to formula (4), the continuous position index S position is the maximum velocity of the interpolated motion data C, Max(||v C ||) and the average velocity ||v of the first motion data A and the second motion data B in multiple frames before and after the interpolated motion data C. AB The smaller the difference between || and , the larger the calculated continuous position index S position A large calculated value of indicates, in other words, a state in which the joint positions of the motion data are moving continuously.

[0112] In addition, the physical index includes the speed index S, which indicates the degree of smoothness of the speed of each motion data. velocity may be included. velocity may be an index based on the difference between the average speed of the interpolated motion data C and the average speeds of the first motion data A and the second motion data B.

[0113] When the average speed of the first motion data A and the second motion data B is significantly different from the average speed of the interpolated motion data C, the interpolated motion data C may suddenly move faster or slower compared to the first motion data A or the second motion data B, resulting in unnatural behavior. The speed index S is used as an index to reduce such unnatural behavior. velocity is defined, and the smaller the value obtained by the following formula (5), the higher the speed index S velocity is calculated to be large.

[0114]

number

[0115] According to formula (5), the speed index S velocity is the average velocity of the interpolated motion data C ||Average(v C )|| and the average velocity ||v AB The smaller the difference between || and ||, the larger the calculated speed index S velocity In other words, a larger calculated value of indicates that each piece of motion data is moving more smoothly.

[0116] In addition, the physical index includes an acceleration index S, which indicates the degree to which the jerk (the rate of change of acceleration) of the interpolated motion data C is small. jerky may be included. jerky may be an index based on a differential component of the acceleration of the interpolated motion data C.

[0117] If the interpolated motion data C generated by the generation unit 241 suddenly accelerates or decelerates, it may become an unnatural behavior. As an index for reducing such unnatural behavior, the acceleration index S jerky is defined, and the smaller the value obtained by the following formula (6), the higher the acceleration index S jerky is calculated to be large.

[0118]

number

[0119] According to formula (6), the acceleration index S jerky The acceleration index S is calculated to be large when the jerk, which is the differential value of the interpolated motion data C, becomes small. jerky A large calculated value of indicates, in other words, that the jerk of each piece of motion data is small (that is, there is no sudden acceleration or deceleration).

[0120] In addition, the physical index includes the foot index S, which is the foot index of the interpolated motion data C that indicates whether the foot does not slip when it touches the ground. foot may be included. foot is an index related to the speed of the tip of the foot of the interpolated motion data C when the tip of the foot is in contact with the ground.

[0121] When the toe of the interpolated motion data C is in contact with the ground, if the magnitude of the velocity of the toe is 0 or more, the toe of the interpolated motion data C is slipping on the ground. In many cases, foot slipping can be an unnatural behavior. As an index for reducing such unnatural behavior, the toe index S foot is defined, and the smaller the value obtained by the following formula (7), the smaller the toe index S foot is calculated to be large.

[0122]

number

[0123] According to formula (7), the toe index S foot is calculated to be large when the magnitude of the toe velocity of the interpolated motion data C becomes small. foot A large calculation of α indicates, in other words, a state in which the toes of the interpolated motion data C are not slipping on the ground.

[0124] A specific example of the physical indicator has been described above. Next, an example of the heuristic indicator included in the formula (3) will be described.

[0125] (Heuristic Indicators) The heuristic metrics include meta-metrics S, which are different for each use case. meta may be included. meta may be an index according to a use case, based on the difference between an attribute of the interpolated motion data and an attribute required for the interpolated motion data.

[0126] If the attributes of the interpolated motion data C generated by the generation unit 241 are significantly different from the user's sensibility or use case, the user may feel that the data is unnatural. For example, if the first motion data A and the second motion data B are motions performed by a woman, and the interpolated motion data C generated by the generation unit 241 is a motion that looks like it was performed by a large man, the combined motion data obtained by combining the respective motion data may result in unnatural behavior. As an index for reducing such unnatural behavior according to the use case, the meta index S meta is defined, and F is a function for determining the attribute of the interpolated motion data C, and m is the attribute to be sought. The smaller the value obtained by the following formula (8), the higher the meta-index S meta is calculated to be large.

[0127]

number

[0128] According to formula (8), the meta-index S meta is the attribute m determined according to the use case and the attribute F(M C The smaller the difference between and , the larger the calculated meta index S meta In other words, the fact that the calculated value is large indicates that the interpolated motion data C generated by the generation unit 241 corresponds to the user's sensibility and use case.

[0129] After the calculation unit 243 calculates the total index s as described above, the generation unit 241 changes the values ​​of the interpolation parameters and generates the interpolated motion data x C Then, the calculation unit 243 again generates the generated interpolated motion data x C Calculate the total index s from

[0130] Such a series of processes from modifying the interpolation parameters to calculating the total index s may be repeated multiple times.

[0131] That is, the generation unit 241 modifies the interpolation parameters and generates the interpolated motion data x C The calculation unit 243 generates the interpolated motion data x C A series of processes for calculating the total index s from may be repeated multiple times.

[0132] The calculation method of the physical index is not limited to the above-mentioned formulas, and a more precise calculation method may be used. Furthermore, the calculation method of the heuristic index may use an index suited to other use cases.

[0133] In addition, in the calculation formulas for the physical index and the heuristic index described above, an example has been described in which the smaller the value obtained by each calculation formula, the larger the index S becomes. However, the formulas may be modified so that the larger the value obtained by each calculation formula, the larger the index S becomes.

[0134] The above describes specific examples of the index calculated by the calculation unit 243. According to the series of processes described above, a plurality of combinations of the interpolated motion data C and the scores (total index or index) calculated from the plurality of motion data C are prepared.

[0135] Here, the generation unit 241 may add the interpolated motion data C whose score (total index or index) has been calculated by the calculation unit 243 to the candidate list.

[0136] The generation unit 241 may then sort the candidate list of the interpolated motion data C in order of score (more specifically, in descending order of score), and the operation display unit 220 may display the sorted candidates of the interpolated motion data.

[0137] This allows the user to check candidates for the interpolated motion data C sorted in descending order of score, thereby improving user convenience in viewing the interpolated motion data C.

[0138] Next, an example of a GUI (Graphical User Interface) according to the present disclosure will be described with reference to FIG.

[0139] <3.5. GUI (Graphical User Interface)> 9 is an explanatory diagram for explaining an example of a GUI according to the present disclosure. First, the user uses the operation display unit 220 to select the first motion data A and insert it into an arbitrary section M1 on the timeline.

[0140] Next, the user selects the second motion data B using the operation display unit 220, and inserts it into another arbitrary section M2 on the timeline. Note that the arbitrary sections M1 and M2 may be specified by an operation such as drag and drop, or by inputting the time.

[0141] Here, the generation unit 241 sets the section from the end position of the first motion data A (i.e., the end position of the arbitrary section M1) to the start position of the second motion data B (i.e., the start position of another arbitrary section M2) as the interpolation section M3.

[0142] Furthermore, the user may use the operation and display unit 220 to manually set some of the above-mentioned interpolation parameters.

[0143] Then, when the user performs an operation related to generating interpolated motion data (for example, pressing an execute button not shown), the generation unit 241 generates the interpolated motion data, and the calculation unit 243 calculates the score of the interpolated motion data generated by the generation unit 241.

[0144] The generation unit 241 then sorts the interpolated motion data C in order of score, and the operation display unit 220 displays a candidate list of the interpolated motion data sorted in order of score, as shown in FIG.

[0145] The candidate list shown in Fig. 9 is a candidate list in which the scores are calculated to be highest for the first interpolated motion data C1, the second interpolated motion data C2, the third interpolated motion data C3, and the fourth interpolated motion data C4. If the upper limit of the interpolated motion data to be included in the candidate list is set to four, the four interpolated motion data C1 to C4 as shown in Fig. 9 are displayed on the operation display unit 220, but if the upper limit of the interpolated motion data to be included in the candidate list is set to five or more, for example, the user can scroll down the candidate list to check the interpolated motion data C5 to which the scores are calculated to be low.

[0146] Then, when the user selects one piece of interpolated motion data C from the candidate list, the generation unit 241 generates linked motion data by linking the first motion data A, the interpolated motion data C selected by the user, and the second motion data B.

[0147] Then, the operation display unit 220 displays the linked motion data generated by the generation unit 241.

[0148] Although an example of the GUI according to the present disclosure has been described above, the GUI is not limited to the above example. For example, the operation display unit 220 may display linked motion data that simply links the first motion data A and the second motion data B, or linked motion data that is interpolated by motion blending.

[0149] The information processing system according to the present disclosure has been described in detail above. Next, an example of the operation process of the PC 20 according to the present disclosure will be described with reference to FIG.

[0150] <<4. Operation processing example>> 10 is a flowchart for explaining an example of the operation process of the PC 20 according to the present disclosure. First, the user selects the first motion data A using the operation display unit 220 (S101).

[0151] Next, the user uses the operation display unit 220 to select the second motion data B (S105).

[0152] Next, the generation unit 241 automatically sets at least one or more interpolation parameters (S109). Here, the user may use the operation display unit 220 to manually set some of the interpolation parameters.

[0153] Then, the generating unit 241 generates the interpolated motion data C based on the first motion data A and second motion data B selected by the user and the interpolation parameters set by the generating unit 241 (or the user) (S113).

[0154] Next, the calculation unit 243 calculates a score (a total index or index) based on the interpolated motion data C generated by the generation unit 241 and the first motion data A and second motion data B selected by the user (S117).

[0155] Next, the generation unit 241 adds the interpolated motion data C whose score has been calculated by the calculation unit 243 to the candidate list (S121). Note that the generation unit 241 may add to the candidate list the interpolated motion data C whose score calculated by the calculation unit 243 is equal to or greater than a predetermined value, and may not add to the candidate list the interpolated motion data C whose score is less than the predetermined value.

[0156] Next, the generation unit 241 sorts the candidate list for the interpolated motion data C in order of score (more specifically, in order of highest score) (S125).

[0157] Then, the generating unit 241 determines whether or not the amount of the interpolated motion data C added to the candidate list has reached the upper limit of the candidate list (S129). If the upper limit of the candidate list has been reached (S129: YES), the process proceeds to S133. If the upper limit of the candidate list has not been reached (S129: NO), the process returns to S109 again, where the interpolation parameters are reset (changed), and the processes of S109 to S129 are repeated until the upper limit of the candidate list is reached in S129. Note that the upper limit of the candidate list here corresponds to a predetermined number of times set in advance, and may be specified by the user or may be set automatically.

[0158] When the amount of the interpolated motion data C reaches the upper limit of the candidate list (S129: YES), the operation display unit 220 displays a candidate list of the interpolated motion data C sorted in order of score (S133).

[0159] Next, the user selects one of the interpolated motion data C included in the candidate list on the operation display unit 220 (S137).

[0160] Then, the generation unit 241 generates linked motion data by interpolating the interpolated motion data C selected by the user between the first motion data A and the second motion data B (S141), and the PC 20 according to the present disclosure ends the processing.

[0161] <<5. Examples of Actions and Effects>> According to the present disclosure described above, various operational effects can be obtained. For example, the generation unit 241 according to the present disclosure generates the interpolated motion data C that interpolates between the first motion data A and the second motion data B that are independent in time and space, and the calculation unit 243 calculates an index related to the motion speed of each motion data based on the first motion data A, the second motion data B, and the interpolated motion data C generated by the generation unit 241. This allows the user to determine the degree to which the connected motion data, which is expected when the first motion data A, the second motion data B, and the interpolated motion data C are connected, will behave more naturally. As a result, the generation unit 241 may be able to generate connected motion data that connects a plurality of motion data more naturally.

[0162] Furthermore, the generation unit 241 changes the interpolation parameters to generate the interpolated motion data C multiple times, and the calculation unit 243 calculates a score using each piece of interpolated motion data C generated by the generation unit 241. Furthermore, the operation display unit 220 displays a candidate list of the interpolated motion data C sorted in order of score. This makes it easy for the user to select the interpolated motion data C that the user desires. As a result, the user can confirm linked motion data that meets the user's wishes.

[0163] <<6. Modifications>> (Motion capture interpolation technology) The interpolation process by generating the interpolated motion data according to the present disclosure can also be applied to the interpolation technology in motion capture. For example, when motion is recorded by a motion capture system, some of the motion may be lost due to occlusion or the like.

[0164] In this case, the generation unit 241 may generate interpolated motion data C that smoothly interpolates the first motion data A and the second motion data B, respectively, for the motion before and after the lost section, to interpolate the motion of the lost section.

[0165] In this case, when setting the interpolation parameters, among the multiple interpolation parameters, the interpolation parameters related to the spatial position, the interpolation time, the posture of the first motion data A, and the posture of the second motion data B are fixed. The PC 20 modifies the other interpolation parameters (i.e., the contact condition, the position constraint condition, the semantic condition, the random number, etc.) to repeatedly generate the interpolated motion data and calculate the score. Then, after candidates for the interpolated motion data are displayed, the user can specify one of the interpolated motion data to interpolate the lost section. Note that the user does not necessarily have to specify one of the interpolated motion data, and the generation unit 241 may automatically specify the interpolated motion data C with the highest calculated score or the interpolated motion data with a score exceeding a threshold value to interpolate the lost section.

[0166] In addition, a case where the motion of a part of the body (for example, a hand) is lost is also considered. In this case, the generation unit 241 may determine a lost section for the part of the body (referred to as a target part).

[0167] The generating unit 241 may generate the interpolated motion data C that smoothly interpolates the target part by treating the motion before and after the target part is lost as the first motion data A and the second motion data B, respectively. This allows the motion of the body excluding the target part to be directly obtained by motion capture, making it possible to interpolate the target part where the motion is lost.

[0168] In addition, when the interpolation process of the target part is executed, the target part to be interpolated on the timeline of the application may be branched (for example, only the right hand, etc.). In addition, there may be a plurality of target parts whose motion is to be interpolated (for example, the right hand + the left hand, etc.).

[0169] Furthermore, the generating unit 241 may set position constraint conditions for each part other than the target part to be interpolated as an interpolation parameter. In this way, interpolated motion data may be generated in which only the target part (for example, only the right hand) is interpolated while the whole body other than the target part moves according to the motion data.

[0170] Furthermore, the calculation unit 243 may calculate the above-mentioned score from the target parts generated by the generation unit 241. When there are multiple target parts, the calculation unit 243 may change the value of the interpolation parameter for each target part and calculate an index by weighting each target part. Then, the operation display unit 220 may display the interpolated motion data of the target part and the whole body motion excluding the target part in combination.

[0171] In addition, in VR games and applications in the metaverse, a motion capture system is used to reflect the user's movements on the avatar in real time. In some cases, pre-registered animations such as hand gestures and emote motions are added during real-time motion capture.

[0172] Here, the generating unit 421 may generate interpolated motion data C by taking real-time motion as the first motion data A and emote motion as the second motion data B, and smoothly interpolating the first motion data A and the second motion data B. This may enable a natural transition from real-time motion to emote motion. Also, new expressions (e.g., VR live) that combine real-time motion and pre-recorded motion become possible.

[0173] (Compositing with Motion Blending) In addition, the interpolation process according to the present disclosure may be a combination of the interpolation process by generating interpolated motion data and the interpolation process by motion blending. For example, the generation unit 241 may perform the interpolation process by motion blending for the upper body of the motion data, and the interpolation process by generating the interpolated motion data for the lower body.

[0174] In motion blending, the generation unit 421 may, for example, perform linear interpolation from the interpolation start posture of the first motion data A to the interpolation end posture of the second motion data B to interpolate the connection position of the motion data.

[0175] <<7. Hardware configuration example>> The embodiment of the present disclosure has been described above. The information processing such as generating the interpolated motion data and calculating the score (total index and index) described above is realized by cooperation between software and the hardware of the PC 20 described below. Note that the hardware configuration described below can also be applied to the server 10.

[0176] 11 is a block diagram showing a hardware configuration of a PC 20 according to the present disclosure. The PC 20 according to the present disclosure includes a CPU (Central Processing Unit) 2001, a ROM (Read Only Memory) 2002, a RAM (Random Access Memory) 2003, and a host bus 2004. The PC 20 also includes a bridge 2005, an external bus 2006, an interface 2007, an input device 2008, an output device 2010, a storage device (HDD) 2011, a drive 2012, and a communication device 2015.

[0177] The CPU 2001 functions as an arithmetic processing device and a control device, and controls the overall operation of the PC 20 in accordance with various programs. The CPU 2001 may also be a microprocessor. The ROM 2002 stores programs and arithmetic parameters used by the CPU 2001. The RAM 2003 temporarily stores programs used in the execution of the CPU 2001 and parameters that change appropriately during the execution. These are connected to each other by a host bus 2004 that is composed of a CPU bus and the like. The functions of the generation unit 241 and the calculation unit 243 described with reference to FIG. 3 can be realized by cooperation between the CPU 2001, the ROM 2002, and the RAM 2003 and software.

[0178] The host bus 2004 is connected to an external bus 2006, such as a PCI (Peripheral Component Interconnect / Interface) bus, via a bridge 2005. It is not necessary to configure the host bus 2004, bridge 2005, and external bus 2006 separately, and these functions may be implemented in a single bus.

[0179] The input device 2008 is composed of input means such as a mouse, keyboard, touch panel, button, microphone, switch, and lever for the user to input information, and an input control circuit which generates an input signal based on the user's input and outputs it to the CPU 2001. The user of the PC 20 can input various data to the PC 20 and instruct processing operations by operating the input device 2008.

[0180] The output device 2010 includes, for example, a display device such as a liquid crystal display device, an OLED device, and a lamp. Furthermore, the output device 2010 includes an audio output device such as a speaker and a headphone. The output device 2010 outputs, for example, reproduced content. Specifically, the display device displays various information such as reproduced video data as text or images. Meanwhile, the audio output device converts the reproduced audio data, etc. into audio and outputs it.

[0181] The storage device 2011 is a device for storing data. The storage device 2011 may include a storage medium, a recording device for recording data on the storage medium, a reading device for reading data from the storage medium, and a deleting device for deleting data recorded on the storage medium. The storage device 2011 is configured, for example, with an HDD (Hard Disk Drive). This storage device 2011 drives a hard disk and stores programs executed by the CPU 2001 and various data.

[0182] The drive 2012 is a reader / writer for a storage medium, and is built into or externally attached to the PC 20. The drive 2012 reads out information recorded on a removable storage medium 30, such as an attached magnetic disk, optical disk, magneto-optical disk, or semiconductor memory, and outputs the information to the RAM 2003. The drive 2012 can also write information to the removable storage medium 30.

[0183] The communication device 2015 is, for example, a communication interface configured with a communication device for connecting to the network 1. The communication device 2015 may be a wireless LAN compatible communication device, a LTE (Long Term Evolution) compatible communication device, or a wired communication device that performs wired communication.

[0184] <<8. Supplementary Information>> Although the preferred embodiment of the present disclosure has been described in detail above with reference to the accompanying drawings, the present disclosure is not limited to such examples. It is clear that a person having ordinary knowledge in the technical field to which the present disclosure belongs can conceive of various modified or amended examples within the scope of the technical ideas described in the claims, and it is understood that these also naturally belong to the technical scope of the present disclosure.

[0185] For example, the PC 20 may further include all or a part of the functional configuration of the server 10 according to the present disclosure. When the PC 20 includes all the functional configuration of the server 10 according to the present disclosure, the PC 20 can execute a series of processes from generation of the interpolated motion data to calculation of the index without communication via the network 1.

[0186] Furthermore, the server 10 may include the generation unit 241 and the calculation unit 243. In this case, an example of processing relating to the generation of the interpolated motion data to the calculation of the index may be performed on the server, and the result of the processing (for example, a candidate list sorted in order of score) may be transmitted to the PC 20.

[0187] Furthermore, in the case where the server 10 includes a generating unit, the PC 20 may acquire the interpolated motion data generated by the generating unit of the server 10 from the server 10. In this case, the communication unit 210 included in the PC 20 corresponds to the acquiring unit.

[0188] The software may be used as a standalone application for searching and editing motion data, or the motion interpolation editing function may be used as a plug-in for an existing DCC (Digital Content Creation) tool.

[0189] In addition, although the generation of interpolated motion data and motion blending have been exemplified as specific examples of interpolation editing, the interpolation editing according to the present disclosure is not limited to these examples. For example, the generation unit 241 may interpolate motion data by combining interpolation editing by generating interpolated motion data and interpolation editing by text search of motion data. For example, a user may perform a text search of motion data of an arbitrary body part, and select one motion data from multiple motion data displayed as a search result. In this case, the generation unit 241 may interpolate the movement of the body part using the motion data specified by the user, and may interpolate the movement of other body parts by generating interpolated motion data.

[0190] Furthermore, depending on the combination of the first motion data A and the second motion data B, there is a possibility that changing the interpolation parameters will not increase the score calculated by the calculation unit 243. In such a case, the operation display unit 220 may present recommendation information such as changing the second motion data B to another second motion data B'.

[0191] In addition, the steps in the processing of the PC 20 in this specification do not necessarily have to be processed in chronological order according to the order described in the flowchart. For example, the steps in the processing of the PC 20 may be processed in an order different from the order described in the flowchart.

[0192] It is also possible to create a computer program for causing hardware such as a CPU, ROM, and RAM built into the server 10 and the PC 20 to perform functions equivalent to those of the above-described configurations of the server 10 and the PC 20. A storage medium storing the computer program is also provided.

[0193] In addition, the effects described in this specification are merely descriptive or exemplary and are not limiting. In other words, the technology according to the present disclosure may achieve other effects that are apparent to a person skilled in the art from the description of this specification, in addition to or in place of the above effects.

[0194] Note that the following configurations also fall within the technical scope of the present disclosure. (1) an acquisition unit that acquires interpolated motion data that is interpolated between first motion data and second motion data that are independent in time and space; a calculation unit that calculates an index related to a motion speed of each piece of motion data based on the first motion data, the second motion data, and the interpolated motion data acquired by the acquisition unit; An information processing device comprising: (2) The indicator is an index based on a difference between an average speed of the interpolated motion data and an average speed of the first motion data and the second motion data; The information processing device according to (1). (3) The indicator is an index based on a difference between a maximum value of the velocity of the interpolated motion data and an average velocity of the first motion data and the second motion data in a plurality of frames before and after the interpolated motion data; The information processing device according to (1) or (2). (4) The indicator is an index based on a differential component of the acceleration of the interpolated motion data; The information processing device according to any one of (1) to (3). (5) The indicator is the interpolated motion data including an index related to a velocity of the tip of the foot when the tip of the foot is in contact with the ground; The information processing device according to any one of (1) to (4). (6) The calculation unit is Calculating an index according to a use case based on a difference between an attribute of the interpolated motion data and an attribute required for the interpolated motion data. The information processing device according to any one of (1) to (5). (7) The calculation unit is Calculating a plurality of indices related to the speed of each motion data and a plurality of indices related to the use case, and calculating a total index by multiplying each of the plurality of indices by a weighting coefficient and adding the results; The information processing device according to (6). (8) The acquisition unit is obtaining the interpolated motion data based on the first motion data and the second motion data; The information processing device according to (7). (9) The acquisition unit is obtaining the interpolated motion data based on the first motion data, the second motion data, and at least one interpolated parameter associated with a movement; The information processing device according to (8). (10) The acquisition unit is acquiring the interpolated motion data using a generative model obtained by learning two pieces of temporally and spatially independent motion data and a relationship between the interpolation parameters and the interpolated motion data; The information processing device according to (9). (11) The acquisition unit is modifying the interpolation parameters to obtain the interpolated motion data a plurality of times; The calculation unit is calculating the total index from each of the plurality of interpolated motion data acquired by the acquisition unit; The information processing device according to (10). (12) the at least one interpolation parameter is a plurality of interpolation parameters, One or more of the interpolation parameters included in the plurality of interpolation parameters are automatically set based on a difference in velocity or spatial position of a root joint between the first motion data and the second motion data. The information processing device according to any one of (9) to (11). (13) The acquisition unit is obtaining a candidate list of interpolated motion data according to the magnitude of the total index; The information processing device according to (11). (14) The acquisition unit is obtaining a candidate list in which the interpolated motion data is sorted in order according to the magnitude of the sum index; The information processing device according to (13). (15) The acquisition unit is modifying an interpolation parameter a predetermined number of times to obtain the predetermined number of interpolated motion data, and obtaining a candidate list of the predetermined number of interpolated motion data sorted in descending order of the total index; The information processing device according to (14). (16) The acquisition unit is acquiring linked motion data by linking the first motion data, the second motion data, and the interpolated motion data; The information processing device according to any one of (13) to (15). (17) The acquisition unit is acquiring linked motion data by linking the first motion data, the second motion data, and one piece of interpolated motion data selected by a user from the candidate list; The information processing device according to (16). (18) Obtaining interpolated motion data that interpolates between first and second motion data that are temporally and spatially independent; Calculating an index related to a motion speed of each motion data based on the first motion data, the second motion data, and the interpolated motion data; 2. An information processing method implemented by a computer, comprising: (19) On the computer, an acquisition function for acquiring interpolated motion data that is interpolated between first motion data and second motion data that are independent in time and space; a calculation function that calculates an index related to a motion speed of each piece of motion data based on the first motion data, the second motion data, and the interpolated motion data acquired by the acquisition function; A program to achieve this. [Explanation of symbols]

[0195] 1 Network 10 Server 110 Storage section 120 Learning Department 130 Communications Department 20 PC 210 Communications Department 220 Operation display section 230 Storage section 240 Control Unit 241 Generation part 243 Calculation Section

Claims

1. Generate interpolated motion data that interpolates between first motion data and second motion data; generating linked motion data by linking the first motion data, the interpolated motion data, and the second motion data in chronological order; Displaying the linked motion data on a display unit; The first motion data and the second motion data are independent from each other in time and space. method.

2. The first motion data or the second motion data is selected based on a first user operation; the first motion data selected by the first user operation or the second motion data selected by the first user operation is inserted into a timeline displayed on the display unit based on a second user operation; The method of claim 1.

3. The first motion data is inserted into a first section of the timeline, and the second motion data is inserted into a second section different from the first section. The method of claim 2.

4. The second user operation is drag and drop. The method of claim 2.

5. A section from the end position of the first section to the start position of the second section is set as an interpolation section for generating the interpolated motion data. The method of claim 3.

6. An execution button for executing generation of the interpolated motion data is displayed on the display unit, and the interpolated motion data is generated based on an operation input by a user pressing the execution button. The method of claim 1.

7. The first spatial position of the first motion data and the second spatial position of the second motion data are manually set by a user. The method of claim 1.

8. The first spatial position and the second spatial position are expressed in xyz format. The method of claim 7.

9. The interpolation interval is manually set by a user. The method of claim 5.

10. The time-independent state is a state in which the time from the start time to the end time of the first motion data and the time from the start time to the end time of the second motion data do not overlap, The method of claim 1 , wherein the spatially independent state is a state in which a first spatial location of the first motion data and a second spatial location of the second motion data are spatially separated.

11. The end time of the first motion data and the start time of the second motion data are manually set by a user. The method of claim 1.

12. A first avatar reflecting the first motion data or a second avatar reflecting the second motion data is displayed on the display unit. The method of claim 1.

13. Displaying the first avatar or the second avatar at the top of the display unit; displaying a timeline for inserting the first motion data or the second motion data at the bottom of the display unit; The method of claim 12.

14. The interpolated motion data is generated based on interpolation parameters. The method of claim 1.

15. The interpolation parameters include parameters relating to the spatial position of the motion data.

15. The method of claim 14.

16. The interpolation parameters include a parameter related to an interpolation time interpolated by the interpolated motion data.

15. The method of claim 14.

17. The interpolation parameters include parameters relating to the posture of the first motion data and the posture of the second motion data.

15. The method of claim 14.

18. The interpolation parameters include a parameter related to the number of steps between an interpolation start time and an interpolation end time.

15. The method of claim 14.

19. The interpolation parameters include parameters related to random numbers or noise.

15. The method of claim 14.

20. A computer comprising: a function of generating interpolated motion data that interpolates between the first motion data and the second motion data; a function of generating linked motion data by linking the first motion data, the interpolated motion data, and the second motion data in chronological order; a function of displaying the linked motion data on a display unit; To achieve this, The first motion data and the second motion data are independent from each other in time and space. program.

Images