Information processing device, information processing method, and program

Abstract

The present disclosure relates to an information processing device, an information processing method, and a program which make it possible to automatically generate a more appropriate motion. A motion applied to an NPC is determined by: estimating a context estimation value indicating a degree of influence which each of a plurality of motion candidates generated on the basis of interaction information have on the context of a motion that is performed by the NPC; and estimating a degree of suitability of each of the plurality of motion candidates for the context, with reference to at least the context estimation value for each of the plurality of motion candidates and motion context information based on the context. The present technology can be applied to, for example, a system in which a motion to be applied to an NPC in a virtual space such as a game or a metaverse is automatically generated.

Description

Information processing device, information processing method, and program

[0001] The present disclosure relates to an information processing device, an information processing method, and a program, and in particular to an information processing device, an information processing method, and a program that are capable of automatically generating more appropriate motions.

[0002] In recent years, with advances in conversational AI (Artificial Intelligence) technology, development is underway to automatically generate speech for NPCs (Non-Player Characters) in virtual spaces such as games and the Metaverse using conversational AI, and to automatically generate motions to be applied to the NPCs on the spot in response to the speech.

[0003] For example, Patent Document 1 discloses a technique for generating action information indicating actions to be performed by a non-player character from the action history of a player in a virtual space.

[0004] JP 2023-148489 A

[0005] However, with conventional technology, the same motion may be repeatedly applied to an NPC, or a motion may be applied to an NPC that ignores the previous context (e.g., the flow of the dialogue, the context of the previous and following motions as a series of connections, the character of the NPC, etc.) In this way, when motions are automatically generated without taking the context into consideration, the NPC may perform a motion that does not adequately reflect consistency, naturalness, character, etc., which may impair the immersion in the virtual world and the presence of the NPC.

[0006] The present disclosure has been made in light of such circumstances, and makes it possible to automatically generate more appropriate motions.

[0007] An information processing device according to one aspect of the present disclosure includes a motion candidate generation unit that generates a plurality of motion candidates to be applied to an NPC based on dialogue information required for the NPC to perform a motion during dialogue; a motion context estimation unit that estimates a context estimation value indicating the degree of influence that each of the plurality of motion candidates has on the context of the motion performed by the NPC; a motion context compatibility estimation unit that refers at least to the context estimation value for each of the plurality of motion candidates and motion context information based on the context, and estimates a context compatibility for each of the motion candidates indicating its compatibility with the context; and a motion determination unit that determines a motion to be applied to the NPC based on the context compatibility for each of the motion candidates.

[0008] An information processing method or program according to one aspect of the present disclosure includes generating a plurality of motion candidates to be applied to an NPC based on dialogue information required for the NPC to perform a motion during dialogue; estimating a context estimation value indicating the degree of influence that each of the plurality of motion candidates has on the context of the motion performed by the NPC; estimating a context compatibility value for each of the plurality of motion candidates indicating its compatibility with the context by referring to at least the context estimation value for each of the plurality of motion candidates and motion context information based on the context; and determining a motion to be applied to the NPC based on the context compatibility value for each of the motion candidates.

[0009] In one aspect of the present disclosure, multiple motion candidates to be applied to an NPC are generated based on dialogue information required for the NPC during a dialogue to perform a motion, a context estimation value indicating the degree of influence that each of the multiple motion candidates will have on the context of the motion performed by the NPC is estimated, and a context compatibility indicating the compatibility with the context for each motion candidate is estimated by referring to at least the context estimation value for each of the multiple motion candidates and motion context information based on the context, and a motion to be applied to the NPC is determined based on the context compatibility for each motion candidate.

[0010] FIG. 1 is a diagram illustrating a motion that takes context into consideration. FIG. 1 is a block diagram illustrating a configuration example of an embodiment of an automatic motion generation system to which the present technology is applied. FIG. 2 is a diagram illustrating an example of target context information. FIG. 3 is a diagram illustrating a specific example and effect of automatic motion generation processing. FIG. 4 is a flowchart illustrating automatic motion generation processing. FIG. 5 is a flowchart illustrating target context information setting processing. FIG. 6 is a flowchart illustrating motion context information resetting processing. FIG. 7 is a diagram illustrating a modified example in which a motion is determined by reflecting the context of a motion to be executed later. FIG. 8 is a diagram illustrating an application example that devise ways to speed up motion determination. FIG. 9 is a sequence diagram illustrating automatic motion generation processing. FIG. 10 is a sequence diagram illustrating target context information setting processing and motion context information resetting processing. FIG. 11 is a block diagram illustrating a configuration example of an embodiment of a computer to which the present technology is applied.

[0011] Hereinafter, specific embodiments to which the present technology is applied will be described in detail with reference to the drawings.

[0012] <Context-Aware Motion> First, an example of context-aware motion will be described with reference to FIG.

[0013] 1 shows an example of motions executed by an NPC based on four pieces of dialogue information for scenarios A and B. Scenario A is, for example, a farewell scene, such as when someone is leaving town, and is composed of dialogue information A-1 "You're leaving town...", dialogue information A-2 (I'll come visit you once in a while), dialogue information A-3 "See you later", and dialogue information A-4 "Don't catch a cold." Scenario B is, for example, a mundane farewell scene after enjoying a live concert together, and is composed of dialogue information B-1 "Today's concert was fun", dialogue information B-2 "I'm already looking forward to the next concert", dialogue information B-3 "See you later", and dialogue information B-4 "Don't catch a cold."

[0014] As shown in Figure 1A, in scenario A, it is appropriate to apply consistently sad motions A-1 to A-4 to the NPC, which behave in a manner appropriate for a separation scene, in accordance with the dialogue information A-1 to A-4. On the other hand, in scenario B, it is appropriate to apply consistently happy motions B-1 to B-4 to the NPC, which behave in a manner appropriate for an everyday farewell scene, in accordance with the dialogue information B-1 to B-4. In this case, the dialogue information A-3 and A-4 in scenario A and the dialogue information B-3 and B-4 in scenario B have the same content, but it is important that different motions are applied according to each scene.

[0015] For example, even if the dialogue information A-3 and A-4 in scenario A and the dialogue information B-3 and B-4 in scenario B have the same content, if their motions are switched and applied as shown in B of Figure 1, both will give an unnatural impression. In other words, if motions A-3' and A-4' similar to motions B-3 and B-4 are applied in scenario A in response to dialogue information A-3 and A-4, the impression will be that the character has suddenly become more frank. On the other hand, if motions B-3' and B-4' similar to motions A-3 and A-4 are applied in scenario B in response to dialogue information B-3 and B-4, the impression will be that the character has suddenly become ill.

[0016] In this way, even if the content of the dialogue information is the same, it is important for the NPC to perform appropriate motions taking into account the context of each scene, which makes it possible to avoid losing the sense of immersion in the virtual world and the presence of the NPC, etc. Furthermore, by having the NPC perform motions that take the context into consideration, it is possible to avoid giving the user the impression that it is unnatural for the NPC to perform only certain motions in succession, or that the NPC should not perform these motions in succession.

[0017] <Configuration Example of Automatic Motion Generation System> FIG. 2 is a block diagram showing a configuration example of an embodiment of an automatic motion generation system to which the present technology is applied.

[0018] 2 can automatically generate motions to be applied to NPCs in response to a motion request from an NPC control system 12 that controls the dialogue and actions of NPCs in a virtual space such as a game or metaverse. For example, the NPC control system 12 makes a motion request to the motion automatic generation system 11 in a scene where it becomes necessary to apply some motion to a certain NPC in response to an event such as a dialogue in the virtual space.

[0019] The automatic motion generation system 11 includes a storage unit 21 and an automatic motion generation processing unit 22. The storage unit 21 includes a target context storage unit 31 and a motion context storage unit 32. The automatic motion generation processing unit 22 includes a motion candidate generation unit 41, a motion context estimation unit 42, a motion context compatibility estimation unit 43, a motion determination unit 44, and a motion context update unit 45.

[0020] The target context storage unit 31 stores target context information indicating elements that continuously affect multiple motions in accordance with the character characteristics (personality settings) of the NPC. The target context information is input to the target context storage unit 31 from the NPC control system 12. For example, the NPC control system 12 inputs the target context information to the target context storage unit 31 when setting the character characteristics of the NPC or when an event that updates the status of the NPC occurs in the virtual space.

[0021] For example, the target context information may include unique items consisting of elements unique to each NPC, such as gender, personality, and age, elements that vary depending on the NPC's situation, such as emotions, physical strength, and level of busyness, and context items consisting of elements that indicate the susceptibility of these variable elements to change (e.g., the ease with which emotions change), or a combination of these elements.The target context information may also be information obtained by analyzing dialogue information.

[0022] 3 shows an example of target context information. As shown in the figure, the target context information may use age and gender as specific items for each NPC, and may use positivity, joy, sadness, and neutrality as context items for each NPC. The target context information is stored in this data format in the target context storage unit 31. Note that the target context information may have missing values ​​in any of its elements.

[0023] The motion context storage unit 32 stores motion context information indicating elements that provide consistency to a series of motions according to the motion contexts previously executed by the NPC (in accordance with the context). The motion context information can be reset (overwritten) according to motion context reset information input from the NPC control system 12. For example, the NPC control system 12 inputs motion context reset information to the motion context storage unit 32 when an event related to the passage of time in the virtual space occurs, when a scene in the virtual space changes, or when an event that interrupts the time series due to initialization occurs. In addition, the motion context information is updated by the motion context update unit 45 each time a motion to be applied to an NPC is automatically generated, as described below.

[0024] For example, the motion context information may use context items that are common to the target context information (elements that vary depending on the NPC's situation, such as emotion, stamina, and busyness, as well as elements that indicate the ease with which these variable elements change). Also, the motion context information may set individual initial values ​​for each NPC. For example, the motion context information may use information that indicates what expressions have been included in the motions that have been applied to the NPC up to now.

[0025] Here, it is preferable to store the target context information and motion context information in a data format that makes it easy to calculate the degree of similarity between common elements, and for example, it is preferable to store each element of each NPC's context item as a real number greater than or equal to 0 and less than or equal to 1.

[0026] When a motion request is generated by the NPC control system 12, the motion candidate generation unit 41 receives dialogue information from the NPC control system 12, which indicates various pieces of information required for an NPC in dialogue to execute a motion. For example, the dialogue information may include the NPC's own lines, dialogue information of the dialogue partner, motion information of the dialogue partner, information about events that occurred around the NPC during the dialogue, emotional information to be applied to the NPC (e.g., joy, anger, sadness, or happiness), or a combination of these pieces of information. The dialogue information may be in the form of text data or tag information linked to dialogue information, motion information, or the like.

[0027] The motion candidate generation unit 41 then generates a plurality of motion candidates to be applied to the NPC based on the dialogue information, and supplies them to the motion context estimation unit 42. For example, the motion candidate generation unit 41 can use a conventional motion generation method to generate motion candidates representing gestures based on the dialogue information, or to extract motion candidates that match the dialogue information by searching a motion data set. In this case, because the context is taken into account in the subsequent motion context estimation unit 42, the motion candidate generation unit 41 can generate motion candidates without taking the context into account.

[0028] The motion context estimation unit 42 estimates a context estimation value indicating the degree of influence that the motion candidate will have on the previous context when each motion candidate is adopted, for each of the multiple motion candidates supplied from the motion candidate generation unit 41. For example, the context estimation value uses context items (elements that vary depending on the NPC's situation, such as emotion, stamina, and busyness, and elements that indicate the ease of change of these variable elements) that are common to the target context information and the motion context information. The motion context estimation unit 42 then supplies the multiple motion candidates and the context estimation value calculated for each motion candidate to the motion context compatibility estimation unit 43.

[0029] For example, an example will be described in which the element "positive-negative" is used as the context estimation value. If the motion context estimation unit 42 estimates that adopting motion candidate A will give the user a slightly positive impression, it can calculate the context estimation value as "positive: +3." If it estimates that adopting motion candidate B will give the user a significantly negative impression, it can calculate the context estimation value as "positive: -10." Note that the motion context estimation unit 42 can use a conventional estimation method to calculate the context estimation value, and for example, it may calculate the context estimation value based on time-series data of the contents of motion data (such as values ​​and fluctuation amounts of joint angles of the entire body) that indicate the body movements of the NPC in accordance with the motion candidates.

[0030] The motion context compatibility estimation unit 43 refers to the target context information stored in the target context storage unit 31, the motion context information stored in the motion context storage unit 32, and the context estimation value for each motion candidate supplied from the motion context estimation unit 42, and estimates the context compatibility indicating the compatibility with the previous context for each motion candidate supplied from the motion context estimation unit 42.

[0031] For example, if the target context information is "positive: +8", the motion context information is "positive: +6", and the context estimation value of motion candidate A is "positive: +3", the motion context information after executing motion candidate A will transition to "positive: +9". Therefore, in this case, the motion context compatibility estimation unit 43 estimates the context compatibility of motion candidate A to be 90% because the difference with the target context information "positive: +8" is 1.

[0032] In addition, if the target context information is "positive: +8", the motion context information is "positive: +6", and the context estimation value of motion candidate B is "positive: -10", the motion context information after executing motion candidate B will transition to "positive: -4". Therefore, in this case, the motion context compatibility estimation unit 43 estimates the context compatibility of motion candidate A to be 0% because the difference with the target context information "positive: +8" is 12.

[0033] The motion determination unit 44 determines a motion to be applied to the NPC based on the context compatibility of each motion candidate estimated by the motion context compatibility estimation unit 43. For example, the motion determination unit 44 can directly adopt the motion candidate with the highest context compatibility as the motion to be applied to the NPC. Alternatively, the motion determination unit 44 may determine a motion to be applied to the NPC by varying the probability according to the context compatibility so that a motion candidate with a higher context compatibility is most likely to be selected probabilistically.

[0034] Then, the motion determination unit 44 outputs motion information indicating the motion determined to be applied to the NPC to the NPC control system 12, and the NPC control system 12 applies the motion according to the motion information to the NPC and presents it to the user.

[0035] When the motion to be applied to the NPC is determined by the motion determination unit 44, the motion context update unit 45 estimates motion context information after the execution of the motion based on the context estimation value estimated from the motion candidate of the motion and the motion context information before the execution of the motion. For example, the motion context update unit 45 can obtain motion context information after the execution of the motion by adding or subtracting the context estimation value obtained from the motion candidate of the motion to be applied to the NPC to the motion context information before the execution of the motion. Furthermore, the motion context update unit 45 may obtain motion context information after the execution of the motion by changing the degree of updating according to the length of time the motion is actually performed.

[0036] The motion context update unit 45 then updates (overwrites) the motion context information stored in the motion context storage unit 32 with the motion context information after the execution of the motion. As a result, the motion context information after the execution of the motion is referenced in subsequent processing. Note that the process by the motion context update unit 45 to obtain the motion context information after the execution of the motion and update the motion context information stored in the motion context storage unit 32 may be executed in parallel with the process by the NPC control system 12 to apply the motion to the NPC.

[0037] The automatic motion generation system 11 is configured as described above, and can automatically generate more appropriate motions and apply them to NPCs by taking into account the previous context.

[0038] By using the dialogue information described above, the automatic motion generation system 11 can apply motions to the NPC that are in line with the flow of the dialogue. By using the dialogue information including the NPC's own line information and the dialogue information of the dialogue partner, the NPC can execute motions that match the content of the dialogue. By using the dialogue information including the dialogue partner's motion information, the NPC can execute interactive motions in line with the dialogue, such as high-fiving, shaking hands, or waving. By using information related to an event as dialogue information, the NPC can execute motions that take into account the influence of the event, such as being surprised when a surprising event occurs in the surrounding area even during a dialogue.

[0039] By using the target context information (see FIG. 3 ) described above, the automatic motion generation system 11 can apply motions to an NPC according to its state while preserving the NPC's character traits for a series of motions. For example, by using information unique to each NPC, it is possible to implement behaviors in which each NPC performs a different motion. By using elements that change depending on the NPC's situation, it is possible to implement behaviors that adapt to the situation even for the same NPC. By using elements that indicate the ease with which the elements that change depending on the NPC's situation change, it is possible to reflect individual personalities in the NPC, such as a "fickle character," a "character that dwells on one thing for a long time," a "weak character," or a "character with endless stamina."

[0040] Furthermore, the automatic motion generation system 11 can automatically update target context information by estimating target context information from the content of the dialogue. For example, if the dialogue content is an NPC's own line information, and an analysis of the NPC's lines reveals a shift from a sad topic to a happy one, the emotional portion of the NPC's target context information can be overwritten. This allows the NPC to perform motions that follow the content of the lines.

[0041] By using the motion context information described above, the automatic motion generation system 11 can provide consistency to a series of motions when executing the next motion by reflecting the motion context information updated as a result of executing the previous motion. Furthermore, by using context items common to the target context information as the motion context information, it becomes possible to estimate the context compatibility for each motion candidate.

[0042] In addition, the automatic motion generation system 11 can store pre-update motion context information in the motion context storage unit 32 in case, for example, the application of an automatically generated motion is interrupted. This makes it possible to deal with motion interruptions. Furthermore, in the automatic motion generation system 11, when motion context information is reset in response to a scene change, the motion context information before resetting can be stored in the motion context storage unit 32 in anticipation of returning to the original scene. This makes it possible to express motions that match the respective time series even when scenes change back and forth.

[0043] In addition, in the automatic motion generation system 11, by having target context information and motion context information each have a numerical value for each context item of each NPC, the discrepancy between the NPC's character and the NPC's current state (the series of motions up to now) can be used to select motion candidates.

[0044] In the automatic motion generation system 11, the timing at which motion context resetting information is input can be set to when an event related to the passage of time in the virtual space occurs, when a scene in the virtual space changes, or when an event that interrupts the timeline occurs due to initialization, thereby enabling motion control that is not dependent on the previous context in situations where it is not desirable to maintain the context. For example, it becomes possible to execute a motion that suddenly takes into account a context that will affect the motion applied thereafter, such as "a scene starting from an angry and tired state." Furthermore, in the automatic motion generation system 11, by setting different motion context resetting information for each NPC, it becomes possible for that NPC to behave like that NPC immediately after a scene change.

[0045] A specific example and effect of the automatic motion generation process performed by the automatic motion generation processing unit 22 will be described with reference to FIG.

[0046] 4, when the motion candidate generation unit 41 acquires the dialogue information "See you later" from the NPC control system 12, it generates two motion candidates, motion candidate A and motion candidate B, based on the dialogue information. In the example shown, motion candidate A is a happy motion, and motion candidate B is a sad motion.

[0047] For example, the motion candidate generation unit 41 may use a mechanism for extracting multiple motion candidates by searching a motion data set in which a large number of motions are registered for a motion that matches the dialogue information supplied from the NPC control system 12 to a certain degree or higher. Alternatively, the motion candidate generation unit 41 may use a mechanism for generating motion candidates by assigning appropriate motions from the dialogue information using a motion classifier that classifies motions based on deep learning or rules, or that randomly classifies motions. Alternatively, the motion candidate generation unit 41 may dynamically generate motions from the dialogue information and divide them into certain time segments, and use each divided motion as a motion candidate.

[0048] Next, the motion context estimation unit 42 estimates the context estimation values ​​when motion candidate A and motion candidate B are adopted. In the example shown, the context estimation value (Positive = 0.95) when motion candidate A is adopted and the context estimation value (Positive = 0.15) when motion candidate B is adopted are calculated.

[0049] For example, the motion context estimation unit 42 can calculate a rule-based context estimate based on the values ​​and fluctuations of the joint angles of the entire body in motion data indicating the body movements of the NPC in accordance with the motion candidates. In this case, the motion context estimation unit 42 may take into account the weight of each body part divided into joints of the NPC, calculate the amount of energy consumed by the NPC moving in accordance with the motion data, and use the energy consumption to calculate a context estimate. The motion context estimation unit 42 may also calculate a context estimate by focusing on a portion of the motion data (e.g., the percentage of time an arm is raised above the shoulder, or the maximum or median value of the degree to which the corners of the mouth are raised). The motion context estimation unit 42 can also create a neural network that receives time-series data of changes in the joint angles of the entire body included in the motion data as input and outputs a context estimate, and use the neural network to calculate a context estimate.

[0050] The motion context estimation unit 42 may use a conventional method as an estimator for estimating a context estimate. In this case, the context items used as the context estimate must be the same as the context items of the target context information and the motion context information, as described above, and matching between them is essential for subsequent processing.

[0051] In this way, the motion context estimation unit 42 calculates a context estimate for each motion candidate, thereby enabling the selection of a motion that matches the target context information. For example, the motion context estimation unit 42 calculates a context estimate based on rules, enabling the motion of an NPC to be controlled so that it always matches the context according to a certain standard. Furthermore, the motion context estimation unit 27 calculates a context estimate using a neural network, which is expected to enable learning-based calculation of a context estimate that would be difficult to achieve using a rule-based calculation. Furthermore, by calculating a context estimate using energy consumption as described above, it is possible to avoid, for example, the NPC constantly performing large motions, such as jumping around with its entire body, which would make the user feel unhuman, and thereby enable control to apply more natural motions to the NPC.

[0052] Next, the motion context compatibility estimation unit 43 estimates the context compatibility for each motion candidate by referencing the target context information, motion context information, and the context estimation value for each motion candidate. In the illustrated example, the context compatibility (Fitness = 80%) of motion candidate A is estimated based on the target context information (Positive = 0.8), motion context information (Positive = 0.6), and the context estimation value for motion candidate A (Positive = 0.95). Similarly, the context compatibility (Fitness = 20%) of motion candidate B is estimated based on the target context information (Positive = 0.8), motion context information (Positive = 0.6), and the context estimation value for motion candidate B (Positive = 0.15).

[0053] For example, the motion context fitness estimation unit 43 may estimate the context fitness for each motion candidate by increasing the weight of the target context information when emphasizing the consistency of the NPC's character, and by increasing the weight of the motion context information when emphasizing the continuity with the previous motion (consistency with the previous and next motions). Specifically, the motion context fitness estimation unit 43 can calculate the context fitness Fitness according to the following formula (1) using the target context information T_Con., the motion context information M_Con., the context estimate Estimate, and the weight α.

[0054]

[0055] In this formula (1), the weight α is a real number greater than or equal to 0 and less than or equal to 1, and N represents conversion to a normalization function [0, 1], so that the possible values ​​of the context fitness Fitness can be limited to real numbers greater than or equal to 0 and less than or equal to 1. The closer the context fitness Fitness is to 1, the higher the fitness is.

[0056] When a plurality of context items are used for the target context information and the motion context information, the motion context compatibility estimation unit 43 may change the weight applied to each context item.

[0057] In this way, by having the motion context compatibility estimation unit 28 estimate the context compatibility for each motion candidate, it becomes possible to select a motion that takes into consideration consistency in the NPC's character and continuity with previous motions. Furthermore, for example, by setting the weight α according to the length of the motion, it becomes easier to strike a balance between prioritizing consistency in the NPC's character and continuity with previous motions.

[0058] Alternatively, the motion context fitness estimation unit 43 may calculate a target value in a PID control manner using the magnitude of the deviation between the target context information and the motion context information, the integral value of the deviation, the derivative value of the deviation, etc., and determine the proximity of the context estimated value to the target value as the context fitness. Specifically, the motion context fitness estimation unit 43 can calculate the context fitness Fitness according to the following equation (2) using the target context information T_Con., the motion context information M_Con., and the target value PID of the PID control.

[0059]

[0060] Note that the context fitness Fitness may be a real number greater than or equal to 0 and less than or equal to 1, or may be expressed as a percentage between 0 and 100%. In this way, by using PID control, it is possible to apply not only the current deviation but also the change in deviation over time between the target context information T_Con. and the motion context information M_Con. This makes it possible to expect behavior such as, for example, selecting motion context information M_Con. that exceeds the target context information T_Con. because it has been below the target context information T_Con. for a long time, rather than simply obtaining a context fitness Fitness that simply brings it closer to the target context information T_Con.

[0061] When multiple context items are used for the target context information and the motion context information, the motion context compatibility estimation unit 43 can control the influence of prioritized and non-prioritized elements among the multiple context items by changing the degree of influence that each matching degree has on the estimation of context compatibility. For example, if any element of a context item has a missing value, that element can be ignored in the calculation of context compatibility. For example, if there is a motion expression that should be avoided due to the character traits of an NPC, it becomes possible to select a motion that avoids that expression.

[0062] Furthermore, the context suitability may be used as an index that indicates the order of suitability when comparing a plurality of motion candidates.

[0063] Next, the motion determination unit 44 determines a motion to be applied to the NPC based on the context suitability of each motion candidate. In the illustrated example, motion candidate A is determined as the motion to be applied to the NPC based on the context suitability of motion candidate A (Fitness = 80%) and the context suitability of motion candidate B (Fitness = 20%).

[0064] In this way, the motion determination unit 44 can determine a motion to be applied to an NPC from among a plurality of motion candidates, reflecting the context compatibility of each motion candidate. Also, when the motion determination unit 44 determines a motion probabilistically using the context compatibility, a motion candidate with low context compatibility may be selected, which can result in variations in the behavior of an NPC even if the dialogue information and motion context information are exactly the same.

[0065] Then, the motion determination unit 44 outputs the motion information of the motion to be applied to the NPC to the NPC control system 12 .

[0066] For example, motion information is information for making an NPC behave in a conversation with a player by expressing physical expressions, emotions, etc., and can use joint angle and position information for moving the NPC's body (2D or 3D). Specifically, data in FBX (Film Box) format or BVH (Biovision Hierarchy) format can be used as motion information. Furthermore, parameters for controlling the NPC's facial expressions can be used as motion information; specifically, Blendshape information for a character model can be used. Alternatively, cartoon symbols (such as exclamation marks and question marks) that express the NPC's emotions can be used as motion information.

[0067] In this way, the motion determination unit 44 outputs motion information that applies a motion to the NPC according to the dialogue, allowing the user to understand which NPC the user is conversing with in the virtual space. Then, the NPC executes a motion that is automatically generated based on the dialogue information, allowing the user to easily recognize the content and intent of the NPC's speech.

[0068] For example, if the NPC is 2D, it is possible to visually express a variety of emotions and user interactions through animation or manga-like expressions, even if the NPC itself does not have a great degree of freedom in its expressiveness. Also, if the NPC is 3D, it can function as a dialogue interface similar to the real world, providing the user with natural interactions with the NPC.

[0069] Next, the motion context update unit 45 calculates motion context information after the execution of the motion based on the context estimate value obtained from the motion candidate of the motion determined by the motion determination unit 44 and the motion context information at the time point before the execution of the motion. Specifically, the motion context compatibility estimation unit 43 can calculate motion context information after the execution of the motion according to the following equation (3) using the context estimate value next_motion_context obtained from the motion candidate of the motion to be applied to the NPC, the motion context information before the execution of the motion, and the weight α.

[0070]

[0071] In this equation (3), the weight α is a real number greater than or equal to 0 and less than or equal to 1, and the closer the weight α is to 1, the more likely the motion context information after the execution of the motion is to fluctuate.

[0072] The motion context update unit 45 may also change the degree to which the motion context information is updated depending on the length of the motion. For example, it may be possible to change the motion context information to a greater extent for a long motion and to a lesser extent for a short motion. This makes it possible to change the motion context information in accordance with the duration of the motion (the duration of the motion's display).

[0073] In this way, the motion context update unit 45 updates the motion context information, thereby updating the degree of influence of the motion applied to the NPC on the motion context information. This makes it possible to calculate, for example, the degree of influence that a newly executed motion will have on the motion context information, and to control whether to prioritize continuity with the previous motion or consistency with the NPC's character.

[0074] <Processing Examples of Automatic Motion Generation Processing, Target Context Information Setting Processing, and Motion Context Information Resetting Processing> Processing executed in the automatic motion generation system 11 will be described with reference to the flowcharts shown in FIGS.

[0075] FIG. 5 is a flowchart illustrating the automatic motion generation process.

[0076] For example, the automatic motion generation process is started in response to a motion request from the NPC control system 12, and in step S11, dialogue information is input from the NPC control system 12 to the automatic motion generation system 11.

[0077] In step S12, the motion candidate generation unit 41 generates a plurality of motion candidates to be applied to the NPC based on the dialogue information input in step S11, and supplies the motion candidates to the motion context estimation unit .

[0078] In step S13, the motion context estimation unit 42 estimates the influence that each of the motion candidates provided in step S12 will have on the current context if that motion candidate is adopted, and calculates a context estimation value indicating the degree of influence.The motion context estimation unit 42 then provides the motion candidates and the context estimation value calculated for each of the motion candidates to the motion context compatibility estimation unit 43.

[0079] In step S14, the motion context compatibility estimation unit 43 references the target context information stored in the target context storage unit 31, the motion context information stored in the motion context storage unit 32, and the context estimation value for each motion candidate supplied in step S13, and estimates the context compatibility for each motion candidate supplied in step S13. Then, the motion context compatibility estimation unit 43 supplies the context compatibility for each motion candidate to the motion determination unit 44.

[0080] In step S15, the motion determination unit 44 determines a motion to be applied to the NPC based on the context suitability for each motion candidate supplied in step S14.

[0081] In step S16, the motion determination unit 44 outputs motion information indicating the motion determined in step S15 to the NPC control system 12, and the NPC control system 12 applies a motion according to the motion information to the NPC. In parallel with the processing of step S16, the processing of steps S17 and S18 is performed.

[0082] In step S17, the motion context update unit 45 obtains motion context information after the execution of the motion from the context estimates of the motion candidates for the motion determined in step S15 and the motion context information before the execution of the motion.

[0083] In step S18, the motion context update unit 45 updates (overwrites) the motion context information stored in the motion context storage unit 32 with the motion context information after execution of the motion obtained in step S17.

[0084] After the processing of steps S16 and S18, the automatic motion generation processing ends.

[0085] By executing the automatic motion generation process described above, the automatic motion generation system 11 can automatically generate more appropriate motions and apply them to the NPC, taking into account the context up to that point.

[0086] For example, in the automatic motion generation process, by managing the elements and attributes of an NPC that act across multiple lines as target context information and motion context information, it is possible to ensure consistency in the series of consecutive actions taken by the NPC. Furthermore, by ensuring consistency in the NPC's actions, it is possible to present the NPC's behavior to the user who is presented with it as having a consistent consciousness and personality. Furthermore, it is possible to reduce the impression that the NPC's behavior is being manipulated by the game system or the virtual space program, thereby preventing the sense of immersion in the virtual space itself from being impaired.

[0087] For example, the more motion candidates generated in step S12, the more diverse dialogue expressions an NPC can have, but it is expected that it will be more difficult to select a motion appropriate for the context of the dialogue from among them. In contrast, the automatic motion generation process makes it difficult for an NPC to execute a motion that is inappropriate for its behavior during dialogue, even if the number of motion candidates is increased. This can improve the diversity of NPC expressions during dialogue.

[0088] Furthermore, in step S14, by estimating the context compatibility by referring to target context information including information related to the personality of the NPC, it is possible to prevent the occurrence of behavior that would cause a sudden change in the personality of the NPC.

[0089] For example, if the same dialogue information is input in step S11, the same motion candidates are generated in step S12, but by setting different elements for each NPC as the target context information referenced in step S14, it is possible to select different motions from the motion candidates for each NPC. This makes it possible to express a unique character for each NPC.

[0090] In the automatic motion generation process, the motion context information stored in the motion context storage unit 32 is updated with the post-execution motion context information obtained based on the motion applied to the NPC, so that when the next motion is requested, the motion to be applied to the NPC can be determined by referring to the post-execution motion context information.

[0091] FIG. 6 is a flowchart illustrating the target context information setting process.

[0092] For example, the target context information setting process is started when the character characteristics of an NPC are set or when an event that updates the state of an NPC occurs in the virtual space.

[0093] In step S21, target context information is input from the NPC control system 12 to the automatic motion generation system 11.

[0094] In step S22, the target context storage unit 31 stores (or updates) the target context information input in step S21, and then the target context information setting process is terminated.

[0095] By executing the target context information setting process as described above, the target context information that is referenced when estimating the context compatibility for each motion candidate in step S14 of the automatic motion generation process in Fig. 5 can be stored in the target context storage unit 31. This makes it possible to select a different motion for each NPC in the automatic motion generation process, or to change the policy for selecting motion depending on the state of the NPC, making it possible to express the character traits of the NPC, the inner thoughts of the NPC, and so on.

[0096] For example, when the element "positive-negative" is used as the target context information, if a certain NPC has a very positive character trait, the target context information can be set to "positive: +10." Furthermore, if an event occurs in the virtual space and the scene shows the NPC feeling sad, the target context information can be changed to "positive: -2."

[0097] The target context storage unit 31 can store not only one element as target context information but also a plurality of elements.

[0098] FIG. 7 is a flowchart illustrating the motion context information resetting process.

[0099] For example, the motion context information resetting process is started when an event occurs related to the passage of time in the virtual space, when an event occurs that switches scenes in the virtual space, when an event occurs that interrupts the timeline due to initialization, etc. Specifically, the motion context information resetting process can be started when, during a conversation with the user, a scene occurs in which the NPC recalls a past event.

[0100] In step S31, the NPC control system 12 inputs motion context resetting information to the automatic motion generation system 11.

[0101] In step S32, the motion context storage unit 32 resets the motion context information in accordance with the motion context resetting information input in step S31. At this time, the motion context storage unit 32 may retain the motion context information before resetting. Thereafter, the motion context information resetting process ends.

[0102] By executing the above-described motion context information resetting process, it becomes possible to overwrite the motion context information, and in addition to the effect of using the target context information described above, it becomes possible to respond to events that change scenes or events that transition the timeline, such as recollections of the past. Furthermore, by retaining the motion context information before it was overwritten, it is possible to resume the behavior of the NPC in the state it was in before the time change. This makes it possible to respond to time changes in the virtual world in which the NPC exists.

[0103] When the scene changes, including the flow of time, the next motion will no longer depend on the context, so the motion context information may be reset to the default value. Also, if there is some special event after the scene changes, motion context information corresponding to that event may be input instead of the default value.

[0104] <Modifications and Applications> Modifications and applications of the automatic motion generation process executed in the automatic motion generation system 11 will be described.

[0105] With reference to FIG. 8, a modified example in which a motion is determined by reflecting the context of a motion to be executed later will be described.

[0106] For example, when a series of lines to be spoken by an NPC has been determined, or when the next line to be spoken by an NPC in a dialogue can be predicted, the automatic motion generation system 11 can acquire in advance line information indicating lines that will be spoken in the future rather than the current line. Therefore, in this case, the automatic motion generation system 11 assumes that multiple motion candidates have been executed in accordance with the current line (dialogue information) for which a motion is to be automatically generated, and that future motion candidates have been executed following each motion candidate. Then, based on the context suitability estimated from the series of motions, the automatic motion generation system 11 can select the optimal motion to be actually executed by the NPC from the multiple motion candidates corresponding to the current line.

[0107] For example, in the example shown in FIG. 8, the automatic motion generation system 11 acquires line information 1 to 4 as a series of lines uttered by an NPC.

[0108] First, in the automatic motion generation system 11, motion 1 is applied to the NPC according to line information 1, and then the motion candidate generation unit 41 generates two motion candidates, motion 2a and motion 2b, according to line information 2 for which motions are to be automatically generated. At this time, it is initially assumed that motion candidate 2a and motion candidate 2b are respectively executed by the NPC.

[0109] Then, assuming that motion candidate 3a corresponding to dialogue information 3 is executed as a secondary hypothesis following motion candidate 2a, and that motion candidate 4a corresponding to dialogue information 4 is executed as a tertiary hypothesis following motion candidate 3a, the motion context compatibility estimation unit 43 calculates the average value of the context compatibility estimated from motion candidates 2a to 4a. In the example shown in Fig. 8, the average value of the context compatibility calculated from the motion candidate time series of motion candidates 2a to 4a is 0.8.

[0110] Similarly, assuming that motion candidate 3b corresponding to dialogue information 3 is executed as a secondary hypothesis following motion candidate 2b, and that motion candidate 4b corresponding to dialogue information 4 is executed as a tertiary hypothesis following motion candidate 3b, the motion context compatibility estimation unit 43 calculates the average value of the context compatibility estimated from motion candidates 2b to 4b. In the example shown in Fig. 8, the average value of the context compatibility calculated from the motion candidate time series of motion candidates 2b to 4b is 0.65.

[0111] Therefore, in this example, since the average value of the context compatibility obtained from the motion candidate time series consisting of motion candidates 2a to 4a is high, motion candidate 2a is selected from the two motion candidates 2a and 2b as the most suitable motion to actually be performed by the NPC.

[0112] In this way, the automatic motion generation system 11 can reflect the context of a motion to be executed later by selecting a motion that corresponds to the current line based on the context suitability determined from the motion candidate time series of multiple motion candidates generated by repeating assumptions. This allows the automatic motion generation system 11 to automatically generate an appropriate motion that matches the context surrounding a series of lines, rather than determining a motion based on the motions that have been executed up to that point.

[0113] If it is assumed that all motion candidates generated by the motion candidate generation unit 41 have been executed, the number of motion candidate time series to be considered will increase exponentially, so it is preferable to limit the number of assumptions, the number of assumed motion candidates, etc. This allows the automatic motion generation system 11 to appropriately adjust the range of consideration of motions to be determined at the present time while also taking future motions into consideration.

[0114] An application example for speeding up motion determination will be described with reference to FIG.

[0115] For example, in the automatic motion generation system 11, the motion candidate generation unit 41 can generate only one motion candidate in principle. If the context compatibility estimated from that one motion candidate is equal to or greater than a predetermined threshold, that motion candidate is determined as the motion to be applied to the NPC as is. In other words, in this case, since the motion context information obtained from that one motion candidate does not deviate from the target context information, applying it to the NPC as is allows the execution of a motion that takes context into consideration.

[0116] On the other hand, if the context compatibility estimated from that one motion candidate is less than a predetermined threshold, the motion candidate generation unit 41 generates a second motion candidate (hereinafter referred to as a second-generation motion candidate). If the context compatibility estimated from the second-generation motion candidate is equal to or greater than the predetermined threshold, the motion candidate is determined to be the motion to be applied to the NPC. On the other hand, if the context compatibility estimated from the second-generation motion candidate is less than the predetermined threshold, a third motion candidate (hereinafter referred to as a third-generation motion candidate) is generated, and the same process is repeated until the context compatibility reaches or exceeds the predetermined threshold.

[0117] 9, the context compatibility estimated from the motion candidate generated according to dialogue information 1 is 90%, and motion 1 of the motion candidate is applied to the NPC as is. Similarly, the context compatibility estimated from the motion candidate generated according to dialogue information 2 is 85%, and motion 2 of the motion candidate is applied to the NPC as is.

[0118] On the other hand, if the context compatibility estimated from the motion candidate generated according to dialogue information 3 is 50%, a second generation of motion candidates is generated. Then, the context compatibility estimated from the second generation of motion candidates is 85%, and motion candidate 3' is applied to the NPC. Then, if the context compatibility estimated from the motion candidate generated according to dialogue information 4 is 80%, motion candidate 4 is applied to the NPC as is.

[0119] In this way, if the context compatibility estimated from the motion candidate is low, the automatic motion generation system 11 can repeatedly generate a second generation of motion candidate, then a third generation of motion candidate, and ultimately have the NPC perform a motion with high context compatibility.

[0120] Furthermore, the automatic motion generation system 11 can reduce the time required for the motion candidate generation unit 41 to generate a motion candidate by, in principle, generating only one motion candidate, thereby speeding up motion determination.

[0121] The automatic motion generation system 11 may set a limit to the number of times that motion candidate generation is repeated, and select the motion candidate with the highest context compatibility from among the motion candidates generated up to that limit. This ensures that the NPC will always perform some kind of motion.

[0122] When a motion candidate with low context compatibility is generated, the automatic motion generation system 11 feeds back information on how the motion candidate was generated, so that a motion candidate with high context compatibility will be generated next. For example, if compatibility with the target context information is low due to insufficient positivity, a motion candidate with a higher positivity can be generated next. This makes it easier to generate motion candidates with higher context compatibility, and reduces the number of motion candidates to be generated.

[0123] Furthermore, for example, the automatic motion generation system 11 can reflect the mental age of the NPC in the motion.

[0124] In the automatic motion generation system 11, when the motion context compatibility estimation unit 43 estimates the context compatibility, the smaller the difference between the current motion context information and the context estimation value for each motion candidate, the higher the context compatibility is, thereby making it possible to present an NPC with a character that has a strong and consistent ego and is less susceptible to the influence of momentary dialogue events.

[0125] Conversely, when the motion context compatibility estimation unit 43 estimates the context compatibility, it can portray an NPC as having a moody character by not placing importance on the difference between the current motion context information and the context estimation value for each motion candidate, or by making the context compatibility higher the larger the difference.

[0126] In this way, by changing the importance of the difference between the current motion context information and the context estimation value for each motion candidate based on the age set for the NPC, the mental age of the NPC can be reflected in the motion. Furthermore, by reflecting changes in the age of the NPC in the importance of this difference, it is possible to reflect in the motion the ease with which the NPC's state of mind changes according to age.

[0127] Furthermore, for example, the automatic motion generation system 11 can control the amount of energy consumption, thereby making it possible for the NPC to perform motions that express natural human-like behavior.

[0128] The automatic motion generation system 11 uses the amount of energy consumed by an NPC performing a motion as motion context information, and the amount of energy that can be consumed by each NPC per unit time as target context information. Then, using such target context information and target context information, the motion context compatibility estimation unit 43 can estimate the context compatibility.

[0129] This allows the NPC to perform motions that express natural human behavior, suppressing unnatural human behavior, such as repeatedly selecting large motions. Conversely, the energy consumption context can also be used when you want to give the user an energetic impression or when you want to enhance the impression of a character with a non-human motif.

[0130] Furthermore, for example, the automatic motion generation system 11 can cause the NPC to perform a motion that expresses turn-taking during a conversation with the user.

[0131] The automatic motion generation system 11 adjusts the target context information based on information indicating whether the user or the NPC will speak during a conversation with the user, so that when the NPC speaks, a motion candidate resulting in a larger movement can be selected. By adjusting the target context information in this way, when the NPC speaks, a motion resulting in a larger movement can be performed by the NPC, and an expression that attracts the user's attention can be made.

[0132] On the other hand, when the user speaks, the target context information can be adjusted so that motion candidates resulting in smaller movements are selected. By adjusting the target context information in this way, when the user speaks, the NPC can perform motions resulting in smaller movements, and expressions that do not interfere with the user's speech can be made.

[0133] In this way, when an NPC speaks, it makes a large motion to attract the user's attention, and when the user speaks, it makes a small, discreet motion, thereby expressing turn-taking during the conversation. This makes it possible to guide the overall flow of the conversation between the user and the NPC through the selection of motions.

[0134] <Processing Examples of Automatic Motion Generation Processing, Target Context Information Setting Processing, and Motion Context Information Resetting Processing> Processing executed in the automatic motion generation system 11 will be described with reference to the sequence diagrams shown in FIGS.

[0135] FIG. 10 is a sequence diagram illustrating the automatic motion generation process.

[0136] In step S41, the NPC control system 12 determines whether a motion request has been made to the automatic motion generation system 11. For example, the NPC control system 12 determines that a motion request has been made to the automatic motion generation system 11 in a scene where it becomes necessary to apply some kind of motion to a certain NPC in response to an event such as a conversation in the virtual space.

[0137] Then, in step S41, the process waits until it is determined that a motion request has been made to the automatic motion generation system 11, and when it is determined that a motion request has been made to the automatic motion generation system 11, the process proceeds to step S42.

[0138] In step S42, the NPC control system 12 inputs dialogue information accompanying the motion request to the automatic motion generation processing unit 22.

[0139] In step S43, in the automatic motion generation processing unit 22, the motion candidate generation unit 41 generates multiple motion candidates to be applied to the NPC based on the dialogue information input in step S42, and supplies them to the motion context estimation unit 42.

[0140] In step S44, the motion automatic generation processing unit 22 causes the motion context estimation unit 42 to obtain a context estimation value for each of the plurality of motion candidates supplied in step S43, and supplies the obtained value to the motion context compatibility estimation unit 43.

[0141] In step S45 , in the automatic motion generation processing unit 22 , the motion context compatibility estimation unit 43 reads out the target context information from the target context storage unit 31 .

[0142] In step S46, in the automatic motion generation processing unit 22, the motion context compatibility estimation unit 43 reads out the motion context information from the motion context storage unit 32.

[0143] In step S47, the motion context compatibility estimation unit 43 refers to the target context information read in step S45, the motion context information read in step S46, and the context estimation value for each motion candidate supplied in step S44, and estimates the context compatibility for each motion candidate supplied in step S44. Then, the motion context compatibility estimation unit 43 supplies the context compatibility for each motion candidate to the motion determination unit 44.

[0144] In step S48, the motion determination unit 44 determines a motion to be applied to the NPC based on the context suitability for each motion candidate supplied in step S47, and outputs the motion information to the NPC control system 12.

[0145] In step S49, the NPC control system 12 applies to the NPC a motion according to the motion information output from the motion determination unit 44 in step S48.

[0146] In step S50, in the automatic motion generation processing unit 22, the motion context update unit 45 obtains motion context information after the execution of the motion from the context estimation value of the motion candidate for the motion determined in step S47 and the motion context information at the time before the motion is executed.

[0147] In step S51, the motion context update unit 45 updates the motion context information stored in the motion context storage unit 32 with the motion context information after execution of the motion obtained in step S50.

[0148] Then, the automatic motion generation process is completed.

[0149] FIG. 11 is a sequence diagram illustrating the target context information setting process and the motion context information resetting process.

[0150] In step S61, the NPC control system 12 determines whether an event that updates the status of an NPC has occurred in the virtual space, for example. Then, in step S61, the processing waits until it is determined that an event that updates the status of an NPC has occurred in the virtual space. If it is determined that an event that updates the status of an NPC has occurred in the virtual space, the processing proceeds to step S62.

[0151] In step S62, the NPC control system 12 inputs the target context information to the target context storage unit 31.

[0152] In step S62, the target context storage unit 31 updates the stored target context information in accordance with the target context information input in step S62, and notifies the NPC control system 12 that the processing has been performed.

[0153] Then, in step S71, the NPC control system 12 determines whether an event that changes the scene in the virtual space has occurred, for example. Then, in step S71, the processing waits until it is determined that an event that changes the scene in the virtual space has occurred, and if it is determined that an event that changes the scene in the virtual space has occurred, the processing proceeds to step S72.

[0154] In step S72, the NPC control system 12 inputs the motion context resetting to the motion context storage unit 32.

[0155] In step S73, the motion context storage unit 32 resets the stored motion context information in accordance with the motion context resetting input in step S72, and notifies the NPC control system 12 that this processing has been performed.

[0156] Thereafter, the target context information setting process and the motion context information resetting process are terminated.

[0157] 1, but may be configured to include at least a motion candidate generation unit 41, a motion context estimation unit 42, a motion context compatibility estimation unit 43, and a motion determination unit 44. The motion context compatibility estimation unit 43 may estimate the context compatibility by referring to at least the motion context information.

[0158] <Example of Computer Configuration> Next, the above-described series of processes (information processing method) can be performed by hardware or software. When the series of processes is performed by software, a program constituting the software is installed in a general-purpose computer or the like.

[0159] FIG. 12 is a block diagram showing an example of the configuration of an embodiment of a computer in which a program for executing the above-described series of processes is installed.

[0160] The program can be recorded in advance on the hard disk 105 or ROM 103 as a recording medium built into the computer.

[0161] Alternatively, the program can be stored (recorded) on a removable recording medium 111 driven by the drive 109. Such a removable recording medium 111 can be provided as a so-called package software. Here, examples of the removable recording medium 111 include a flexible disk, a CD-ROM (Compact Disc Read Only Memory), an MO (Magneto Optical) disk, a DVD (Digital Versatile Disc), a magnetic disk, and a semiconductor memory.

[0162] The program can be installed into the computer from the removable recording medium 111 as described above, or can be downloaded to the computer via a communication network or a broadcasting network and installed on the built-in hard disk 105. That is, the program can be transferred to the computer wirelessly from a download site via an artificial satellite for digital satellite broadcasting, or transferred to the computer by wire via a network such as a LAN (Local Area Network) or the Internet.

[0163] The computer includes a CPU (Central Processing Unit) 102 , to which an input / output interface 110 is connected via a bus 101 .

[0164] When a user inputs a command via the input / output interface 110 by operating the input unit 107, the CPU 102 executes a program stored in a read-only memory (ROM) 103 in accordance with the command. Alternatively, the CPU 102 loads a program stored on a hard disk 105 into a random access memory (RAM) 104 and executes the program.

[0165] As a result, the CPU 102 performs processing according to the flowchart described above or processing performed by the configuration of the block diagram described above. Then, the CPU 102 outputs the processing results from the output unit 106 via the input / output interface 110, transmits them from the communication unit 108, or records them on the hard disk 105, as necessary.

[0166] The input unit 107 is made up of a keyboard, a mouse, a microphone, etc. The output unit 106 is made up of an LCD (Liquid Crystal Display), a speaker, etc.

[0167] In this specification, the processing performed by a computer according to a program does not necessarily have to be performed in chronological order according to the order described in the flowchart. In other words, the processing performed by a computer according to a program also includes processing that is executed in parallel or individually (for example, parallel processing or object-based processing).

[0168] The program may be processed by a single computer (processor), or may be distributed among multiple computers. Furthermore, the program may be transferred to and executed on a remote computer.

[0169] Furthermore, in this specification, a system refers to a collection of multiple components (devices, modules (components), etc.), regardless of whether all of the components are contained in the same housing. Therefore, multiple devices housed in separate housings and connected via a network, and a single device housed in a single housing with multiple modules, are both systems.

[0170] Also, for example, a configuration described as one device (or processing unit) may be divided and configured as multiple devices (or processing units). Conversely, configurations described above as multiple devices (or processing units) may be combined and configured as one device (or processing unit). Of course, configurations other than those described above may be added to the configuration of each device (or each processing unit). Furthermore, as long as the configuration and operation of the entire system are substantially the same, part of the configuration of one device (or processing unit) may be included in the configuration of another device (or other processing unit).

[0171] Furthermore, for example, the present technology can be configured as a cloud computing system in which a single function is shared and processed collaboratively by a plurality of devices via a network.

[0172] Furthermore, for example, the above-described program can be executed in any device, as long as the device has the necessary functions (functional blocks, etc.) and can obtain the necessary information.

[0173] Also, for example, each step described in the above flowchart can be executed by one device or can be shared and executed by multiple devices. Furthermore, if one step includes multiple processes, the multiple processes included in that one step can be executed by one device or can be shared and executed by multiple devices. In other words, multiple processes included in one step can be executed as multiple step processes. Conversely, processes described as multiple steps can be executed collectively as a single step.

[0174] In addition, the processing of the steps of a program executed by a computer may be executed in chronological order according to the order described in this specification, or may be executed in parallel or individually at the required timing, such as when a call is made. In other words, as long as no contradiction occurs, the processing of each step may be executed in an order different from the order described above. Furthermore, the processing of the steps of this program may be executed in parallel with the processing of another program, or may be executed in combination with the processing of another program.

[0175] It should be noted that the present technologies described in this specification can be implemented independently and singly, unless a contradiction arises. Of course, any two or more of the present technologies can also be implemented in combination. For example, part or all of the present technologies described in any embodiment can be implemented in combination with part or all of the present technologies described in other embodiments. Furthermore, part or all of any of the present technologies described above can also be implemented in combination with other technologies not described above.

[0176] <Examples of Combinations of Configurations> The present technology can also be configured as follows: (1) An information processing device comprising: a motion candidate generation unit that generates a plurality of motion candidates to be applied to a Non-Player Character (NPC) during a conversation, based on dialogue information required for the NPC to perform a motion; a motion context estimation unit that estimates a context estimation value indicating the degree of influence that each of the plurality of motion candidates has on a context of a motion performed by the NPC; a motion context compatibility estimation unit that references at least the context estimation value for each of the plurality of motion candidates and motion context information based on the context, and estimates a context compatibility for each of the motion candidates indicating compatibility with the context; and a motion determination unit that determines a motion to be applied to the NPC based on the context compatibility for each of the motion candidates. (2) The information processing device described in (1) above, wherein the motion context compatibility estimation unit estimates the context compatibility by further reference to target context information based on personality settings for each of the NPCs. (3) The information processing device according to (2) above, wherein the dialogue information uses at least one or more of the following information required to cause the NPC to make a motion: line information of the NPC itself, line information of a dialogue partner, motion information of the dialogue partner, information about events that occurred around the NPC during the dialogue, and emotional information to be applied to the NPC, or a combination of these pieces of information. (4) The information processing device according to (2) above, wherein the motion context information indicates an element that gives consistency to a series of motions in accordance with the context, and the target context information indicates an element that continuously affects a plurality of motions in accordance with the personality setting of each of the NPCs.(5) The information processing device according to any one of (2) to (4) above, wherein the target context information uses at least one of a unique item consisting of an element unique to each NPC, an element that varies depending on the situation of the NPC, and a context item consisting of an element indicating the ease of change of the variable element, or a combination of these elements. (6) The information processing device according to any one of (2) to (5) above, wherein the target context information is input when the personality of the NPC is set or when an event that updates the state of the NPC occurs. (7) The information processing device according to any one of (2) to (6) above, wherein the motion context information is reset when an event related to the passage of time occurs, when a scene changes, or when an event that causes initialization to interrupt a time series occurs. (8) The information processing device according to any one of (2) to (7) above, wherein elements common to both the target context information and the motion context information are used. (9) The information processing device according to any one of (1) to (8), further comprising: a motion context update unit that, when a motion to be applied to the NPC is determined, updates the motion context information with the context estimation value estimated from the motion candidate of the motion and motion context information after execution of the motion estimated based on the motion context information at a point in time before the motion is executed. (10) The information processing device according to any one of (1) to (9), further comprising: a motion context update unit that, when a motion to be applied to the NPC is determined, updates the motion context information with the context estimation value estimated from the motion candidate of the motion and motion context information after execution of the motion estimated based on the motion context information at a point in time before the motion is executed.(11) The information processing device according to any one of (1) to (10) above, wherein the motion candidate generation unit generates one of the motion candidates, and if the context compatibility estimated from the one motion candidate is equal to or greater than a predetermined threshold, the motion candidate is determined to be a motion to be applied to the NPC, and if the context compatibility estimated from the one motion candidate is less than the predetermined threshold, the generation of the motion candidates is repeated until the context compatibility becomes equal to or greater than the predetermined threshold. (12) The information processing device according to (2) above, wherein the target context information is adjusted so that the motion candidate resulting in a larger movement is selected when the NPC speaks during a dialogue. (13) An information processing method including: an information processing device generating a plurality of motion candidates to be applied to an NPC (Non-Player Character) during a dialogue based on dialogue information required for the NPC to perform a motion; estimating a context estimation value indicating the degree of influence that each of the plurality of motion candidates has on the context of the motion performed by the NPC; estimating a context compatibility level for each of the plurality of motion candidates indicating compatibility with the context by referring to at least the context estimation value for each of the plurality of motion candidates and motion context information based on the context; and determining a motion to be applied to the NPC based on the context compatibility level for each of the motion candidates.(14) A program for causing a computer of an information processing device to execute information processing including: generating a plurality of motion candidates to be applied to an NPC (Non-Player Character) during a dialogue based on dialogue information required for the NPC to execute a motion; estimating a context estimation value indicating the degree of influence that each of the plurality of motion candidates has on the context of the motion executed by the NPC; estimating a context compatibility level for each of the plurality of motion candidates indicating compatibility with the context by referring at least to the context estimation value for each of the plurality of motion candidates and motion context information based on the context; and determining a motion to be applied to the NPC based on the context compatibility level for each of the motion candidates.

[0177] It should be noted that the present embodiment is not limited to the above-described embodiment, and various modifications are possible within the scope of the gist of the present disclosure. Furthermore, the effects described in this specification are merely examples and are not intended to be limiting, and other effects may also be obtained.

[0178] REFERENCE SIGNS LIST 11 Automatic motion generation system, 12 NPC control system, 21 Memory unit, 22 Automatic motion generation processing unit, 31 Target context memory unit, 32 Motion context memory unit, 41 Motion candidate generation unit, 42 Motion context estimation unit, 43 Motion context compatibility estimation unit, 44 Motion determination unit, 45 Motion context update unit

Claims

1. An information processing device comprising: a motion candidate generation unit that generates multiple motion candidates to be applied to an NPC (Non-Player Character) during a dialogue based on dialogue information required for the NPC to perform a motion; a motion context estimation unit that estimates a context estimation value indicating the degree of influence that each of the multiple motion candidates will have on the context of the motion performed by the NPC; a motion context compatibility estimation unit that references at least the context estimation value for each of the multiple motion candidates and motion context information based on the context, and estimates a context compatibility level for each of the motion candidates indicating its compatibility with the context; and a motion determination unit that determines a motion to be applied to the NPC based on the context compatibility level for each of the motion candidates.

2. The information processing device according to claim 1, wherein the motion context compatibility estimation unit estimates the context compatibility by further referring to target context information based on the personality settings of each NPC.

3. The information processing device according to claim 2, wherein the dialogue information includes at least one of the following information required to cause the NPC to make a motion: line information of the NPC itself, line information of the dialogue partner, motion information of the dialogue partner, information about events that occurred around the NPC during the dialogue, and emotional information to be applied to the NPC, or a combination of these pieces of information.

4. An information processing device as described in claim 2, wherein the motion context information indicates elements that give consistency to a series of motions according to the context, and the target context information indicates elements that continuously influence multiple motions in accordance with the personality settings of each NPC.

5. The information processing device of claim 4, wherein the target context information uses at least one of the following: a unique item consisting of elements unique to each NPC; an element that changes depending on the situation of the NPC; and a context item consisting of an element that indicates the ease of change of the variable element; or a combination of these elements.

6. The information processing device according to claim 4, wherein the target context information is input when the personality of the NPC is set or when an event occurs that updates the state of the NPC.

7. The information processing device according to claim 4, wherein the motion context information is reset when an event related to the passage of time occurs, when a scene changes, or when an event occurs that interrupts the time series due to initialization.

8. The information processing device according to claim 4, wherein the target context information and the motion context information use elements that are common to each other.

9. The information processing device of claim 1, further comprising a motion context update unit that, when a motion to be applied to the NPC is determined, updates the motion context information with the context estimation value estimated from the motion candidates for that motion and the motion context information after the motion is executed that is estimated based on the motion context information at a point in time before the motion is executed.

10. An information processing device as described in claim 1, which assumes that a plurality of the motion candidates have been executed, and selects an optimal motion from the plurality of motion candidates based on the context suitability estimated from a series of the motion candidates assumed to have been executed following each of the motion candidates.

11. The information processing device of claim 1, wherein the motion candidate generation unit generates one of the motion candidates, and if the context compatibility estimated from the one of the motion candidates is greater than or equal to a predetermined threshold, the motion candidate is determined to be the motion to be applied to the NPC, and if the context compatibility estimated from the one of the motion candidates is less than the predetermined threshold, the generation of the motion candidate is repeated until the context compatibility becomes greater than or equal to the predetermined threshold.

12. The information processing device according to claim 2, wherein the target context information is adjusted so that the motion candidate resulting in a larger movement is selected when the NPC speaks during a dialogue.

13. An information processing method including: an information processing device generating a plurality of motion candidates to be applied to an NPC (Non-Player Character) during a dialogue based on dialogue information required for the NPC to perform a motion; estimating a context estimation value indicating the degree of influence that each of the plurality of motion candidates has on the context of the motion performed by the NPC; estimating a context compatibility level for each of the plurality of motion candidates indicating its compatibility with the context by referring to at least the context estimation value for each of the plurality of motion candidates and motion context information based on the context; and determining a motion to be applied to the NPC based on the context compatibility level for each of the motion candidates.

14. A program for causing a computer of an information processing device to execute information processing including: generating a plurality of motion candidates to be applied to an NPC (Non-Player Character) during a dialogue based on dialogue information required for the NPC to perform a motion; estimating a context estimation value indicating the degree of influence that each of the plurality of motion candidates will have on the context of the motion performed by the NPC; estimating a context compatibility level for each of the plurality of motion candidates indicating its compatibility with the context by referring to at least the context estimation value for each of the plurality of motion candidates and motion context information based on the context; and determining a motion to be applied to the NPC based on the context compatibility level for each of the motion candidates.

Images