Information processing system, information processing method, and program

Abstract

[Problem] To enable a character in a virtual space to perform a more natural response. [Solution] Provided is an information processing system comprising: an analysis unit that analyzes a motion pertaining to a user avatar in a virtual space; and a control unit that controls a non-verbal response of a character in the virtual space on the basis of the result of analysis by the analysis unit.

Description

Information Processing System, Information Processing Method, and Program

[0001] The present disclosure relates to an information processing system, an information processing method, and a program.

[0002] In recent years, content using virtual spaces has become widespread. Also, technologies for controlling the behavior of characters in virtual spaces have been developed. For example, Patent Document 1 discloses a technology for controlling the behavior of an NPC (Non-Player Character) in a game space according to the progress of the game.

[0003] Japanese Unexamined Patent Application Publication No. 2016-112471

[0004] However, the technology disclosed in Patent Document 1 only selects the behavior of the NPC from among a plurality of preset candidates.

[0005] According to one aspect of the present disclosure, there is provided an information processing system including an analysis unit that analyzes the motion of a user avatar in a virtual space, and a control unit that controls the non-verbal response of a character in the virtual space based on the analysis result of the analysis unit.

[0006] Also, according to another aspect of the present disclosure, there is provided an information processing method in which a processor analyzes the motion of a user avatar in a virtual space and controls the non-verbal response of a character in the virtual space based on the analysis result.

[0007] Also, according to another aspect of the present disclosure, there is provided a program that causes a computer to function as an information processing system including an analysis unit that analyzes the motion of a user avatar in a virtual space, and a control unit that controls the non-verbal response of a character in the virtual space based on the analysis result of the analysis unit.

[0008] This is a diagram illustrating an overview of one embodiment of the present disclosure. This is a block diagram illustrating an example of the functional configuration of the information processing system 1 according to the same embodiment. This is a block diagram illustrating an example of the functional configuration of the engine 10 according to the same embodiment. This is a diagram illustrating mirroring control by the engine 10 according to the same embodiment. This is a diagram illustrating the detection of body gestures based on the state of the rig of the user avatar 60 according to the same embodiment. This is a diagram illustrating the detection of hand gestures based on the state of the rig of the user avatar 60 according to the same embodiment. This is a diagram illustrating an example of control according to the distance between the user avatar 60 and the character 70 according to the same embodiment. This is a diagram illustrating an example of response control based on information regarding the concept of the metaverse space, events in the metaverse space, etc., according to the same embodiment. This is a diagram illustrating an example of collision avoidance control between the character 70 and other objects 75 according to the same embodiment. This is a diagram illustrating an example of collision avoidance control between the character 70 and other objects 75 according to the same embodiment. This is a diagram illustrating an example of collision control avoidance between the character 70 and the user avatar 60 or other character 70 according to the same embodiment. This is a diagram illustrating a collision control avoidance between the character 70 and the user avatar 60 or other character 70 according to the same embodiment. This is a diagram illustrating movement control to an arbitrary point according to the same embodiment. This is a diagram illustrating an example of recovery control after collision avoidance according to the same embodiment. This is a diagram illustrating turning control based on the field of view according to the same embodiment. This is a diagram illustrating the details of turning control according to the same embodiment. This is a diagram showing an example of gaze control of character 70 according to the same embodiment. This is a block diagram showing an example of the hardware configuration of the information processing device 90 according to the same embodiment.

[0009] Preferred embodiments of this disclosure will be described in detail below with reference to the attached drawings. In this specification and the drawings, components having substantially the same functional configuration are denoted by the same reference numerals, and redundant descriptions will be omitted.

[0010] Furthermore, in this specification and drawings, when describing multiple identical components to distinguish them, letters or other symbols may be added to the end of the reference numerals. On the other hand, when there is no need to distinguish multiple identical components, the letters or other symbols may be omitted, and a description common to all identical components may be provided.

[0011] The explanation will be presented in the following order: 1. Embodiments 1.1. Overview 1.2. Example of Functional Configuration 1.3. Example of Control 2. Example of Hardware Configuration 3. Summary

[0012] <1. Implementation Examples> <<1.1. Overview>> In recent years, content using various virtual spaces such as game spaces and metaverse spaces has become widespread.

[0013] Furthermore, in such virtual spaces, NPCs may be placed and interact with the PC (Player Character) or avatar (also called a user avatar) controlled by the user.

[0014] However, if, for example, an NPC repeatedly performs a set of routine actions, it is conceivable that the user may get the impression that the NPC is mechanical (artificial), significantly reducing their sense of immersion in the virtual space.

[0015] To avoid the situations described above, technologies have been developed, such as the one disclosed in Patent Document 1, which aims to enable NPCs to behave in a more natural and human-like manner.

[0016] However, the technology disclosed in Patent Document 1 merely selects the NPC's actions from a set of preset candidates and cannot be said to achieve human-like behavior.

[0017] In particular, user expectations are high for game spaces and metaverse spaces that pursue realism in recent years, and the quality of NPC behavior can directly impact the evaluation of the content.

[0018] The technical concept of one embodiment of this disclosure was conceived with the above-mentioned points in mind, and enables characters in virtual space to respond more naturally.

[0019] To this end, one of the features of an information processing method according to one embodiment of the present disclosure is that it analyzes the motion of a user avatar in a virtual space and controls the response of the character in the virtual space based on the results of the analysis.

[0020] Figure 1 is a diagram illustrating an overview of one embodiment of the present disclosure.

[0021] Engine 10 is an example of an information processing device that operates as an analysis unit for analyzing the motion related to the user avatar 60, and a control unit for controlling the behavior of the character 70 in the virtual space based on the analysis results.

[0022] Engine 10 receives various types of information used for analyzing the motion of the user avatar 60.

[0023] The above information includes, for example, information used for analysis such as face orientation, facial expression, gaze, body orientation, body gestures, hand gestures, distance between body and hands, and distance between body and feet. The methods for obtaining this information will be explained in detail later.

[0024] The engine 10 analyzes the motion related to the user avatar 60 based on the input information, and controls the behavior of the character 70, especially its response, based on the analysis results.

[0025] The right side of Figure 1 shows an example of the response of the character 70, which is realized by control by the engine 10.

[0026] The engine 10 may, for example, control a response that expresses the emotions of the character 70.

[0027] The emotions mentioned above may include, for example, Joy, Fun, Anger, Sorrow, Positive, Negative, and Neutral.

[0028] For example, if the user avatar 60 gazes at the face of the character 70, the engine 10 may cause the character 70 to perform a response expressing an emotion such as Joy, Fun, or Positive.

[0029] On the other hand, if the user avatar 60 stares intently at the character 70's feet, or if the user avatar 60 suddenly approaches the character 70, the engine 10 may cause the character 70 to execute a response expressing an emotion such as Anger, Sorrow, or Negative.

[0030] Thus, the response of character 70 may be a response to the behavior of user avatar 60. In other words, engine 10 may control the character's response to user avatar 60.

[0031] On the other hand, the engine 10 may cause one character 70 to execute a response to the behavior of another character 70.

[0032] Furthermore, the engine 10 may at least control the nonverbal response of the character 70.

[0033] Through the control methods described above, it becomes possible to achieve more natural and diverse responses from the character to the user avatar 60, regardless of language.

[0034] Furthermore, this is expected to prevent a decrease in users' immersion in the virtual space and improve their evaluation of the content.

[0035] In particular, recent metaverse spaces, while possessing vast areas, sometimes limit the number of users who can log in simultaneously to reduce processing load, while simultaneously deploying a large number of characters (70).

[0036] If a large number of characters (70) are seen standing stiffly, in unnatural postures or orientations, with blank expressions, and showing no reaction, the sense of immersion in the metaverse space will be significantly reduced, and the evaluation of the content may also be significantly lowered.

[0037] On the other hand, according to the above control, by realizing human-like behaviors by a large number of characters 70, an impression that the metaverse space is lively can be given to the user, and the activation of the metaverse space can be achieved.

[0038] <<1.2. Functional configuration example>> Next, a functional configuration example of an information processing system 1 that implements an information processing method according to an embodiment of the present disclosure will be described.

[0039] FIG. 2 is a block diagram showing a functional configuration example of the information processing system 1 according to the present embodiment.

[0040] As shown in FIG. 2, the information processing system 1 according to the present embodiment may include, for example, an engine 10, a tracking device 20, a display device 30, and an operation device 40.

[0041] (Engine 10) As described above, the engine 10 according to the present embodiment is an example of an information processing device that operates as an analysis unit that analyzes the motion related to the user avatar 60 and a control unit that controls the behavior of the character 70 in the virtual space based on the analysis result.

[0042] The engine 10 is connected to the tracking device 20, the display device 30, and the operation device via a network.

[0043] (Tracking device 20) The tracking device 20 in the present embodiment is an example of a configuration that acquires information used for analyzing the motion related to the user avatar 60.

[0044] The tracking device 20 may be, for example, a device that tracks the motion of the entire body of the user corresponding to the user avatar 60 (so-called full tracking), or a device that tracks the motion of a part of the user's body.

[0045] Also, the tracking device 20 may be an optical device or an inertial sensor type device.

[0046] Also, the tracking device 20 may include a camera, a microphone, etc.

[0047] The tracking device 20 transmits the sensing information collected for the user corresponding to the user avatar 60 to the engine 10 via the network.

[0048] However, as will be described later, the engine 10 according to the present embodiment may analyze the motion related to the user avatar 60 based on the state of the limb of the user avatar 60. In this case, the information processing system 1 does not necessarily need to include the tracking device 20.

[0049] (Display device 30) The display device 30 according to the present embodiment displays content including a virtual space under the control of the engine 10.

[0050] The display device 30 may be, for example, an HMD (Head Mounted Display), a smartphone, a tablet, or the like.

[0051] Note that the display device 30 may photograph a part of the user's body such as the face and hands, and transmit the photographed image to the engine 10. These images can be used for analyzing the motion related to the user avatar 60.

[0052] Further, the display device 30 may transmit information regarding the sound recorded by the microphone to the engine 10.

[0053] (Operation device 40) The operation device 40 is a configuration used to detect the operation by the user.

[0054] The operation device 40 may be various controllers held by the user's hand, or may be a ring-type device worn on the user's finger.

[0055] Further, the operation device 40 may be a touch panel formed integrally with the display device 30.

[0056] The operation device 40 transmits the information regarding the detected user operation to the engine 10.

[0057] The above describes an example of the functional configuration of an information processing system 1 according to one embodiment of this disclosure. However, the above functional configuration described with reference to Figure 2 is merely an example, and the functional configuration of the information processing system 1 according to this embodiment is not limited to this example.

[0058] For example, each component of engine 10 does not necessarily have to be present in a single device. In other words, the functions described as those of engine 10 may be realized through the cooperation of multiple devices.

[0059] On the other hand, at least a portion of the configuration of the engine 10 can be formed integrally with the display device 30 or the tracking device 20.

[0060] The functional configuration of the information processing system 1 according to this embodiment can be flexibly modified according to specifications, operation, etc.

[0061] Next, an example of the functional configuration of the engine 10 according to this embodiment will be described in detail. Figure 3 is a block diagram showing an example of the functional configuration of the engine 10 according to this embodiment.

[0062] As shown in Figure 3, the engine 10 according to this embodiment may include an analysis unit 110, a control unit 120, and a communication unit 130.

[0063] (Analysis Unit 110) The analysis unit 110 according to this embodiment analyzes the motion related to the user avatar 60.

[0064] Furthermore, if the information processing system 1 is equipped with a tracking device 20, the analysis unit 110 analyzes the user's motion to be reflected in the user avatar 60 based on the sensing information collected by the tracking device. This user motion can be said to be one of the motions related to the user avatar 60.

[0065] The analysis unit 110 according to this embodiment may include a gaze detection unit 111, a gesture detection unit 112, a speech recognition unit 113, a positional relationship detection unit 114, a context acquisition unit 115, and a learning unit 116, etc.

[0066] The functions of the analysis unit 110 are realized through the cooperation of various processors and memory.

[0067] (Gaze detection unit 111) The gaze detection unit 111 according to this embodiment detects the user's gaze point based on sensing information collected by the tracking device 20 and the display device 30.

[0068] The gaze detection unit 111 may further detect the user's face orientation, gaze direction, body orientation, etc.

[0069] (Gesture detection unit 112) The gesture detection unit 112 according to this embodiment detects the user's gestures based on sensing information collected by the tracking device 20 and the display device 30.

[0070] On the other hand, the gesture detection unit 112 according to this embodiment may detect gestures based on the state of the rig of the user avatar 60 operated by the user using the operating device 40, as will be described later.

[0071] The gestures mentioned above include both hand gestures and body gestures.

[0072] Furthermore, the gesture detection unit 112 may also detect the user's overall body pose, the distance between the user's body and hands, the distance between the user's body and feet, and so on.

[0073] (Speech Recognition Unit 113) The speech recognition unit 113 according to this embodiment recognizes the user's speech, etc., from the voice collected by the tracking device 20 and the display device 30.

[0074] (Position Relationship Detection Unit 114) The position relationship detection unit 114 according to this embodiment detects the position relationship between virtual objects in the virtual space.

[0075] The above virtual objects broadly include user avatars 60, characters 70, and other objects.

[0076] (Context acquisition unit 115) The context acquisition unit 115 according to this embodiment acquires the context from a storage unit (not shown) or a server or the like, which is provided separately from the engine 10.

[0077] The above context includes, for example, the concept of a virtual space, events in the virtual space, the user profile corresponding to the user avatar 60, the character profile 70, and so on.

[0078] (Learning Unit 116) The learning unit 116 in this embodiment learns the responses made by the user avatar 60.

[0079] The above responses include responses from one user avatar 60 to the behavior of another user avatar 60, and responses from one user avatar 60 to the behavior of a character 70.

[0080] The above describes an example of the functional configuration of the analysis unit 110 according to this embodiment. However, the above functional configuration described with reference to Figure 3 is merely an example, and the functional configuration of the analysis unit 110 according to this embodiment is not limited to this example.

[0081] For example, the analysis unit 110 does not necessarily need to have all of the configurations listed above. On the other hand, the analysis unit 110 may perform analyses other than those listed above.

[0082] Furthermore, the analysis unit 110 may perform further analysis by combining the results of each analysis. For example, the analysis unit 110 may analyze the user's emotions, user intentions, etc., based on the results of gaze detection, gesture detection, and speech recognition.

[0083] (Control Unit 120) The control unit 120 according to this embodiment controls the behavior of the user avatar 60 and the character 70 based on the results of various analyses performed by the analysis unit 110.

[0084] In particular, the control unit 120 controls the response of the character 70 based on the results of various analyses.

[0085] As shown in Figure 3, the control unit 120 may include a facial expression generation unit 121, a motion generation unit 122, and a behavior control unit 123.

[0086] The functions of the control unit 120 are realized through the cooperation of various processors and memory.

[0087] (Expression generation unit 121) The expression generation unit 121 according to this embodiment determines the expression to be made by the user avatar 60 or character 70 based on the results of various analyses by the analysis unit 110, and generates data (expression data) related to that expression.

[0088] (Motion generation unit 122) The motion generation unit 122 according to this embodiment determines the motion to be performed by the user avatar 60 or character 70 based on the results of various analyses by the analysis unit 110, and generates data (motion data) related to that motion.

[0089] (Behavior control unit 123) The behavior control unit 123 according to this embodiment controls the behavior of the user avatar 60 and the character 70 based on the facial expression data generated by the facial expression generation unit 121 and the motion data generated by the motion generation unit 122.

[0090] In particular, one of the features of the behavior control unit 123 according to this embodiment is that it controls the response of the character 70.

[0091] The above describes an example of the functional configuration of the control unit 120 according to this embodiment. However, the above functional configuration described with reference to Figure 3 is merely an example, and the functional configuration of the control unit 120 according to this embodiment is not limited to this example.

[0092] For example, the control unit 120 may consist only of a behavior control unit 123. In this case, the facial expression generation unit 121 and the motion generation unit 122 may be provided as separate components of the engine 10, or they may be provided in a device separate from the engine 10.

[0093] (Communication unit 130) The communication unit 130 according to this embodiment performs information communication with other devices via a network.

[0094] For example, the communication unit 130 receives various types of sensing information from the tracking device 20 and the display device 30.

[0095] Furthermore, for example, the communication unit 130 receives information about user operations from the operating device 40.

[0096] Furthermore, for example, the communication unit 130 transmits the control signal generated by the control unit 120 to the display device 30.

[0097] The above describes an example of the functional configuration of the engine 10 according to this embodiment. However, the above functional configuration described with reference to Figure 3 is merely an example, and the functional configuration of the engine 10 according to this embodiment is not limited to this example.

[0098] For example, some of the functions of the analysis unit 110 described above may be implemented as functions of the tracking device 20 or the display device 30.

[0099] Furthermore, for example, the behavior control unit 123 may be provided as part of the display device 30.

[0100] The functional configuration of the engine 10 according to this embodiment can be flexibly modified according to specifications, operation, etc.

[0101] <<1.3. Control Example>> Next, a control example using the engine 10 according to this embodiment will be described in detail.

[0102] First, let's discuss the mirroring control provided by engine 10.

[0103] The engine 10 may control the response of the character 70, which mimics the motion of the user avatar 60.

[0104] In other words, the engine 10 may mirror the actions performed by the user avatar 60 to the character 70.

[0105] Figure 4 is a diagram illustrating the mirroring control by the engine 10 according to this embodiment.

[0106] The first row of Figure 4 illustrates a situation where the user avatar 60 gives a thumbs-up. In this case, the engine 10 causes the character 70 to mirror the thumbs-up gesture based on the user avatar 60's gesture.

[0107] Furthermore, the second row of Figure 4 illustrates a situation where the user avatar 60 makes a gesture of raising one hand. In this case, the engine 10 causes the character 70 to mirror the gesture of raising one hand, based on the fact that the user avatar 60 has made that gesture.

[0108] Furthermore, the third row of Figure 4 illustrates a situation in which the user avatar 60 makes a gesture of raising both hands. In this case, the engine 10, based on the fact that the user avatar 60 has made a gesture of raising both hands, causes the character 70 to mirror the gesture of raising both hands.

[0109] Furthermore, the fourth row of Figure 4 illustrates a situation where the user avatar 60 makes a gesture to ask for a high-five. In this case, the engine 10, based on the user avatar 60 making a gesture to ask for a high-five, causes the character 70 to mirror the gesture.

[0110] As illustrated above, the response of the character 70 that imitates the motion of the user avatar 60 according to this embodiment includes hand gestures, body gestures, and the like.

[0111] In addition to the examples mentioned above, the engine 10 may also be made to mimic gestures made by the user avatar 60, such as shaking hands, waving one hand, waving both hands, asking for a knuckle touch, bowing, and jumping.

[0112] As described above, mirroring control increases the likelihood of having character 70 perform the response requested by the user, and is also expected to create a positive impression on the user through the mirroring effect.

[0113] Furthermore, the detection of the gestures performed by the user avatar 60 may be performed based on sensing information collected by the tracking device 20.

[0114] Furthermore, the gestures performed by the user avatar 60 may be detected (recognized) based on images captured by the tracking device 20 or the display device 30. In this case, gesture recognition can be achieved, for example, using a recognizer that has been trained to recognize a predetermined gesture from an image.

[0115] Furthermore, the gestures performed by the user avatar 60 may be detected based on the state of the user avatar 60's rig.

[0116] Figure 5 is a diagram illustrating the detection of body gestures based on the state of the rig of the user avatar 60 according to this embodiment.

[0117] As shown in Figure 5, let H be the height of the user avatar 60, and let h be the threshold value for determining whether the hand is raised, defined as h = k * H. Note that k is an appropriate constant.

[0118] In this case, by comparing the height of the rig for the user avatar 60's right hand and the height of the rig for its left hand, h, it is possible to detect whether both hands are raised or only one hand is raised.

[0119] For example, in the case shown in the upper part of Figure 5, the engine 10 can detect that the user avatar 60 is making a gesture of raising its right hand because the height of the rig RW for the user avatar 60's right hand is higher than h, and the height of the rig LW for the left hand is lower than h.

[0120] Furthermore, for example, in the case shown in the lower part of Figure 5, the engine 10 can detect that the user avatar 60 is making a gesture of raising both hands because the height of both the rig RW for the user avatar 60's right hand and the rig LW for the user avatar 60 are higher than h.

[0121] Figure 6 is a diagram illustrating the detection of hand gestures based on the state of the rig of the user avatar 60 according to this embodiment.

[0122] Figure 6 illustrates the first to third joint rigs of the fingers of the user avatar 60 using circles.

[0123] Since the action of bending (clenching) a human finger largely depends on the flexion of the second joint, the bending of each finger can be detected by obtaining the rotation angles of the rig T2 of the second joint of the thumb, the rig I2 of the second joint of the index finger, the rig M2 of the second joint of the middle finger, the rig R2 of the second joint of the ring finger, and the rig L2 of the second joint of the little finger.

[0124] From the above, for example, if the engine 10 detects that all fingers except the thumb are bent on the right or left hand of the user avatar 60, it can detect that the hand is making a thumbs-up gesture.

[0125] Furthermore, for example, if engine 10 detects that all fingers except the index and middle fingers are bent on the right or left hand, it can detect that the hand is making a peace sign.

[0126] As described above, the engine 10 according to this embodiment can detect the hand gestures of the user avatar 60 based on the rotation angle of the rig at the second joint of each finger of the user avatar 60.

[0127] According to the detection method described above, even when a user is operating the user avatar 60 using a touch panel (an example of the operation device 40) on a display device 30 such as a smartphone, or when a user is operating the user avatar 60 using a game controller (an example of the operation device 40), it is possible to detect the actions performed by the user avatar 60 and mirror those actions to the character 70.

[0128] Next, we will describe the control based on the distance between the user avatar 60 and the character 70 according to this embodiment.

[0129] Figure 7 shows an example of control based on the distance between the user avatar 60 and the character 70 according to this embodiment.

[0130] The upper part of Figure 7 illustrates a situation where the distance L between the user avatar 60 and the character 70 engaging in dialogue is relatively close. In this case, the engine 10 may execute a whispering response to the character 70 based on the distance L.

[0131] Furthermore, the middle section of Figure 7 illustrates a situation where the distance L between the user avatar 60 and the character 70 engaging in dialogue is moderate. In this case, the engine 10 may cause the character 70 to execute a response with a general speaking style based on the distance L.

[0132] Furthermore, the lower part of Figure 7 illustrates a situation where the distance L between the user avatar 60 and the character 70 engaging in dialogue is relatively far. In this case, the engine 10 may cause the character 70 to respond by shouting (raising its voice) based on the distance L.

[0133] Furthermore, the engine 10 may perform the above-described distance-based control based, for example, on the concept of personal space (or interpersonal distance) in real space. For example, the engine 10 may determine that the distance L is relatively close if it falls below the far phase of the intimate distance in personal space. Also, for example, the engine 10 may determine that the distance L is moderate if it falls below the social distance in personal space. Also, for example, the engine 10 may determine that the distance L is relatively far if it falls above the public distance in personal space.

[0134] As illustrated above, the engine 10 according to this embodiment may control the response of the character 70 according to its distance from the user avatar 60.

[0135] With the control described above, it becomes possible to make character 70 perform natural behaviors similar to those that real people exhibit.

[0136] In the above example, the engine 10 controls the response of the character 70 according to the physical distance, but the engine 10 may also control the response of the character 70 according to psychological distance, social distance, etc.

[0137] In this case, psychological distance and social distance may be defined by a type of context, such as the user's profile or the character 70's profile. For example, in a user's profile, the psychological distance between that user and a certain character 70 may be defined as equivalent to a close friendship.

[0138] Other examples of psychological distance include relationships with strangers, friends, family, colleagues, romantic partners, tutors, and neighbors.

[0139] Furthermore, even if the engine 10 does not perform distance-dependent control as described above, it may control the response of the character 70 based on the context.

[0140] For example, the engine 10 may control the response of character 70 based on both the character 70's profile and the user's profile, or either one of them.

[0141] Examples of character 70 and user profiles include gender, age, origin, personality, and social role.

[0142] In terms of personality traits, examples include serious / unserious, honest / insincere, responsible / irresponsible, cheerful / gloomy, sociable / introverted, optimistic / pessimistic, cautious / rash, bold / timid, curious / indifferent, etc.

[0143] Examples of the social roles mentioned above include occupations (manager, employee, teacher, actor, student, housewife, househusband, unemployed, etc.), domestic roles (parent, child, spouse, etc.), and designated intellectual property (IP) characters.

[0144] By using response control based on the profile described above, it is possible to achieve a wider variety of responses that correspond to the characteristics of character 70 or the user.

[0145] Furthermore, the context according to this embodiment may include information regarding the concept of a virtual space, events in the virtual space, etc.

[0146] In the following explanation, we will use the example of a metaverse space where the virtual space provides virtual live performances by artists and other users.

[0147] Figure 8 shows the concept of the metaverse space according to this embodiment, and an example of response control based on information about events in the metaverse space.

[0148] The upper part of Figure 8 illustrates the situation at the start of a virtual live performance. At the start of a virtual live performance, the engine 10 causes each of the characters 70 to execute a response appropriate for the start of a live performance, based on the context that the virtual live performance has started.

[0149] For example, engine 10 may cause character 70 to perform responses such as waving to the appearing artist or being moved by the sight of the artist.

[0150] Furthermore, in this case, the engine 10 can control the response of the character 70 based on each profile.

[0151] The middle section of Figure 8 illustrates a virtual live climax. In situations where the atmosphere is exciting, such as a climax, the engine 10 may mirror the gestures of the user avatar 60 to multiple surrounding characters 70.

[0152] The phenomenon of one participant's gestures spreading to other participants is a frequent occurrence in real-world live events. Therefore, mirroring control as described above can make virtual live events feel more natural and simultaneously enhance the excitement of the virtual live performance.

[0153] Furthermore, in this case, the engine 10 may control its response according to the distance between the user avatar 60 and the character 70.

[0154] For example, the engine 10 may mirror the gestures of the user avatar 60, starting with the character 70 closest to the user avatar 60.

[0155] For example, if the user avatar 60 is located in the first-floor seating area, the engine 10 may mirror the gestures of the character 70 located in the first-floor seating area, the character 70 located in the second-floor seating area, and the character 70 located in the third-floor seating area in that order.

[0156] The control described above enables more natural gesture propagation and allows for audience performances such as the so-called "wave."

[0157] Furthermore, engine 10 may control the response using tools. Examples of tools in a virtual live performance include penlights, balloons, etc.

[0158] For example, the engine 10 may perform control to mirror the operation, light color, etc., of the penlight used by the user avatar 60 to the character 70.

[0159] Furthermore, the engine 10 may control mirroring with respect to multiple user avatars 60.

[0160] For example, the engine 10 may cause each of the characters 70A located in block A to mirror the actions of the user avatar 60A located in block A, and each of the characters 70B located in block B to mirror the actions of the user avatar 60B located in block B.

[0161] With the control described above, it becomes possible to create performances similar to so-called cheering competitions.

[0162] Furthermore, the lower part of Figure 8 illustrates the situation after the virtual live performance has ended. After the virtual live performance has ended, the engine 10 causes each of the characters 70 to execute a response appropriate for the end of the live performance, based on the context that the virtual live performance has ended.

[0163] In this case, the engine 10 can control the response of the character 70 based on its respective profile.

[0164] For example, engine 10 can cause each of the characters 70 to perform responses such as a couple holding hands on their way home, friends discussing their impressions of a live performance, or one person looking at their smartphone.

[0165] The above describes the concept of the metaverse space according to this embodiment, and the response control based on information about events in the metaverse space, with specific examples.

[0166] While the above example uses a virtual live performance, the concept of the metaverse space, events, etc., are not limited to this example.

[0167] Engine 10 may, for example, control the response of character 70 in a metaverse space that provides a venue for watching sports. In the following explanation, a soccer match will be used as an example.

[0168] When a match is played between Team A and Team B, the audience can usually be broadly divided into those supporting Team A and those supporting Team B. Also, seating arrangements may be set up separately for each team (home / away, etc.).

[0169] In such a situation, the engine 10 may control the responses of each character 70 based on the team supported by the user avatar 60 and the team supported by each of the characters 70. The teams to be supported may be defined in the profile.

[0170] For example, if the user avatar 60 is supporting Team A, the engine 10 will mirror the gestures of the user avatar 60 only to the character 70, which is also supporting Team A.

[0171] The control described above makes it possible to avoid unnatural situations such as gestures made by the user avatar 60 during a match being propagated to the enemy's territory.

[0172] Specific examples of the above unnatural situations include the gesture of user avatar 60 celebrating Team A's goal being transmitted to the cheering section of Team B, and the cheering gesture unique to Team A performed by user avatar 60 being transmitted to the cheering section of Team B.

[0173] However, before or after a match, the engine 10 may perform mirroring control regardless of which team is being supported (for example, during the national anthem).

[0174] Furthermore, the engine 10 may control the response of the character 70 according to the match situation, regardless of the gestures of the user avatar 60.

[0175] For example, if Team A scores a goal, the engine 10 may cause the character 70 supporting Team A to perform a positive response, and the character 70 supporting Team B to perform a negative response.

[0176] Next, we will describe the control based on the positional relationship between the character 70 and other objects according to this embodiment.

[0177] In this embodiment, the engine 10 may control the response of the character 70 according to the positional relationship between the character 70 and other objects.

[0178] For example, the engine 10 may control the response of the character 70 to avoid collisions between the character 70 and other objects.

[0179] Figures 9 to 11 illustrate an example of collision avoidance control between the character 70 and other objects 75 according to this embodiment.

[0180] In this example, as shown in Figure 9, we assume a situation where another object 75 exists between the user avatar 60 and the character 70. In this example, the other object 75 may be a static obstacle.

[0181] In the situation described above, let's consider a case where character 70 is instructed to respond by approaching user avatar 60. If the positional relationship between character 70 and other objects 75 is not considered, as shown in Figure 10, character 70 will approach user avatar 60 by the shortest distance, and as a result, character 70 may overlap with other objects 75.

[0182] Such object overlaps significantly reduce the sense of immersion in the virtual space.

[0183] Therefore, the engine 10 controls the character 70 to move closer to the user avatar 60 while avoiding collisions with the other object 75, based on the relative positions of the character 70, the other object 75, and the user avatar 60.

[0184] The above-described control method can prevent a decrease in immersion caused by overlapping objects.

[0185] Furthermore, as shown in Figure 11, when the engine 10 performs collision avoidance control as described above, it may cause the character 70 to rotate.

[0186] By having character 70 perform rotational movements similar to those performed by real humans, it is possible to create a more natural evasive action.

[0187] Furthermore, other objects 75 that may collide with character 70 are not limited to static obstacles, but also include user avatars 60 or other characters 70.

[0188] Figures 12 and 13 illustrate an example of collision control and avoidance between the character 70 and a user avatar 60 or other character 70 according to this embodiment.

[0189] Figure 12 illustrates a situation in which a user avatar 60 (or another character 70) approaches a character 70. The trajectory of the user avatar 60 is shown by a dashed line.

[0190] Engine 10 controls the character 70 to take evasive action if the distance between the user avatar 60 and the character 70 falls below a threshold.

[0191] For example, the engine 10 can cause the character 70 to rotate by +θ° or -θ° around the trajectory of the user avatar 60 as an axis (±0°).

[0192] However, as shown in the example in Figure 12, if another object 75 lies on the extension of +θ° or -θ°, the character 70 may collide with the other object due to rotational movement.

[0193] Therefore, in the example shown in Figure 12, the engine 10 may perform evasive control such as moving the character 70 backward by several tens of centimeters.

[0194] On the other hand, as shown in the example in Figure 13, if no other object 75 is present on the trajectory of the rotational movement of the user avatar 60 with respect to the tangent (shown as a dashed line) of the arc relating to the rotational movement trajectory, the engine 10 may control the avoidance action based on the rotation angle r of the user avatar 60's movement vector with respect to the tangent (shown as a dashed line) of the arc relating to the rotational movement trajectory.

[0195] For example, if the rotation angle r of the movement vector is greater than or equal to a threshold Y, the engine 10 may determine that the user avatar 60 is passing to the right of the character 70 and cause the character 70 to rotate in a counterclockwise direction.

[0196] On the other hand, if the rotation angle r of the movement vector is below the threshold Y, the engine 10 may determine that the user avatar 60 is passing to the left of the character 70 and cause the character 70 to rotate in a clockwise direction.

[0197] Furthermore, the engine 10 may perform control to move the character 70 to an arbitrary point (goal) while avoiding collisions with other objects 75.

[0198] Figure 14 is a diagram illustrating the control of movement to an arbitrary point according to this embodiment.

[0199] As illustrated in Figure 14, if there are multiple other objects 75 in the path from the start to the goal, the engine 10 can, for example, repeatedly execute the following processes 1 to 3 to allow the character 70 to reach the goal while avoiding collisions with other objects.

[0200] (Process 1) Move the character 70 forward in the direction of the goal to a position where it does not collide with other objects 75.

[0201] (Process 2) If the character approaches another object 75 by a predetermined distance or more, the movement vector is rotated in a direction to avoid collision, and the character 70 is moved forward.

[0202] (Process 3) If the character passes another object 75, the character 70 is moved forward with its movement vector directed towards the goal.

[0203] On the other hand, the engine 10 may use a general reinforcement learning method to achieve collision avoidance and reach the goal.

[0204] Next, the recovery control after collision avoidance according to this embodiment will be described. If the character 70 has interacted with another object 75 before the collision, the engine 10 may control the character 70 to naturally return to that interaction after collision avoidance control.

[0205] An example of the above interaction is a conversation with another character 70 or user avatar 60.

[0206] Figure 15 is a diagram illustrating an example of recovery control after collision avoidance according to this embodiment.

[0207] The upper part of Figure 15 shows a situation in which the user avatar 60 approaches character 70A while characters 70A and 70B are having a conversation.

[0208] In this case, the engine 10 causes character 70A to take action to avoid collision with user avatar 60, as shown in the middle of Figure 15. In this example, it is assumed that the engine 10 causes character 70A to move backward to avoid collision with user avatar 60 and other objects 75.

[0209] Subsequently, the engine 10 moves character 70A to the position before the evasive action, as shown in the lower part of Figure 15, and resumes dialogue with character 70B.

[0210] According to the control described above, it is possible to prevent unnatural situations such as character 70A, which performed an evasive action, continuing to converse with character 70B at the position it was in when the evasive action ended.

[0211] Furthermore, interactions such as dialogue may be controlled based on the relationships with other objects 75 defined in the profile of character 70.

[0212] The collision avoidance control according to this embodiment has been described above. The engine 10 according to this embodiment may also control the avoidance control described above in conjunction with emotional expression.

[0213] For example, the engine 10 may cause character 70 to take evasive action and also to express expressions of surprise or anger. Furthermore, such emotional expression control may be performed based on character 70's profile.

[0214] Next, the turning control according to this embodiment will be described. The engine 10 according to this embodiment may control the response such that the character 70, upon noticing the presence or gaze of the user avatar 60, turns to face the user avatar 60.

[0215] In this case, the engine 10 may perform response control based on the field of view set for the character 70.

[0216] Figure 16 is a diagram illustrating the field-of-view-based turning control according to this embodiment.

[0217] Figure 16 shows the range of ±θ° when the horizontal gaze direction of character 70 is set to 0°. The value of θ may be determined based on, for example, the midpoint between the effective field of view and the peripheral field of view of a human.

[0218] In other words, the range of ±θ° can be said to be the range in which character 70 can see user avatar 60.

[0219] Therefore, the engine 10 will turn the character 70 towards the user avatar 60 only when the user avatar 60 is within the range of ±θ°, and does not need to turn the character 70 when the user avatar 60 is outside the range of ±θ° (indicated by dots).

[0220] Furthermore, if the user avatar 60 is within the range of ±θ°, the engine 10 may perform the following controls to make the character 70 turn more naturally.

[0221] Figure 17 is a diagram illustrating the details of the turning control according to this embodiment.

[0222] As a premise, as shown on the left side of Figure 17, let φ° be the angle between the direction in which character 70 is looking and the direction in which user avatar 60 is positioned relative to character 70.

[0223] First, as shown in the center of Figure 17, the engine 10 starts rotating only the head of the character 70. When the amount of rotation of the head reaches φ° / 2, the engine 10 rotates the chest of the character 70 by φ° / 4, as shown on the right side of Figure 17. The engine 10 may also smoothly rotate the waist and lower body in accordance with the movement of the chest.

[0224] The control methods described above can be used to achieve a more human-like turning motion.

[0225] Next, the gaze control of the character 70 according to this embodiment will be described. In order to achieve a more human-like response from the character 70, the engine 10 according to this embodiment may control the gaze of the character 70 by mimicking the characteristics of how a human's gaze moves.

[0226] For example, when people converse with each other, they often direct their gaze towards the other person's eyes. However, if the other person offers something, if something suddenly approaches their face, or if something conspicuous (such as a rapidly moving object) comes into their field of vision, they will typically shift their gaze in the direction of those objects.

[0227] Therefore, when the engine 10 detects an object to be gazed upon within the character 70's field of view, it may control the engine so that the character 70's gaze is directed toward the object to be gazed upon.

[0228] Figure 18 shows an example of gaze control for character 70 according to this embodiment.

[0229] Figure 18 shows a situation in which user avatar 60 suddenly presents a bouquet of flowers to character 70. The bouquet of flowers is an example of the object of attention mentioned above.

[0230] In this case, the engine 10 detects that the bouquet has suddenly entered the character 70's field of view and controls the system so that the character 70's gaze is directed towards the bouquet.

[0231] The control described above makes it possible to achieve more human-like eye movements.

[0232] Next, we will describe the learning-based response control according to this embodiment. The engine 10 according to this embodiment may learn the responses made by the user avatar 60 and control the responses of the character 70 based on the results of the learning.

[0233] For example, the engine 10 may learn the responses that one user avatar 60 makes to other user avatars 60.

[0234] Furthermore, for example, the engine 10 may learn the responses that a certain user avatar 60 makes to the character 70. In this case, the engine 10 may also cause the character 70 to perform a predetermined action in order to obtain the user avatar 60's response.

[0235] It is presumed that the responses made by user avatar 60 will vary depending on the individual user's characteristics, cultural background, and relationship (psychological distance) with the object of the response.

[0236] For example, when asked for a high-five, some users might readily agree, others might agree hesitantly, some might not agree at all, and some might only agree if it's a request from a friend.

[0237] Therefore, by learning the responses made by a user avatar 60, it becomes possible to have the character 70 perform the responses that the user themselves would perform.

[0238] Furthermore, it is assumed that the responses provided by the users themselves are likely to be natural or leave a positive impression on them.

[0239] Therefore, by having character 70 perform the responses that the user would normally perform, it is expected that the user's evaluation will be improved.

[0240] Furthermore, since the learning-based response control described above has elements that mimic the actions of the user avatar 60, it can be considered a type of mirroring control as described above.

[0241] The response control according to this embodiment has been described in detail with specific examples.

[0242] In addition, the engine 10 according to this embodiment may control the actions of the character 70 other than the response.

[0243] For example, the engine 10 may control the actions of the character 70 based on a profile, random elements, etc.

[0244] Furthermore, the engine 10 may cause the character 70 to perform actions that are set according to the concept, events, etc., of the virtual space.

[0245] Actions that are set according to the concept and events of the virtual space include, for example, drinking water, wiping away sweat, dancing, waving, and checking the time.

[0246] Furthermore, the engine 10 may cause the character 70 to perform some action and express emotions corresponding to that action.

[0247] Through the control methods described above, it is possible to create unique and distinctive movements and behaviors, not just in terms of response time.

[0248] Furthermore, the engine 10 may further control the behavior of the user avatar 60 when not in operation, based on the user's profile, the concept of the virtual space, the context of events, and random elements.

[0249] The above-mentioned "non-operational state" refers to a state in which there is no input from the controller when the user avatar 60 is being operated using a controller, etc.

[0250] This type of control makes it possible to avoid unnatural situations such as the user avatar 60 remaining still while the surrounding characters 70 are performing some kind of action.

[0251] <2. Hardware Configuration Example> Next, a hardware configuration example of the engine 10 according to one embodiment of the present disclosure will be described.

[0252] Figure 19 is a block diagram showing an example of the hardware configuration of an information processing device 90 according to one embodiment of the present disclosure. The information processing device 90 may be a device having the same hardware configuration as the engine 10.

[0253] As shown in Figure 19, the information processing device 90 includes, for example, a processor 871, a ROM 872, a RAM 873, a host bus 874, a bridge 875, an external bus 876, an interface 877, an input device 878, an output device 879, a storage device 880, a drive 881, a connection port 882, and a communication device 883. Note that the hardware configuration shown here is just an example, and some of the components may be omitted. Furthermore, the information processing device 90 may include components other than those shown here.

[0254] (Processor 871) The processor 871 functions, for example, as an arithmetic processing unit or a control unit, and controls the overall operation or part thereof of each component based on various programs recorded in the ROM 872, RAM 873, storage 880, or removable storage medium 901.

[0255] (ROM 872, RAM 873) ROM 872 is a means for storing programs loaded into the processor 871 and data used for calculations. RAM 873 temporarily or permanently stores, for example, programs loaded into the processor 871 and various parameters that change as needed when executing those programs.

[0256] (Host bus 874, bridge 875, external bus 876, interface 877) The processor 871, ROM 872, and RAM 873 are interconnected via, for example, the host bus 874, which is capable of high-speed data transmission. On the other hand, the host bus 874 is connected to the external bus 876, which has a relatively low data transmission speed, via, for example, the bridge 875. The external bus 876 is also connected to various components via the interface 877.

[0257] (Input device 878) The input device 878 may include, for example, a mouse, keyboard, touch panel, buttons, switches, and levers. Furthermore, the input device 878 may also include a remote controller (hereinafter referred to as a remote control) capable of transmitting control signals using infrared rays or other radio waves. The input device 878 may also include an audio input device such as a microphone.

[0258] (Output device 879) The output device 879 is a device that can visually or audibly notify the user of acquired information, such as a display device such as a CRT (Cathode Ray Tube), LCD, or organic EL, an audio output device such as a speaker or headphones, a printer, a mobile phone, or a facsimile. The output device 879 according to this disclosure also includes various vibration devices capable of outputting tactile stimuli.

[0259] (Storage 880) Storage 880 is a device for storing various types of data. Examples of storage devices used for storage 880 include magnetic storage devices such as hard disk drives (HDDs), semiconductor storage devices, optical storage devices, or magneto-optical storage devices.

[0260] (Drive 881) Drive 881 is a device that reads information recorded on a removable storage medium 901, such as a magnetic disk, optical disk, magneto-optical disk, or semiconductor memory, or writes information to the removable storage medium 901.

[0261] (Removable storage medium 901) The removable storage medium 901 is, for example, DVD media, Blu-ray® media, HD DVD media, various semiconductor storage media, etc. Of course, the removable storage medium 901 may also be, for example, an IC card equipped with a contactless IC chip, or an electronic device, etc.

[0262] (Connection port 882) Connection port 882 is a port for connecting external devices 902, such as a USB (Universal Serial Bus) port, an IEEE 1394 port, a SCSI (Small Computer System Interface), an RS-232C port, or an optical audio terminal.

[0263] (External connected device 902) The external connected device 902 is, for example, a printer, a portable music player, a digital camera, a digital video camera, or an IC recorder.

[0264] (Communication device 883) The communication device 883 is a communication device for connecting to a network, and is, for example, a communication card for wired or wireless LAN, Bluetooth®, or WUSB (Wireless USB), a router for optical communication, a router for ADSL (Asymmetric Digital Subscriber Line), or a modem for various types of communication.

[0265] <3. Summary> As described above, the information processing system 1 according to one embodiment of the present disclosure comprises an analysis unit 110 that analyzes the motion of a user avatar 60 in a virtual space, and a control unit 120 that controls the nonverbal response of a character 70 in a virtual space based on the analysis results by the analysis unit 110.

[0266] The above configuration makes it possible to enable characters in the virtual space to respond more naturally.

[0267] While preferred embodiments of the present disclosure have been described in detail above with reference to the attached drawings, the technical scope of the present disclosure is not limited to such examples. It is clear to any person with ordinary skill in the art of the present disclosure that various modifications or alterations may be conceived within the scope of the technical ideas described in the claims, and these will naturally also fall within the technical scope of the present disclosure.

[0268] Furthermore, the series of processes performed by each device described in this disclosure may be implemented by a program stored on a non-transitory computer-readable storage medium. Each program is, for example, loaded into RAM when executed by a computer and executed by a processor such as a CPU. The storage medium is, for example, a magnetic disk, an optical disk, a magneto-optical disk, or flash memory. Alternatively, the program may be distributed without using a storage medium, for example, via a network.

[0269] Furthermore, the effects described herein are merely descriptive or illustrative and not limiting. In other words, the technology relating to this disclosure may produce other effects that are obvious to those skilled in the art from the description herein, in addition to or instead of the effects described herein.

[0270] Furthermore, the following configurations also fall within the technical scope of this disclosure: (1) An information processing system comprising: an analysis unit for analyzing motion related to a user avatar in a virtual space; and a control unit for controlling the nonverbal response of a character in the virtual space based on the analysis results by the analysis unit. (2) The information processing system according to (1), wherein the control unit controls the character's response to the user avatar. (3) The information processing system according to either (1) or (2), wherein the control unit controls a response that expresses the character's emotions. (4) The information processing system according to any one of (1) to (3), wherein the control unit controls the character's response that imitates the motion related to the user avatar. (5) The information processing system according to (4), wherein the character's response that imitates the motion related to the user avatar includes at least a gesture. (6) The information processing system according to any one of (1) to (5), wherein the control unit controls the response of the character according to the distance between the user avatar and the character. (7) The information processing system according to any one of (1) to (6), wherein the control unit controls the response of the character according to the context. (8) The information processing system according to (7), wherein the context includes information relating to at least one of the concepts of the virtual space, events in the virtual space, the user profile corresponding to the user avatar, and the character profile. (9) The information processing system according to (4), wherein the control unit causes a plurality of the characters to execute responses that mimic the motions relating to the user avatar. (10) The information processing system according to any one of (1) to (9), wherein the control unit controls the response of the character according to the positional relationship between the character and other objects in the virtual space.(11) The information processing system according to (10), wherein the control unit controls the response of the character to avoid collisions between the character and the other object. (12) The information processing system according to (11), wherein the other object includes at least one of the user avatar or other characters. (13) The information processing system according to any one of (1) to (12), wherein the control unit controls the response of the character according to the character's field of view. (14) The information processing system according to any one of (1) to (13), wherein the control unit further controls the movement of the user avatar when it is not being operated. (15) The information processing system according to any one of (1) to (14), wherein the analysis unit analyzes the motion related to the user avatar based on sensing information collected for the user corresponding to the user avatar. (16) The information processing system according to any one of (1) to (14), wherein the analysis unit analyzes the motion related to the user avatar based on the state of the user avatar's rig. (17) The information processing system according to any one of (1) to (16), wherein the virtual space is a metaverse space. (18) An information processing method comprising: a processor analyzing the motion related to the user avatar in a virtual space; and controlling the nonverbal response of a character in the virtual space based on the analysis results. (19) A program that causes a computer to function as an information processing system comprising: an analysis unit that analyzes the motion related to the user avatar in a virtual space; and a control unit that controls the nonverbal response of a character in the virtual space based on the analysis results by the analysis unit.

[0271] 1 Information Processing System 10 Engine 110 Analysis Unit 111 Gaze Detection Unit 112 Gesture Detection Unit 113 Speech Recognition Unit 114 Position Relationship Detection Unit 115 Context Acquisition Unit 116 Learning Unit 120 Control Unit 121 Facial Expression Generation Unit 122 Motion Generation Unit 123 Behavior Control Unit 130 Communication Unit 20 Tracking Device 30 Display Device 40 Operation Device 60 User Avatar 70 Character 75 Other Objects

Claims

1. An information processing system comprising: an analysis unit for analyzing the motion of a user avatar in a virtual space; and a control unit for controlling the nonverbal response of a character in the virtual space based on the analysis results from the analysis unit.

2. The information processing system according to claim 1, wherein the control unit controls the character's response to the user avatar.

3. The information processing system according to claim 1, wherein the control unit controls a response that expresses the character's emotions.

4. The information processing system according to claim 1, wherein the control unit controls the response of the character that mimics the motion of the user avatar.

5. The information processing system according to claim 4, wherein the response of the character that imitates the motion relating to the user avatar includes at least a gesture.

6. The information processing system according to claim 1, wherein the control unit controls the response of the character according to the distance between the user avatar and the character.

7. The information processing system according to claim 1, wherein the control unit further controls the response of the character based on the context.

8. The information processing system according to claim 7, wherein the context includes information relating to at least one of the following: the concept of the virtual space, events in the virtual space, the user profile corresponding to the user avatar, and the character profile.

9. The information processing system according to claim 4, wherein the control unit causes a plurality of characters to execute responses that mimic the motions related to the user avatar.

10. The information processing system according to claim 1, wherein the control unit controls the response of the character according to the positional relationship between the character and other objects in the virtual space.

11. The information processing system according to claim 10, wherein the control unit controls the response of the character to avoid collisions between the character and other objects.

12. The information processing system according to claim 11, wherein the other object includes at least one of the user avatar or other characters.

13. The information processing system according to claim 1, wherein the control unit controls the response of the character according to the character's field of view.

14. The information processing system according to claim 1, wherein the control unit further controls the operation of the user avatar when it is not being operated.

15. The information processing system according to claim 1, wherein the analysis unit analyzes the motion related to the user avatar based on sensing information collected for the user corresponding to the user avatar.

16. The information processing system according to claim 1, wherein the analysis unit analyzes the motion related to the user avatar based on the state of the user avatar's rig.

17. The information processing system according to claim 1, wherein the virtual space is a metaverse space.

18. An information processing method comprising: a processor analyzing motion related to a user avatar in a virtual space; and controlling the nonverbal response of a character in the virtual space based on the analysis results.

19. A program that causes a computer to function as an information processing system comprising: an analysis unit that analyzes motion related to a user avatar in a virtual space; and a control unit that controls the nonverbal response of a character in the virtual space based on the analysis results from the analysis unit.

Images