Avatar control device, avatar control method, and avatar control program

Abstract

To provide an avatar control device capable of alleviating a predetermined emotion felt by a user of an avatar of another person with whom the user is interacting. [Solution] An interaction determination unit (7) determines whether at least two avatars existing in a virtual reality space are interacting with each other. A habitual movement setting unit (6) sets a predetermined motion as a habitual movement of a user of each avatar existing in the virtual reality space. An emotion analysis unit (9) estimates the emotion of each avatar. When a predetermined emotion of a user of a second avatar, which is another avatar, toward a first avatar among the at least two avatars determined by the interaction determination unit (7) to be interacting with each other is equal to or greater than a predetermined threshold, an avatar movement control unit (3) generates an avatar control signal to control the first avatar to make the first avatar perform the movement set in the habitual movement setting unit (6).

Description

[Technical Field]

[0001] The present invention relates to an avatar control device, an avatar control method, and an avatar control program. [Background technology]

[0002] In recent years, in virtual reality spaces, each user moves an avatar generated for that user and the avatars interact with each other (see Patent Document 1). Typically, a server that controls the avatars and a user terminal carried by each user communicate with each other via a network, allowing the avatars corresponding to each user to interact in the virtual reality space. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Laid-Open No. 2005-276103 Summary of the Invention [Problem to be solved by the invention]

[0004] Avatars in virtual reality spaces may be generated to resemble the appearance of each user, or may resemble the appearance of a completely different person, or may be anthropomorphized animals or characters. Therefore, when avatars of family members or friends are interacting with each other in a virtual reality space, it may be difficult to know whether the avatars they are interacting with are truly the avatars of their family members or friends. This is because a third party may be impersonating the avatars of their family members or friends.

[0005] Therefore, for example, the user of the other avatar may be interacting with their own avatar while feeling negative emotions such as distrust, discomfort, or caution as a predetermined emotion. If the other avatar is feeling negative emotions, it is necessary to reduce the negative emotions.

[0006] An object of the present invention is to provide an avatar control device, an avatar control method, and an avatar control program that can alleviate a predetermined emotion felt by a user of an avatar with whom the user is interacting. [Means for solving the problem]

[0007] The present invention provides an avatar control device comprising: an interaction determination unit that determines whether at least two avatars existing in a virtual reality space are interacting with each other; a habitual movement setting unit that sets a predetermined action as a habitual movement of a user of each avatar existing in the virtual reality space; an emotion analysis unit that estimates the emotion of each avatar; and an avatar movement control unit that generates an avatar control signal that causes the first avatar to perform the movement set in the habitual movement setting unit when the predetermined emotion of a user of a second avatar, which is another avatar, toward a first avatar among the at least two avatars determined by the interaction determination unit to be interacting with each other, is equal to or greater than a predetermined threshold.

[0008] The present invention provides an avatar control method that determines whether at least two avatars existing in a virtual reality space are interacting with each other, estimates the emotions of each avatar existing in the virtual reality space, and, if a predetermined emotion of a user of a second avatar, which is another avatar, toward a first avatar of the at least two avatars determined to be interacting with each other, is equal to or greater than a predetermined threshold, generates an avatar control signal that causes the first avatar to perform a movement that is set in a habitual movement setting unit that sets a predetermined movement as a habitual movement of the user of each avatar.

[0009] The present invention provides an avatar control program that causes a computer to execute the steps of determining whether at least two avatars existing in a virtual reality space are interacting with each other, estimating the emotions of each avatar existing in the virtual reality space, and, if a predetermined emotion of a user of a second avatar, which is another avatar, toward a first avatar among the at least two avatars determined to be interacting with each other, is equal to or greater than a predetermined threshold, generating an avatar control signal that causes the first avatar to perform a movement that is set in a habitual movement setting unit that sets a predetermined movement as a habitual movement of the user of each avatar. [Effects of the Invention]

[0010] According to the avatar control device, avatar control method, and avatar control program of the present invention, it is possible to alleviate a predetermined emotion felt by the user of the avatar of another person with whom one is interacting. [Brief explanation of the drawings]

[0011] [Figure 1] FIG. 1 is a block diagram showing a state in which an avatar control server and a user terminal that constitute an avatar control device according to one embodiment are connected via a network. [Figure 2] FIG. 2 is a diagram showing an example of a virtual reality space image in which a plurality of avatars exist, generated by the avatar control server shown in FIG. [Figure 3] FIG. 3 is a diagram conceptually showing an example of registration information registered in a relationship registration unit included in the avatar control server shown in FIG. [Figure 4] FIG. 4 is a diagram conceptually showing an example of habitual movement setting information set in a habitual movement setting unit included in the avatar control server shown in FIG. [Figure 5] FIG. 5 is a diagram showing an example of a state in which an avatar is moving in accordance with the habitual movements of the user of the avatar. [Figure 6]FIG. 6 is a flowchart showing a first example of the operation of an avatar control server according to an embodiment, an avatar control method according to an embodiment, and processing that an avatar control program according to an embodiment causes a computer to execute. [Figure 7] FIG. 7 is a flowchart showing a second example of the operation of the avatar control server according to an embodiment, the avatar control method according to an embodiment, and the processing that the avatar control program according to an embodiment causes a computer to execute. DETAILED DESCRIPTION OF THE INVENTION

[0012] An avatar control device, an avatar control method, and an avatar control program according to an embodiment will be described below with reference to the accompanying drawings.

[0013] As shown in FIG. 1, avatar control server 100 is connected to user terminals 20a-20c and other user terminals not shown via network 50. Any user terminal among user terminals 20a-20c and other user terminals not shown will be referred to as user terminal 20 without specifying which one it is. The number of user terminals 20 connected to avatar control server 100 via network 50 may be one or any number greater than one. Typically, network 50 is the Internet.

[0014] As shown in the configuration of user terminal 20a, user terminal 20 includes a control unit 21, a motion sensor 22, a camera 23, an operation unit 24, a communication unit 25, and an image display unit 26. The control unit 21 can be configured as a computer central processing unit (CPU). The motion sensor 22 is, for example, a sensor such as an acceleration sensor or a gyro sensor attached to multiple locations on the body of a user holding user terminal 20a. The motion sensor 22 may be any sensor that can detect the movement or posture of the user's body. The motion sensor 22 generates a motion detection signal corresponding to the user's movement (including posture).

[0015] The camera 23 captures at least the face of the user to acquire the user's facial expression. The camera 23 may also perform a function equivalent to the motion sensor 22 by capturing images of the user including parts other than the face. The operation unit 24 is a keyboard, a mouse, or the like. Instead of the motion sensor 22 or the camera 23 generating a motion detection signal corresponding to the user's motion, the user may operate the operation unit 24 to generate a motion instruction signal that instructs the movement of an avatar corresponding to the user.

[0016] A motion detection signal generated by motion sensor 22 or a motion instruction signal input by the user operating operation unit 24 is transmitted to avatar control server 100 via communication unit 25 and network 50. An image signal of the user captured by camera 23 is also transmitted to avatar control server 100 via communication unit 25 and network 50. Image display unit 26 displays an image of a virtual reality space containing one or more avatars, which is generated by avatar control server 100 and transmitted to user terminal 20 as described below. The virtual reality space is sometimes referred to as a metaverse space.

[0017] Avatar control server 100 constitutes an avatar control device according to one embodiment. Avatar control server 100 includes a communication unit 1, a movement signal generation unit 2, an avatar movement control unit 3, a relationship registration unit 4, a habitual movement detection unit 5, a habitual movement setting unit 6, an interaction determination unit 7, a relationship determination unit 8, an emotion analysis unit 9, an avatar position control unit 10, an avatar generation unit 11, an avatar / background setting unit 12, and a rendering unit 13. Avatar generation unit 11 includes a calculation unit 111. Avatar control server 100 can be configured as a computer including a processor and a memory.

[0018] The communication unit 1 can be configured with a communication circuit that communicates with each user terminal 20 using a communication protocol such as the Internet Protocol. The relationship registration unit 4, habitual movement setting unit 6, and avatar / background setting unit 12 can be configured with memory. The movement signal generation unit 2, avatar movement control unit 3, habitual movement detection unit 5, interaction determination unit 7, relationship determination unit 8, emotion analysis unit 9, avatar position control unit 10, avatar generation unit 11, and rendering unit 13 can be configured with a processor.

[0019] The avatar control server 100 may functionally realize the configuration shown in FIG. 1 by including a non-transitory storage medium that stores an avatar control program according to one embodiment and a central processing unit that executes the avatar control program.

[0020] The communication unit 1 receives a motion detection signal or a motion instruction signal and supplies it to the motion signal generation unit 2. The motion signal generation unit 2 generates a motion signal for moving an avatar corresponding to each user based on the input motion detection signal or motion instruction signal. The avatar movement control unit 3 generates an avatar control signal for controlling each avatar to move in accordance with the motion signal and supplies it to the avatar generation unit 11.

[0021] The calculation unit 111 of the avatar generation unit 11 generates each avatar by drawing an avatar that moves in accordance with the avatar control signal. The avatar and background setting unit 12 sets the form of the avatar that the avatar generation unit 11 will generate. The avatar and background setting unit 12 may be configured to set multiple avatar forms, allowing each user to select one of the forms as the avatar form corresponding to the user. Each user may be configured to upload the avatar form to the avatar and background setting unit 12 via the user terminal 20.

[0022] The avatar / background setting unit 12 sets background data that determines the three-dimensional virtual reality space in which the avatar is located and moves. If the avatar is a two-dimensional image, the background data may be two-dimensional image data. The rendering unit 13 acquires the background data from the avatar / background setting unit 12 and performs rendering so that the avatar generated by the avatar generation unit 11 is positioned in the virtual reality space based on the background data.

[0023] Each user can operate the operation unit 24 of the user terminal 20 to move the avatar within the virtual reality space. The control unit 21 of the user terminal 20 generates a movement control signal for moving the avatar in response to the operation of the operation unit 24. The movement control signal is transmitted to the avatar control server 100 via the communication unit 25 and the network 50. The communication unit 1 of the avatar control server 100 receives the movement control signal and supplies it to the avatar position control unit 10. The avatar position control unit 10 controls the rendering unit 13 to move the avatar within the virtual reality space in accordance with the movement control signal.

[0024] FIG. 2 shows an example of a virtual reality space image containing multiple avatars generated by avatar control server 100. In the example shown in FIG. 2, five avatars, avatars Av0 to Av4, exist within virtual reality space 300. Avatar control server 100 transmits image data representing the virtual reality space image shown in FIG. 2 to each user terminal 20 via communication unit 1 and network 50. Avatar Av0 shown in FIG. 2 is the avatar corresponding to the user of user terminal 20a. Communication unit 25 receives the image data representing the virtual reality space image, and control unit 21 controls image display unit 26 to display the virtual reality space image.

[0025] Returning to FIG. 1, the relationship registration unit 4 registers the relationships between the avatar corresponding to each user and other avatars. Each user operates the operation unit 24 to register the relationships between their own avatar and other avatars. FIG. 3 conceptually shows an example of registration information for avatar Av0. The IDs of avatars Av0, Av1, and Av4 are Av0, Av1, and Av4, respectively. The name of avatar Av0, which is the user himself, is Michael. The name of avatar Av1 is Robert, and the relationship is family. The name of avatar Av4 is Sarah, and the relationship is friend.

[0026] Not all avatars other than oneself that exist within virtual reality space 300 are necessarily registered in the registration information. For example, avatars Av3 and Av4 may be avatars that are not registered in the registration information of avatar Av0. Not all avatars registered in the registration information of avatar Av0 are necessarily present within virtual reality space 300. In addition to names and relationships, avatar registration information may also include profiles.

[0027] Returning to FIG. 1, the habitual movement detection unit 5 detects the habitual movements of each user by, for example, analyzing the captured image signals of each user for a predetermined period, such as one month. The habitual movements of each user are so-called habits. Examples of habits include rubbing the nose, touching hair, and frequent blinking. The habitual movements may be the movements of a part of the body, such as the hands, or the movements of a part of the face that creates facial expressions. The habitual movements of each user may be the same movements for multiple users.

[0028] Furthermore, the habitual movement detection unit 5 may register not only one habitual movement pattern but also multiple habitual movement patterns and detect a movement that matches or is similar to a registered pattern as a habitual movement.The habitual movement detection unit 5 may count each movement that matches or is similar to a registered pattern and detect the movement that appears most frequently as the habitual movement.

[0029] For example, the habitual movement detection unit 5 may detect habitual movements by known pattern matching. The habitual movement detection unit 5 performs pattern matching on the entire user, or on a predetermined region of the user's head or each part of the user's face, to extract feature points of the user. For example, the habitual movement detection unit 5 may extract feature points of the user using a motion vector. The habitual movement detection unit 5 generates information indicating the user's movements based on video captured by the camera 23.

[0030] The habitual motion detection unit 5 detects the user's motion vector using, for example, a video that captures the entire user, and generates user motion information based on the detected motion vector. The habitual motion detection unit 5 may detect the motion vector using, for example, the current frame and the previous frame of the user's video. The habitual motion detection unit 5 performs pattern matching on the detected motion vector to extract the user's feature points. Alternatively, the habitual motion detection unit 5 may analyze facial movements in a time series to extract the feature points.

[0031] Note that, as long as the habitual movement detection unit 5 can detect natural movements of the user, it is not limited to generating user movement information using images captured by the camera 23. The habitual movement detection unit 5 may extract user feature points by machine learning or the like other than pattern matching.

[0032] After extracting the feature points of the user's movements, the habitual movement detection unit 5 calculates the positional relationship of the feature points. Based on the positional relationship of the feature points, the habitual movement detection unit 5 identifies the user's movements captured in the captured video as a movement pattern with the same or similar positional relationship of the feature points from among pre-prepared human movement patterns, thereby detecting the user's movements. For example, after extracting the feature points of the user's movements, the habitual movement detection unit 5 records the positional relationship of the feature points as coordinate positions. For example, the habitual movement detection unit 5 refers to a database in which pre-prepared human movements are linked to the feature points of the movements, and if there is movement data corresponding to the same or similar positional relationship of the feature points of the user's movements, it identifies the user's movement as a movement pattern such as "touching hair."

[0033] Furthermore, the habitual movement detection unit 5 calculates the number of times a user's movement appears in the user's captured video. The habitual movement detection unit 5 counts the number of times a movement pattern appears based on the positional relationship of feature points in a database in which a person's habitual movements and movement feature points are linked and stored in advance. The habitual movement detection unit 5 counts the number of times a user's movement in the captured video appears to satisfy the positional relationship of feature points in the database. For example, if the user's movement in the captured video satisfies the positional relationship of feature points for "touching hair" in the database, the habitual movement detection unit 5 counts the number of times the movement "touching hair" appears as one.

[0034] The habitual movement detection unit 5 counts the user's movement in the captured video each time the positional relationship of the feature points in the database satisfies "touching hair," and stores the movement pattern in association with the number of times it appears. In this way, the habitual movement detection unit 5 calculates the number of times the user's movement pattern appears over, for example, a month. Then, the habitual movement detection unit 5 detects the movement pattern that appears most frequently over that month as the user's habitual movement.

[0035] Habitual movement setting unit 6 sets a predetermined movement as the habitual movement of the user of each avatar present in virtual reality space 300. Furthermore, if each user is aware of their own habitual movement, the system may be configured so that each user selects a movement pattern from a database stored in avatar control server 100 that associates people's movement patterns with names of habits, and manually sets the selected movement pattern as their own habitual movement in habitual movement setting unit 6. In this case, avatar control server 100 does not need to include habitual movement detection unit 5.

[0036] Furthermore, the habitual movement detection unit 5 may further subdivide the movement pattern and the number of occurrences for each person interacting with the user. For example, after extracting the feature points of the user's movement, the habitual movement detection unit 5 calculates the positional relationship of the user's feature points for each person interacting with the user at the time of image capture. The habitual movement detection unit 5 subdivides the identified movement pattern and each person interacting with the user and counts the number of occurrences. For example, if the positional relationship of the user's feature points corresponds to "touching hair" and the user was interacting with a family member when the user's movement was detected, the habitual movement detection unit 5 identifies the movement pattern as occurring when the user interacts with a family member, and counts the number of occurrences by linking it to the information about family members.

[0037] Similarly, the habitual movement detection unit 5 identifies each of the user's movements as a movement pattern for each person the user interacts with and counts the number of times it occurs. This allows the habitual movement setting unit 6 to set a habitual movement according to the relationship. Specifically, if the movement pattern that occurs most frequently when the user interacts with family members is "touching hair," the habitual movement setting unit 6 sets "touching hair" as a habitual movement associated with family members. This allows the generated control signal to be changed depending on the relationship.

[0038] Furthermore, the user may manually set a habitual movement in the habitual movement setting unit 6 for each relationship between the avatar corresponding to the user and other avatars.

[0039] 4 shows habitual movement setting information set in the habitual movement setting unit 6. In the habitual movement setting information, habitual movements of the user of each avatar are set in association with the ID of each avatar.

[0040] The interaction determination unit 7 determines whether at least two avatars are interacting with each other. At least two avatars are interacting with each other means that at least two avatars are conversing with each other or are located in close proximity to each other. Although not shown in FIG. 1, each user terminal 20 may be equipped with a microphone and a speaker (or a headphone connection terminal) so that it can converse with other avatars, or may be configured so that it can converse with other avatars by chatting using text input.

[0041] The interaction determination unit 7 can determine whether an avatar is interacting with another avatar based on a conversation with the other avatar provided by the communication unit 1. The interaction determination unit 7 may also determine whether an avatar is interacting with another avatar based on words included in the conversation. For example, the interaction determination unit 7 acquires the user's voice picked up by a microphone of each user terminal 20. The interaction determination unit 7 performs voice recognition on the acquired voice. Furthermore, if a specific keyword is present in the voice recognition result, the interaction determination unit 7 determines that the user and another avatar are interacting with each other. In this way, if it is determined that the user and another avatar are interacting with each other, the interaction determination unit 7 can determine that the user and another avatar are interacting with each other.

[0042] Specifically, the AC determination unit 7 performs, for example, A / D conversion on the voices uttered by the other avatars supplied from the communication unit 1, and detects, from the digitized voice signal, a voice section corresponding to the content uttered by the other avatar. The AC determination unit 7 extracts voice data or feature quantities from the detected voice section. Depending on the usage environment of the AC determination unit 7, noise reduction processing or echo reduction processing such as spectrum subtraction by signal processing may be performed before extracting feature quantities from the voice data.

[0043] The AC determination unit 7 refers to a speech recognition dictionary stored in a speech recognition dictionary database (not shown), performs recognition processing on the extracted speech data or features of the speech data, and obtains a recognition result. The recognition result obtained by the AC determination unit 7 includes at least one of speech section information, a recognition result character string, identification information such as an ID associated with the recognition result character string, or a recognition score indicating likelihood. Here, the recognition result character string is a syllable string, a word, or a word string. The recognition processing by the AC determination unit 7 is performed using a common method such as the HMM (Hidden Markov Model) method.

[0044] The timing at which the interaction determination unit 7 starts the voice recognition process can be set as appropriate. For example, the interaction determination unit 7 can be configured to start voice recognition when each user logs in to the virtual reality space for voice recognition.

[0045] The speech recognition dictionary database stores a speech recognition dictionary. The speech recognition dictionary is a dictionary that the speech recognition unit refers to when performing speech recognition processing on the speech of other avatars, and it defines words that are the subject of speech recognition. Words can be defined in the speech recognition dictionary using common methods, such as enumeration using BNF (Backus-Naur Form), describing word sequences in a network-like format using network grammar, or probabilistic modeling of word chains using a statistical language model.

[0046] Next, the interaction determination unit 7 searches for a keyword registered in a keyword storage unit (not shown) in the recognition result character string described in the recognition result based on the voice input from the microphone. If the keyword registered in the keyword storage unit is present in the recognition result character string, the interaction determination unit 7 determines that the user and another avatar are having a conversation. In this way, when the interaction determination unit 7 determines that the user and another avatar are having a conversation, it can determine that the user and another avatar are interacting.

[0047] The keyword storage unit stores keywords that may appear in a conversation between speakers. Here, a conversation between speakers refers to, for example, a conversation between avatars in the same virtual reality space, or an utterance from one avatar in the virtual reality space to another avatar in the virtual reality space. Furthermore, keywords that may appear in a conversation between speakers refer to, for example, a person's name (such as a surname, first name, full name, or nickname) or a word used to address someone (such as "hey," "hey," or "hey").

[0048] Regarding names, if all names that are expected to appear in conversations between speakers are stored as keywords in the keyword storage unit, there is a high possibility that voices that are not conversations between speakers will be erroneously detected as conversations. To avoid such erroneous detection, the user terminal 20 may perform a process of storing names of speakers that have been estimated in advance from other avatars as keywords in the keyword storage unit.

[0049] Furthermore, the interaction determination unit 7 may determine whether the avatars are interacting with each other based on a virtual reality space image. For example, when another avatar is displayed on the image display unit 26 and the user's avatar approaches a predetermined distance, the interaction determination unit 7 determines that the user's avatar and the other avatar are interacting with each other. Specifically, to calculate the distance between the user's avatar and the other avatar, the interaction determination unit 7 acquires the position coordinates of the user's avatar and the position coordinates of the other avatar and calculates the distance between these coordinates. If this distance is equal to or less than a predetermined threshold, the interaction determination unit 7 determines that the user's avatar and the other avatar are interacting with each other. In the example shown in FIG. 2, avatar Av1 is located near avatar Av0. Therefore, the interaction determination unit 7 determines that avatar Av0 is interacting with avatar Av1. If avatar Av0 is the first avatar, avatar Av1 is the second avatar.

[0050] Take an example where the interaction determination unit 7 determines that avatar Av0 is interacting with avatar Av1. The relationship determination unit 8 determines the relationship between avatar Av0 and avatar Av1, with which avatar Av0 is interacting, in accordance with the relationships registered in the relationship registration unit 4. As shown in FIG. 3, avatar Av1 is a family member. Avatars existing in virtual reality space 300 are displayed with names, and profiles can be displayed by performing a predetermined operation, so the user of avatar Av0 can recognize that avatar Av1 is a family avatar named Robert.

[0051] Similarly, based on the registration information of avatar Av1 (not shown), the user of avatar Av1 can recognize that the user of avatar Av0 is a family avatar named Michael. The relationship determination result of each avatar by the relationship determination unit 8 is supplied to the avatar movement control unit 3.

[0052] A friendly relationship may also be defined as a friend relationship between a user and another user. For example, when a friend request is made in a virtual reality space and mutually accepted, the relationship determination unit 8 can determine that the user and the other user are in a friend relationship and a friendly relationship. Furthermore, even if the relationship is a friend relationship, the relationship determination unit 8 may quantify the intimacy level based on the number of times the users have played together or the content of their conversations. Alternatively, the relationship determination unit 8 may quantify the intimacy level by measuring the number of times the users have contacted each other via chat or the number of times they have met as a couple or in a group.

[0053] Next, the emotion analysis unit 9 estimates the emotion of the user of avatar Av1 who is interacting with the user. The emotion analysis unit 9 analyzes the emotion of the user of avatar Av1 based on a captured image signal transmitted from the user terminal 20 of the user of avatar Av1. The emotion analysis unit 9 can analyze the user's emotion using a known technique for analyzing human emotions based on a captured image of a human face.

[0054] For example, the emotion analysis unit 9 analyzes video captured during operation to estimate the user's emotions. For example, the emotion analysis unit 9 uses a trained model for facial expression recognition that uses pattern matching or machine learning (e.g., deep learning) to analyze image information including the user's face. The trained model for facial expression recognition extracts feature points from the contours of the user's face or each part of the face in the image and tracks the movement of the feature points to recognize the user's facial expression and estimate the user's emotions. Based on the captured image, the emotion analysis unit 9 analyzes the facial expression (corners of the mouth, etc.) of the user of avatar Av1 to determine which of multiple preset emotion types the emotion of the user of avatar Av1 belongs to.

[0055] The emotion analysis unit 9 uses, for example, Plutchik's wheel of emotions to classify emotions into eight basic emotions (joy, trust, fear, surprise, sadness, disgust, anger, and anticipation) and applied emotions, which are combinations of two adjacent emotions, and assigns them to one of the patterns on the wheel of emotions. For example, the emotion analysis unit 9 analyzes the emotion opposite to "trust" or a combination of "fear" and "surprise" as "distrust." The emotion analysis unit 9 analyzes the combination of "surprise" and "disgust" as "discomfort." The emotion analysis unit 9 analyzes the combination of "fear" and "anticipation" as "alarm." In this way, the emotion analysis unit 9 learns by linking facial patterns with emotions.

[0056] The emotion analysis unit 9 quantifies emotions according to their application to the emotion wheel. For example, a correspondence table that associates types of emotions on Plutchik's emotion wheel with numerical values ​​is stored in advance. The emotion analysis unit 9 refers to this correspondence table and calculates a numerical value representing the emotion based on the emotion analysis results. For example, positive emotions (positive emotions) such as joy are represented by a positive value, and negative emotions (negative emotions) such as sadness are represented by a negative value. In this case, the emotion analysis unit 9 increases the value (absolute value) representing the emotion, for example, the stronger the emotion (the further inside the emotion wheel it is). The emotion may be classified into two stages to determine whether it belongs to a positive emotion or a negative emotion. In this case, positive emotions and negative emotions become a total of two types of classified emotions.

[0057] The emotion analysis unit 9 is not limited to analyzing the user's emotion based on the captured image signal transmitted from the user terminal 20, but may analyze the user's emotion based on the user's biometric information. The analysis result of the emotion of the user of the avatar Av1 by the emotion analysis unit 9 is supplied to the avatar movement control unit 3.

[0058] The emotion analysis unit 9 may transmit the analysis result of the emotion of the user of the avatar Av1 to the user terminal 20a of the user of the avatar Av0.

[0059] Based on the analysis results of the emotion of the user of avatar Av1 input to the avatar movement control unit 3, it is assumed that the user of avatar Av1 is feeling a negative emotion, as an example of a predetermined emotion. For example, the user of avatar Av1 may not be sure that avatar Av0, the avatar of the person with whom he or she is interacting, is truly a family member's avatar, and may feel negative emotions such as distrust, discomfort, or caution. In this case, the avatar movement control unit 3 generates an avatar control signal to control avatar Av0 so that avatar Av0 moves in accordance with the habitual movement of avatar Av0 set in the habitual movement setting unit 6, and supplies the generated signal to the avatar generation unit 11.

[0060] The avatar movement control unit 3 may ignore the movement signal provided by the movement signal generation unit 2 and generate an avatar control signal that controls the avatar Av0 to move in accordance with the habitual movement of the avatar Av0 set in the habitual movement setting unit 6. The avatar movement control unit 3 may generate an avatar control signal that controls the avatar Av0 to move in accordance with the habitual movement of the avatar Av0 set in the habitual movement setting unit 6, in addition to the movement in accordance with the movement signal provided by the movement signal generation unit 2. However, the avatar Av0 may perform both the movement in accordance with the movement signal and the movement in accordance with the habitual movement only if it is possible to cause the avatar Av0 to perform both movements without contradiction. For example, if a movement such as "touching hair" is to be generated, the avatar may be controlled by generating a movement in accordance with the movement signal for the user's legs or facial expression.

[0061] The avatar movement control unit 3 generates an avatar control signal for controlling avatar Av0 to make at least the avatar Av0 move in accordance with the habitual movement of avatar Av0 set in the habitual movement setting unit 6. In this case, the avatar movement control unit 3 may set the condition only that the user of avatar Av1 is feeling negative emotions. Alternatively, the avatar movement control unit 3 may set the condition only that the user of avatar Av1 is feeling negative emotions and that the user of avatar Av0 has issued an instruction from the user terminal 20a to make the avatar Av0 move in a habitual manner.

[0062] The avatar movement control unit 3 may generate an avatar control signal to control the avatar Av0 based on the relationship between the avatar Av1 and the avatar Av0, which is based on the relationship determination result supplied from the relationship determination unit 8, and the degree of negative emotion felt by the user of the avatar Av1, which is based on the analysis result supplied from the emotion analysis unit 9.

[0063] As an example, since avatar Av1 is a family member of avatar Av0, if the degree of negative emotion felt by the user of avatar Av1 is equal to or greater than a first threshold as a predetermined threshold, the avatar movement control unit 3 generates an avatar control signal that controls avatar Av0 to make avatar Av0 move in accordance with habitual movements.

[0064] Suppose the avatar interacting with avatar Av0 is avatar Av4, which is not a family member but a friend of avatar Av0. In this case, if the degree of negative emotion felt by the user of avatar Av1 is equal to or greater than a second threshold, which is greater than the first threshold, the avatar movement control unit 3 generates an avatar control signal to control avatar Av0 to move in accordance with habitual movements. That is, if the relationship determination unit 8 determines that the relationship between at least two avatars is a predetermined relationship, the avatar movement control unit 3 changes the predetermined threshold.

[0065] In this way, the avatar movement control unit 3 may set the threshold for determining whether or not to cause one's own avatar to move in accordance with the habitual movement of one's own avatar depending on the relationship with other avatars interacting with one's own avatar. The deeper (more intimate) the relationship with other avatars, the lower the threshold should be set, and the shallower (less intimate) the threshold should be set, the higher the threshold should be. It is not desirable for other avatars to harbor negative emotions even if they are family or friends. Therefore, if the threshold is lowered the deeper the relationship with other avatars, it becomes easier to cause one's own avatar to move in a habitual way, which helps to reduce negative emotions.

[0066] As shown in Fig. 4, the habitual movement of avatar Av0 is touching hair with the right hand. Avatar movement control unit 3 generates an avatar control signal that controls avatar Av0 to make the habitual movement of touching hair with the right hand. Fig. 5 shows avatar Av0 generated by avatar generation unit 11 by drawing it to move in accordance with the avatar control signal.

[0067] If the user of avatar Av0 is a family member of the user of avatar Av1, the user of avatar Av1 is likely to recognize the daily habits of the user of avatar Av0. When avatar Av0 touches its hair with its right hand, the user of avatar Av1 becomes convinced that avatar Av0 is truly a family member avatar, or at least recognizes that there is a high possibility that avatar Av0 is truly a family member avatar. This can reduce or eliminate any negative feelings the user of avatar Av1 may have.

[0068] As described above, if the emotion analysis unit 9 transmits the analysis results of the emotions of the user of avatar Av1 to the user terminal 20a of the user of avatar Av0, the user of avatar Av0 can confirm whether the negative emotions felt by the user of avatar Av1 have been alleviated.

[0069] The avatar movement control unit 3 appropriately sets the intensity or duration for which the avatar Av0 performs the habitual movement. Specifically, the avatar movement control unit 3 may generate an avatar control signal for controlling the avatar Av0 so that the avatar Av0 performs a movement that is more exaggerated or emphasized than the habitual movement set by the habitual movement setting unit 6 of the user of the avatar Av0, or so that the avatar Av0 performs a movement corresponding to the habitual movement for a longer period of time than the habitual movement set by the habitual movement setting unit 6.

[0070] For example, if avatar Av1 interacting with avatar Av0 is determined to be a family member by the relationship determination unit 8, and the degree of negative emotion felt by the user of avatar Av1 is equal to or greater than a second threshold that is greater than the first threshold, the avatar movement control unit 3 generates a signal to exaggerate or emphasize the movement of avatar Av0 touching its hair with its right hand, or to make a movement according to a habitual movement for a longer period of time than the movement duration set by the habitual movement setting unit 6.

[0071] Specifically, the avatar movement control unit 3 generates an avatar control signal for a movement with a larger amount of change than the positional relationship of feature points in the movement set by the habitual movement setting unit 6. Alternatively, the avatar movement control unit 3 generates an avatar control signal to make the avatar Av0 touch its hair with its right hand for a longer period of time than the set habitual movement. This can reduce negative emotions when the user of the avatar with whom the avatar is interacting is feeling negative emotions.

[0072] Furthermore, if the degree of negative emotion felt by the user of avatar Av1 is determined to be equal to or greater than a first predetermined threshold, and the negative emotion felt by the user of avatar Av1 has not been alleviated for a predetermined period of time, the avatar movement control unit 3 may generate a signal to exaggerate or emphasize the movement of avatar Av0 touching its hair with its right hand, or to make a movement corresponding to a habitual movement for a longer period of time than the movement duration set by the habitual movement setting unit 6. In this way, if the negative emotion felt by the user of the avatar Av1 with whom avatar Av0 is interacting has not been alleviated despite the habitual movement of avatar Av0, the negative emotion can be alleviated by exaggerating or emphasizing the expression.

[0073] Avatar control server 100 shown in FIG. 1 includes habitual movement detection unit 5 and emotion analysis unit 9, but user terminal 20 may also include a configuration equivalent to habitual movement detection unit 5, or user terminal 20 may also include a configuration equivalent to emotion analysis unit 9. User terminal 20 may also include a configuration equivalent to each of habitual movement detection unit 5 and emotion analysis unit 9.

[0074] If user terminal 20 has a configuration equivalent to habitual movement detection unit 5, the habitual movements of each user detected in user terminal 20 may be transmitted to avatar control server 100 and set in habitual movement setting unit 6. If user terminal 20 has a configuration equivalent to emotion analysis unit 9, the analysis results of each user's emotion analyzed in user terminal 20 may be transmitted to avatar control server 100 and provided to avatar movement control unit 3.

[0075] Using the flowchart shown in the first example of Figure 6, we will explain the operation of avatar control server 100 according to one embodiment, an avatar control method according to one embodiment, and the processing that an avatar control program according to one embodiment causes a computer to execute.

[0076] 6, when processing by avatar control server 100 is started, the computer constituting avatar control server 100 uses interaction determination unit 7 to determine whether a first avatar is interacting with a second avatar (step S1). If interaction determination unit 7 determines that the first avatar is not interacting with the second avatar (step S1; NO), it generates an avatar control signal that causes the first avatar to move in accordance with a movement signal corresponding to the user's movement (step S7), and proceeds to step S8.

[0077] If the interaction determination unit 7 determines that the first avatar is interacting with the second avatar (step S1; YES), the computer causes the relationship determination unit 8 to determine the relationship between the first avatar and the second avatar (step S2). Next, the computer causes the avatar movement control unit 3 to select a threshold value according to the relationship (step S3). For example, if the relationship determination unit 8 determines that the relationship between the first avatar and the second avatar is "family," the avatar movement control unit 3 sets the threshold to the first threshold. Specifically, the avatar movement control unit 3 references a database in which prepared relationships are stored in association with thresholds of emotional levels, and sets the threshold value according to the relationship.

[0078] Next, the computer causes the emotion analysis unit 9 to estimate the emotion of the second avatar (step S4). For example, the emotion analysis unit 9 estimates the emotion of the second user by performing emotion analysis as described above.

[0079] Furthermore, the computer causes the avatar movement control unit 3 to determine whether the second avatar has a negative emotion equal to or greater than a threshold (step S5). For example, if the relationship determination unit 8 determines that the second user is a "family member," the avatar movement control unit 3 determines whether the negative emotion value is equal to or greater than a first threshold. If the avatar movement control unit 3 determines that the second avatar has a negative emotion equal to or greater than the threshold (step S5; YES), the avatar movement control unit 3 generates an avatar control signal that causes the first avatar to perform a habitual movement (step S6), and proceeds to step S9. For example, the avatar movement control unit 3 generates the avatar control signal by setting a predetermined movement set in the habitual movement setting unit 6 as the user's habitual movement. For example, the habitual movement setting unit 6 sets the movement pattern that appeared most frequently over a one-month period detected by the habitual movement detection unit 5 as the habitual movement.

[0080] On the other hand, if the avatar movement control unit 3 determines that the second avatar does not have negative emotions equal to or greater than the threshold (step S5; NO), it generates an avatar control signal that causes the first avatar to move in accordance with the movement signal corresponding to the user's movement (step S7), and transitions the process to step S8.

[0081] The computer generates a first avatar by drawing an avatar that moves in accordance with the avatar control signal using avatar generation unit 11 (step S8). Next, the computer determines whether or not to end the avatar control operation (step S9). If the avatar control operation is not to be ended (NO), avatar control server 100 returns to step S1 and repeats the subsequent steps. If the avatar control operation is to be ended (YES), avatar control server 100 ends the process. Note that if the user operates operation unit 24 to power off the user terminal or to disconnect from avatar control server 100, avatar control server 100 may end the avatar control operation of the avatar corresponding to user terminal 20 on which such an operation was performed.

[0082] 7, a process will be described in which the avatar movement control unit 3 changes a habitual movement depending on the relationship between the first user and the second user determined by the relationship determination unit 8. In FIG. 7, descriptions common to FIG. 6 may be omitted.

[0083] 7, when processing by avatar control server 100 is started, the computer constituting avatar control server 100 causes interaction determination unit 7 to determine whether the first avatar is interacting with the second avatar (step S21). If interaction determination unit 7 determines that the first avatar is not interacting with the second avatar (step S21; NO), it generates an avatar control signal that causes the first avatar to move in accordance with a movement signal corresponding to the user's movement (step S28), and proceeds to step S29.

[0084] If the interaction determination unit 7 determines that the first avatar is interacting with the second avatar (step S21; YES), the computer causes the relationship determination unit 8 to determine the relationship between the first avatar and the second avatar (step S22). The computer causes the avatar movement control unit 3 to select a threshold value according to the relationship (step S23), and in parallel, select a habitual movement according to the relationship (step S24). For example, if the relationship determination unit 8 determines that the relationship between the first avatar and the second avatar is "family," the avatar movement control unit 3 generates an avatar control signal for the habitual movement associated with "family."

[0085] Next, the computer uses the emotion analysis unit 9 to estimate the emotion of the second avatar (step S25). The computer uses the avatar movement control unit 3 to determine whether the second avatar is feeling a negative emotion equal to or greater than a threshold (step S26). If the avatar movement control unit 3 determines that the second avatar is feeling a negative emotion equal to or greater than the threshold (step S26; YES), it generates an avatar control signal that causes the first avatar to make a habitual movement according to the relationship (step S27), and proceeds to step S29.

[0086] On the other hand, if the avatar movement control unit 3 determines that the second avatar does not have negative emotions equal to or greater than the threshold (step S26; NO), it generates an avatar control signal that causes the first avatar to move in accordance with the movement signal corresponding to the user's movement (step S28), and transitions the process to step S29.

[0087] The computer generates a first avatar by drawing an avatar that moves in accordance with the avatar control signal in avatar generation unit 11 (step S29). Next, the computer determines whether or not to end the avatar control operation (step S30). If the avatar control operation is not to end (NO), avatar control server 100 returns the process to step S21 and repeats the subsequent processes. If the avatar control operation is to end (YES), avatar control server 100 ends the process.

[0088] In this way, avatar control server 100 can alleviate negative emotions when the user of the avatar with whom you are interacting is feeling negative emotions.

[0089] The present invention is not limited to the above-described embodiment, and various modifications are possible without departing from the gist of the present invention. [Explanation of symbols]

[0090] 1,25 Communications Department 2. Motion signal generator 3 Avatar movement control section 4 Relationship Registration Section 5 Habitual movement detection unit 6 Habit Movement Settings 7 AC determination section 8 Relationship determination unit 9 Sentiment Analysis Department 10 Avatar position control section 11Avatar Generation Unit 12 Avatar and Background Settings 13 Rendering section 20a, 20b, 20c User terminal 21 Control section 22 Motion Sensor 23 Camera 24 Control section 26 Image display section 50 Network 100 Avatar Control Server 111 Arithmetic section 300 Virtual Reality Space Av0~Av4 Avatar

Claims

1. an interaction determination unit that determines whether at least two avatars present in the virtual reality space are interacting with each other; a habitual movement setting unit that sets a predetermined movement as a habitual movement of a user of each avatar existing in the virtual reality space; an emotion analysis unit for estimating the emotion of each of the avatars; an avatar movement control unit that generates an avatar control signal for making a first avatar perform the movement set in the habit movement setting unit when a predetermined emotion of a user of a second avatar, which is another avatar, toward a first avatar among the at least two avatars determined by the interaction determination unit to be interacting with each other is equal to or greater than a predetermined threshold; An avatar control device comprising:

2. a relationship determination unit that determines a relationship between the at least two avatars that are determined to be interacting by the interaction determination unit; When the relationship determination unit determines that the relationship between the at least two avatars is a predetermined relationship, the avatar movement control unit changes the predetermined threshold. The avatar control device according to claim 1 .

3. determining whether at least two avatars present in the virtual reality space are interacting with each other; Estimating the emotion of each avatar present in the virtual reality space; When a predetermined emotion of a user of a second avatar, which is another avatar, toward a first avatar among at least two avatars determined to be interacting with each other is equal to or greater than a predetermined threshold, an avatar control signal is generated to make the first avatar perform a movement set in a habit movement setting unit that sets a predetermined movement as a habit movement of the user of each avatar. Avatar control methods.

4. On the computer, determining whether at least two avatars present in the virtual reality space are interacting with each other; estimating an emotion of each avatar present in the virtual reality space; generating an avatar control signal for causing the first avatar to perform a movement set in a habit movement setting unit that sets a predetermined movement as a habit movement of the user of each avatar when a predetermined emotion of a user of a second avatar, which is another avatar, toward a first avatar among the at least two avatars determined to be interacting with each other, is equal to or greater than a predetermined threshold; An avatar control program that executes the above.

Images