Information processing device, information processing method, and program

Abstract

The present invention appropriately corrects audio data of an utterance by another user. An information processing device (30A) displays a virtual space (V) and an avatar (U) on an information terminal (10A), said information processing device comprising: an information acquisition unit (332) that acquires relationship information pertaining to a relationship between avatars (U) corresponding to a first user (5A) and a second user (5B); and an output control unit (336) that, on the basis of the relationship information, outputs information pertaining to correction content of audio data of the second user (5B) to the information terminal (10A) of the first user (5A).

Description

Information Processing Apparatus, Information Processing Method, and Program

[0001] The present disclosure relates to an information processing apparatus, an information processing method, and a program.

[0002] A method of changing voice data of a character in a game operated by a player based on vital data of the game player has been studied. Patent Document 1 describes an information processing apparatus that analyzes the mental state of a game player based on the vital data of the game player and generates virtual voice of a game character operated by the player based on the analyzed mental state.

[0003] Japanese Patent Application Laid-Open No. 2024-1520

[0004] In a virtual space, a user can communicate with other users they are not familiar with in addition to familiar users. At that time, depending on the relationship between users, the speech of other users may be perceived as abusive by the user, causing a feeling of disgust. Therefore, it is required to correct the voice data of the speech by other users to prevent the speech from other users from being perceived as abusive by the user.

[0005] This embodiment has been made in view of the above, and an object thereof is to provide an information processing apparatus, an information processing method, and a program that appropriately correct voice data of speech by other users.

[0006] The information processing apparatus according to this embodiment is an information processing apparatus that causes an information terminal to display a virtual space and an avatar, and includes an information acquisition unit that acquires relationship information, which is information regarding the relationship between the avatars corresponding to a first user and a second user, and an output control unit that outputs information regarding the correction content of the voice data of the second user to the information terminal of the first user based on the relationship information.

[0007] The information processing method according to this embodiment is an information processing method executed by an information processing device that displays a virtual space and an avatar on an information terminal, and comprises the steps of: acquiring relationship information which is information relating to the relationship between the avatars corresponding to a first user and a second user; and outputting information relating to the correction content of the second user's voice data to the first user's information terminal based on the relationship information.

[0008] The program according to this embodiment is a program executed by an information processing device that displays a virtual space and avatars on an information terminal, and causes a computer to perform the following steps: acquiring relationship information, which is information relating to the relationship between the avatars corresponding to a first user and a second user; and outputting information relating to the correction content of the second user's voice data to the first user's information terminal based on the relationship information.

[0009] According to this embodiment, it is possible to provide an information processing device, an information processing method, and a program that appropriately correct audio data of statements made by other users.

[0010] Figure 1 is a schematic diagram of the information processing system according to the first embodiment. Figure 2 is a schematic block diagram of the information terminal according to the first embodiment. Figure 3 is a schematic block diagram of the information processing device according to the first embodiment. Figure 4 is a flowchart illustrating the processing flow of the information processing device according to the first embodiment. Figure 5 is a schematic block diagram of the information terminal according to the second embodiment. Figure 6 is a schematic block diagram of the information processing device according to the second embodiment. Figure 7 is a flowchart illustrating the processing flow of the information processing device according to the second embodiment. Figure 8 is a schematic block diagram of the information processing device according to the third embodiment. Figure 9 is a schematic diagram showing an example of an NG action. Figure 10 is a flowchart illustrating the processing flow of the information processing device according to the third embodiment.

[0011] Preferred embodiments of the present disclosure will be described in detail below with reference to the attached drawings. However, this disclosure is not limited to these embodiments, and if there are multiple embodiments, they may be combinations of these embodiments.

[0012] (First Embodiment) The information processing system will be described using Figure 1. Figure 1 is a schematic diagram of the information processing system according to the first embodiment.

[0013] (Information Processing System) As shown in Figure 1, the information processing system 100 according to this embodiment comprises an information terminal 10A and an information processing device 30A. The information terminal 10A and the information processing device 30A are communicated with each other via a network N. The information processing system 100 provides a virtual space V to the user 5 using the information terminal 10A and the information processing device 30A. The information terminal 10A is an information terminal used by the user 5, and in this embodiment it is an HMD (Head Mount Display). The information terminal 10A provides the virtual space V to the user by displaying an image generated by the information processing device 30A.

[0014] The information processing device 30A is a server that generates images of the virtual space V and manages information terminals 10A used by different users 5. The information processing device 30A manages the images of the virtual space V generated by each information terminal 10A by communicating with each information terminal 10A via the network N. However, the information terminal 10A is not limited to the one that provides the virtual space V; for example, it may be a flat information terminal that displays images. In the example in Figure 1, the number of information terminals 10A to which the information processing device 30A is connected and communicates information can be arbitrary. That is, the number of information terminals 10A connected to the information processing device 30A is determined according to the number of users 5 accessing the virtual space V.

[0015] (Virtual Space) The virtual space V is not the real space (physical space) in which user 5 actually exists, but a space in which user 5 does not exist. The virtual space V is a virtual space that does not actually exist, i.e., VR (Vertical Reality), but it may also be a space that reproduces a real place far from where user 5 is located, i.e., MR (Mixed Reality). The information terminal 10A provides the virtual space V in accordance with user 5's movements (gaze) in the physical space. That is, based on the image of the virtual space V generated by the information processing device 30A, the information terminal 10A displays an image of the virtual space V as seen from the perspective of the avatar U of user 5, assuming that the avatar U of the information terminal 10A exists in the virtual space V (subjective image). Therefore, user 5 can recognize that they are in the virtual space V. However, the information terminal 10A is not limited to displaying subjective images, but may also display an image of the virtual space V from any field of view (for example, an overhead image of the virtual space V).

[0016] In this embodiment, as shown in Figure 1, the virtual space V is envisioned as a place where avatars U corresponding to users 5 who access the virtual space V can interact with each other and participate in games together. Avatars U can move freely within the virtual space V and can converse with each other. When another user 5 speaks to user 5 (when an avatar U operated by user 5 attempts to make contact with an avatar U operated by another user 5 in the virtual space), the information processing device 30A corrects the voice data of the other user 5 based on the relationship between user 5 and the other user 5.

[0017] Furthermore, in the following explanation, when describing the relationships between multiple users 5, a user 5 who attempts to make contact with Avatar Ua, operated by First User 5A, by operating their own Avatar U will be referred to as Second User 5B, and the Avatar U operated by Second User 5B will be referred to as Avatar Ub.

[0018] (Information Terminal) Figure 2 is a schematic block diagram of the information terminal according to this embodiment. The information terminal 10A can also be described as a computer, and as shown in Figure 2, it comprises an input unit 11, an output unit 12, a storage unit 13, a communication unit 14, and a control unit 15. The input unit 11 is a mechanism that accepts user operations, and may be, for example, a controller or microphone provided on an HMD. The output unit 12 is a mechanism that outputs information. In this embodiment, the output unit 12 is an HMD and includes a display device (display unit) that displays images and a speaker (audio output unit) that outputs sound. That is, the output unit 12 provides a virtual space V to the user by outputting information such as images and sound. The memory unit 13 is a memory that stores various information such as the calculation contents and programs of the control unit 15, and includes at least one of the following: a main memory such as RAM (Random Access Memory) and ROM (Read Only Memory), and an external memory such as HDD (Hard Disk Drive). The communication unit 14 is a communication module that communicates with an external device, in this case the information processing device 30A, such as an antenna. The information terminal 10A connects to the network N via wireless communication and communicates with the information processing device 30A, but wired communication may also be used, and the communication method may be arbitrary.

[0019] The control unit 15 is an arithmetic unit, or CPU (Central Processing Unit). The control unit 15 includes a relationship information acquisition unit 151 and a display control unit 152. The control unit 15 reads a program (software) from the storage unit 13 and executes it to realize the relationship information acquisition unit 151 and the display control unit 152, and then executes their processing. The control unit 15 may execute these processes with a single CPU, or it may have multiple CPUs, and the processing may be executed by those multiple CPUs. In addition, at least a part of the relationship information acquisition unit 151 and the display control unit 152 may be realized with hardware circuits. Furthermore, in this embodiment, the control unit 15 is a CPU provided in an information terminal 10A equipped with an output unit 12 (display unit), but it may be a device separate from the information terminal 10A equipped with the output unit 12.

[0020] The relationship information acquisition unit 151 acquires relationship information. The relationship information can be any information relating to the relationship between the first user 5A and other users 5 in the virtual space V. The relationship information according to this embodiment includes at least one of the following: friend information, which is information indicating a person with whom the first user 5A has a friendship relationship in the virtual space V; contact information of the first user 5A; social information, which is information relating to the number of times the first user 5A has met with other users in the virtual space V in pairs or groups; and game history information, which is information relating to games played by the first user 5A in the virtual space V. A friendship relationship is a relationship in which one user sends a friend request, the other user who receives the request approves it, and the two users recognize each other as friends. Contact information is information indicating the person the first user 5A has contacted in the past via the information terminal 10A and the number of times that contact occurred. Furthermore, the game history information can be any information relating to games that the first user 5A has played in the past. For example, it may include information about the types of games that the first user 5A has played in the past, information about who the first user 5A played with, which games and how many times, the number of wins and losses in those games, and information about the balance of the first user 5A's game skills and level. Preferably, the relationship information includes all of the first user 5A's friend information, first user 5A's contact information, first user 5A's social information, and first user 5A's game history information. The relationship information acquisition unit 151 may acquire the relationship information by any method. In this embodiment, the relationship information acquisition unit 151 acquires the first user 5A's friend information, first user 5A's contact information, first user 5A's social information, and first user 5A's game history information, which are stored in the storage unit 13 beforehand. In particular, contact information and social information are obtained from the first user 5A's past call history and the history of messages sent and received by the first user 5A, which are stored in the memory unit 13.

[0021] The display control unit 152 displays an image of the virtual space V on the output unit 12. In this embodiment, the display control unit 152 displays an image of the virtual space V as seen from the perspective of the avatar U (subjective image) on the output unit 12. In this embodiment, the information terminal 10A transmits operation information to the information processing unit 30A. The information processing unit 30A (the space generation unit 331 described later) calculates the position and orientation of the avatar U in the virtual space V based on the operation information and generates an image of the virtual space V in which the avatar U is located at that position and orientation. Then, the information processing unit 30A identifies the field of view of the avatar U based on the position and orientation of the avatar U, converts the image of the virtual space V into an image of the virtual space V as seen from the perspective of the avatar U (subjective image) based on the field of view of the avatar U, and transmits the converted image of the virtual space V (subjective image) to the information terminal 10A. The information terminal 10A displays the subjective image obtained from the information processing unit 30A on the output unit 12. However, it is not limited to this, and the information terminal 10A may generate a subjective image based on an image of the virtual space V. In this case, the display control unit 152 acquires an image of the virtual space V from the information processing device 30A and determines the field of view of the avatar U based on the operation information. Then, the display control unit 152 converts the image of the virtual space V into a subjective image based on the field of view of the avatar U and displays it on the output unit 12.

[0022] (Information Processing Device) Figure 3 is a schematic block diagram of the information processing device according to this embodiment. The information processing device 30A can also be described as a computer (server), and as shown in Figure 3, it comprises a storage unit 31, a communication unit 32, and a control unit 33. The storage unit 31 is a memory that stores various information such as calculation contents and programs of the control unit 33, and includes at least one of the following: RAM, main memory such as ROM, and external storage device such as HDD. The program for the control unit 33 stored in the storage unit 31 may be stored on a recording medium that the information processing device 30A can read. The communication unit 32 is a communication module that communicates with an external device, such as an antenna. The information processing device 30A may use either wireless communication or wired communication.

[0023] The control unit 33 is a computing device, i.e., a CPU. The control unit 33 includes a space generation unit 331, an information acquisition unit 332, a closeness calculation unit 333, a threshold setting unit 334, a correction content determination unit 335, and an output control unit 336. The control unit 33 reads a program (software) from the storage unit 31 and executes it to realize the space generation unit 331, the information acquisition unit 332, the closeness calculation unit 333, the threshold setting unit 334, the correction content determination unit 335, and the output control unit 336, and then executes their processing. The control unit 33 may execute these processes with a single CPU, or it may have multiple CPUs and execute the processing with those multiple CPUs. Furthermore, at least a part of the space generation unit 331, the information acquisition unit 332, the closeness calculation unit 333, the threshold setting unit 334, the correction content determination unit 335, and the output control unit 336 may be realized with hardware circuits.

[0024] (Processing of the Information Processing Device) The processing content of the information processing device 30A will be described below. However, the processing of the information processing device 30A described below may also be performed by the information terminal 10A that outputs images and sound to the user 5. In other words, the information terminal 10A that outputs images and sound to the user 5 may act as an information processing device and perform the processing of the information processing device 30A described below.

[0025] The timing of the audio correction performed by the information processing device 30A, described later, may be assumed to be at any time. However, the information processing device 30A according to this embodiment assumes that the audio correction will be performed during a predetermined period after the second user 5B has performed a predetermined action, when the first user 5A and the second user 5B are playing a game together in the virtual space V. For example, when the first user 5A and the second user 5B are playing a game against each other, the period from when the second user 5B loses or wins the first game and the next game begins may be assumed. Alternatively, for example, when the first user 5A and the second user 5B are playing a game in which they cooperate as a team, the period from when the first user 5A and the second user 5B's team loses the first game and the next game begins may be assumed.

[0026] Generally, after the outcome of a game is determined and the first game ends, there is often a period between the start of the next game during which user 5 does not perform any game operations (there is nothing to operate on). In such a situation, it is anticipated that abusive language and slander may be uttered between users. Therefore, the information processing device 30A according to this embodiment can prevent user 5A from feeling offended by slanderous language from user 5B by correcting the voice for a predetermined period after user 5B performs a predetermined action when user 5A and user 5B are playing a game together in the virtual space V.

[0027] (Space Generation Unit) The space generation unit 331 generates an image of the virtual space V. The space generation unit 331 generates an image of the virtual space V in which avatar Us corresponding to user 5 who has accessed the virtual space V can interact with each other and participate in games together. Note that the virtual space V is not limited to a place that provides games on the virtual space V, but may be any space that can be used for any purpose as long as it is a space in which avatar Us can work together.

[0028] (Information Acquisition Unit) The information acquisition unit 332 acquires relationship information from the relationship information acquisition unit 151 on the information terminal 10A. The acquired relationship information is used for calculating intimacy, which will be described later.

[0029] (Intimacy Calculation Unit) The intimacy calculation unit 333 calculates the intimacy level of the first user 5A with respect to the second user 5B. The intimacy level of the first user 5A with respect to the second user 5B, as referred to here, is an index that indicates the intimacy level of the first user 5A with respect to the second user 5B in the virtual space V. The higher the intimacy level, the less likely the first user 5A is to feel uncomfortable with and tolerate the words and actions of the second user 5B. The calculated intimacy level is used as a criterion for deciding whether or not to perform voice correction by the information processing device 30A by comparing it with the intimacy threshold described later.

[0030] In this embodiment, the intimacy calculation unit 333 acquires relationship information from the relationship information acquisition unit 151 in the information terminal 10A and calculates the intimacy level of the first user 5A with respect to the second user 5B based on this relationship information. More specifically, the intimacy calculation unit 333 calculates the intimacy level based on the first user 5A's friend information, first user 5A's contact information, first user 5A's social information, and first user 5A's game history information, which are included in the relationship information. The intimacy level based on the friend information, contact information, social information, and game history information of the first user 5A may be calculated by any method, but the intimacy calculation unit 333 according to this embodiment calculates each level of intimacy based on the friend information, contact information, social information, and game history information of the first user 5A, and calculates the sum of each level of intimacy as the intimacy level of the first user 5A with respect to the second user 5B based on the relationship information.

[0031] The method for calculating the level of intimacy based on the friend information of the first user 5A is arbitrary, and in this embodiment, the intimacy calculation unit 333 calculates a higher level of intimacy when the second user 5B is a friend of the first user 5A compared to when they are not friends. In this way, by calculating a higher level of intimacy when the second user 5B is a friend of the first user 5A compared to when they are not friends, it is possible to suppress voice correction by the information processing device 30A that the first user 5A and the second user 5B do not want, because the first user 5A and the second user 5B have a close friendship.

[0032] Furthermore, the method for calculating intimacy based on contact information is also arbitrary. In this embodiment, the intimacy calculation unit 333 calculates intimacy so that the more contacts the first user 5A and the second user 5B have, the higher the intimacy value. By calculating a higher intimacy value the more contacts the first user 5A and the second user 5B have, the more intimacy the second user 5B is expected to have a close relationship with the first user 5A, regardless of whether they are friends.

[0033] Furthermore, the method for calculating intimacy based on social information is also arbitrary. In this embodiment, the intimacy calculation unit 333 calculates a higher intimacy value the more often the first user 5A and the second user 5B have met alone in the virtual space V. By calculating a higher intimacy value the more often the first user 5A and the second user 5B have met alone in the virtual space V, it is possible to appropriately calculate the intimacy of the second user 5B, who is expected to be in a close relationship with the first user 5A because the first user 5A frequently interacts with the second user 5B alone in the virtual space V, regardless of whether they are friends.

[0034] Furthermore, the method for calculating intimacy based on game history information is also arbitrary. In this embodiment, the intimacy calculation unit 333 calculates a higher intimacy value the more times the first user 5A and the second user 5B have played the same game together. By calculating a higher intimacy value the more times the first user 5A and the second user 5B have played the same game together, it is possible to appropriately calculate the intimacy of the second user 5B, who is expected to be in a close relationship with the first user 5A because the first user 5A frequently plays the same game with the second user 5B in the virtual space V, regardless of whether they are friends.

[0035] (Threshold Setting Unit) The threshold setting unit 334 sets the intimacy threshold. The intimacy threshold referred to here is a threshold used to determine whether or not to apply correction to the voice data of the second user 5B when the second user 5B speaks to the first user 5A. The method for calculating the threshold can be any method, and in this embodiment, the threshold setting unit 334 sets the intimacy threshold by acquiring an intimacy threshold that has been stored in the storage unit 31 as a default value in advance as a predetermined threshold.

[0036] Furthermore, the threshold setting unit 334 may acquire multiple intimacy thresholds. More specifically, the threshold setting unit 334 may set multiple intimacy thresholds according to predetermined conditions. These predetermined conditions can be set arbitrarily, but for example, the threshold setting unit 334 may acquire and set a lower intimacy threshold from the storage unit 31 for a specific user 5 (a user with whom the user has a particularly close relationship) compared to the default intimacy threshold set for other users 5. This is because, among specific users 5 (users with whom the user has a particularly close relationship), one user's remarks are unlikely to be perceived as abusive by the other user. Rather, when specific users 5 (users with whom the user has a particularly close relationship) are having a conversation such as consultation, there is a possibility that they may not want their voice data to be corrected by the information processing device 30A. Therefore, by setting a lower intimacy threshold for a specific user 5 (a user with whom the user has a particularly close relationship) compared to other users 5, the threshold setting unit 334 can prevent their voice data from being corrected by the information processing device 30A.

[0037] (Correction Content Determination Unit) The correction content determination unit 335 determines what kind of correction to perform on the voice data of the second user 5B output to the information terminal 10A of the first user 5A when the second user 5B speaks to the first user 5A. More specifically, the correction content determination unit 335 determines what kind of correction to perform on the voice data of the second user 5B output to the information terminal 10A based on the intimacy level calculated by the intimacy level calculation unit 333 and the intimacy level threshold set by the threshold setting unit 334. Specifically, the correction content determination unit 335 in this embodiment compares the intimacy level and the intimacy level threshold, and determines what kind of correction to perform on the voice data of the second user 5B output to the information terminal 10A based on the comparison result. More specifically, the correction content determination unit 335 in this embodiment compares the intimacy level and the intimacy level threshold, and if the intimacy level is equal to or greater than the intimacy level threshold, it does not perform any correction on the voice data of the second user 5B output to the information terminal 10A. The correction content determination unit 335 then compares the intimacy level with the intimacy threshold, and if the intimacy level is less than the intimacy threshold, it performs a correction on the voice data of the second user 5B output to the information terminal 10A.

[0038] The correction content determined by the correction content determination unit 335 can be any content, but the correction content determination unit 335 in this implementation changes the frequency of the voice data of the second user 5B output to the information terminal 10A. Specifically, the correction content determination unit 335 changes the frequency of the voice data of the second user 5B output to the information terminal 10A to a predetermined frequency that has been set in advance. It is generally known that voice data that users find cute has a high frequency. Therefore, the correction content determination unit 335 in this implementation corrects the frequency of the voice data of the second user 5B output to the information terminal 10A to a frequency that the first user 5A has set in advance as the frequency of a cute voice, thereby reducing the aversion of the first user 5A who hears the content, even if the content of the voice data is slanderous.

[0039] Here, the method for changing the frequency of the audio data can be any method, and in this embodiment, the correction content determination unit 335 corrects the frequency of the audio data of the second user 5B using a pitch shifter (voice changer) (not shown) connected to the information processing device 30A. The pitch shifter referred to here is a device having a circuit that can arbitrarily change the frequency of the input audio data.

[0040] Furthermore, the correction content determination unit 335 may change the degree to which it corrects the voice data of the second user 5B output to the information terminal 10A of the first user 5A (the amount of change from the original voice data) based on the intimacy level and the intimacy threshold. More specifically, the correction content determination unit 335 according to this embodiment calculates the difference between the intimacy level and the intimacy threshold, and if the difference is small, it may reduce the degree of correction, and conversely, if the difference is large, it may increase the degree of correction. For example, as the difference between the intimacy level and the intimacy threshold increases, the degree of correction may be changed so that the frequency of the voice data of the second user 5B changes to approach a predetermined frequency. In other words, the correction content determination unit 335 according to this embodiment may change the correction content so that the greater the difference between the intimacy level and the intimacy threshold, the greater the correction of the voice data of the second user 5B from the original frequency, and the more it changes to a predetermined frequency set in advance.

[0041] Furthermore, the correction content determination unit 335 may correct only a portion of the second user 5B's voice data. More specifically, the correction content determination unit 335 may correct only predetermined words from the second user 5B's voice data. These predetermined words are stored in a table in the storage unit 31. Predetermined words and phrases are registered in advance in this table, and the correction content determination unit 335 performs corrections by referring to this table. Alternatively, the predetermined words may be configured to be manually set by the first user 5A. The content of the correction at that time can be set arbitrarily, and in this embodiment, the correction content determination unit 335 may, for example, correct only predetermined words from the second user 5B's voice data to silence, and output another pre-registered word as text data to the first user 5A's information terminal 10A as a word to replace the word that has been corrected to silence. Specifically, if the content of the conversation in the voice data of the second user 5B includes words that would cause considerable disgust to the first user 5A who hears them, such as "noisy, shut up, gross," the correction content determination unit 335 may perform a correction to silence words like "noisy, shut up, gross" in the voice data of the second user 5B, and output words that replace the silenced words as text data, for example, chat data, to the information terminal 10A of the first user 5A.

[0042] Alternatively, if the voice data of the second user 5B contains words that would cause considerable aversion to the first user 5A, such as "noisy," "shut up," or "gross," the correction content determination unit 335 may correct all of the second user 5B's voice data to silence and block the output of the second user 5B's voice data to the information terminal 10A. Furthermore, the method for extracting predetermined words from the voice data can be any method; for example, the correction content determination unit 335 converts the voice data into text data using speech recognition technology. A speech recognition model using deep learning is used for the speech recognition technology. Next, the correction content determination unit 335 uses natural language processing (NLP) technology to extract predetermined words from the converted text data. Specifically, it performs morphological analysis to divide the text data into word units. Then, it compares it with a pre-registered list of predetermined words and extracts matching words as predetermined words.

[0043] When the correction content determination unit 335 according to this embodiment corrects the voice data of the second user 5B, it corrects the voice data of the second user 5B based on the determined correction content, and outputs the corrected voice data to the output control unit 336 described later as information regarding the correction content of the voice data of the second user 5B. Further, the correction content determination unit 335 outputs information indicating what kind of correction is to be performed to the output control unit 336 described later as information regarding the correction content of the voice data of the second user 5B, and the information terminal 10A may perform the correction of the voice data of the second user 5B itself.

[0044] (Output control unit) The output control unit 336 outputs information (signal) regarding the correction content of the voice data of the second user 5B to the information terminal 10A based on the related information. In other words, the output control unit 336 controls the output mode of the voice data of the second user 5B. More specifically, the output control unit 336 according to this embodiment outputs the voice data of the second user 5B corrected by the correction content determination unit 335 to the information terminal 10A. The information terminal 10A receives information (data obtained by correcting the voice data of the second user) for controlling the output mode from the output control unit 336, and outputs the corrected voice data of the second user 5B.

[0045] (Processing Flow) Next, the processing flow according to this embodiment will be described. Figure 4 is a flowchart illustrating the processing flow of the information processing apparatus according to the first embodiment. As shown in Figure 4, the information acquisition unit 332 acquires relationship information acquired by the relationship information acquisition unit 151 at the information terminal 10A (step S10). For example, the information acquisition unit 332 acquires from the relationship information acquisition unit 151 whether or not the second user 5B is a friend of the first user 5A. The intimacy calculation unit 333 calculates the intimacy of the first user 5A towards the second user 5B based on the relationship information (step S11). For example, if the first user 5A and the second user 5B are friends, the intimacy calculation unit 333 calculates a higher intimacy compared to the case where they are not friends. If the calculated intimacy level is greater than or equal to the set intimacy threshold (step S12; YES), the correction content determination unit 335 does not correct the voice data of the second user 5B, and the output control unit 336 outputs the voice data of the second user 5B as is to the information terminal 10A (step S13). For example, even if the second user 5B, who is not friends with the first user 5A, loses to the first user 5A in a game, the second user 5B is unlikely to hurl insults at the first user 5A, and even if they say things like "shut up, be quiet, gross," the first user 5B is likely to take it as a joke, so there is no need to correct the voice data. Also, if the calculated intimacy level is less than the set intimacy threshold (step S12; No), the correction content determination unit 335 corrects the voice data of the second user 5B, and the output control unit 336 outputs the voice data of the second user 5B corrected by the correction content determination unit 335 to the information terminal 10A (step S14). For example, if User 2B, who is not friends with User 1A, loses a game to User 1A, User 2B may hurl insults at User 1A. Therefore, the correction content determination unit 335 can perform actions such as muting User 2B's voice. This prevents User 1A from feeling considerable disgust upon hearing the insults. This process is then terminated.

[0046] (Effect) As described above, the information processing apparatus 30A according to the present embodiment calculates the intimacy of the first user 5A with respect to the second user 5B based on the relationship information. In addition, the information processing apparatus 30A sets an intimacy threshold value. Then, when the calculated intimacy of the first user 5A with respect to the second user 5B is equal to or greater than the set intimacy threshold value, the information processing apparatus 30A outputs the voice data of the second user 5B as it is to the information terminal 10A, and when the calculated intimacy of the first user 5A with respect to the second user 5B is less than the set intimacy threshold value, the information processing apparatus 30A outputs the corrected voice data of the second user 5B to the information terminal 10A. According to the present disclosure, the information processing apparatus 30A can appropriately correct the voice data of the utterance by the second user 5B output to the information terminal 10A of the first user 5A based on the comparison result between the intimacy of the first user 5A with respect to the second user 5B and the intimacy threshold value.

[0047] In addition, in the present disclosure, the intimacy calculation unit 333 calculates the intimacy based on the friend information of the first user 5A, the contact information of the first user 5A, the social interaction information of the first user 5A, and the game history information of the first user 5A included in the relationship information. Thereby, the intimacy calculation unit 333 calculates the intimacy with respect to the second user 5B in consideration of not only whether the second user 5B is a friend of the first user 5A but also information such as how often they met and played games together in the virtual space V. Therefore, the intimacy of the first user 5A with respect to the second user 5B can be calculated more accurately.

[0048] In addition, in the present disclosure, the threshold setting unit 334 may acquire a plurality of intimacy threshold values. Thereby, for a specific user 5 (a user with a special acquaintance), the threshold setting unit 334 can set a lower intimacy threshold value compared to the default intimacy threshold value set for other users 5, thereby preventing the voice data of each other from being corrected by the information processing apparatus ......

[0049] (Second Embodiment) Next, a second embodiment will be described. The information terminal 10B according to the second embodiment differs from the first embodiment in that the control unit 15 further includes a user information acquisition unit 153. Also, the information processing device 30B according to the second embodiment differs from the first embodiment in that the control unit 33 further includes an emotion estimation unit 337. Note that other configurations are the same as those of the first embodiment and will not be described.

[0050] (Information Terminal) Figure 5 is a schematic block diagram of the information terminal according to this embodiment. As shown in Figure 5, the information terminal 10B includes a user information acquisition unit 153 in the control unit 15.

[0051] The user information acquisition unit 153 acquires user information, which is information relating to user 5 in the real world. User information can be any information relating to user 5 in the real world, but the user information in this embodiment includes at least one of the following: operation information, user 5's facial expression information, and user 5's biometric information.

[0052] In this context, operation information refers to information including information about actions taken by user 5 in the real world to operate their avatar U in the virtual space V or the controller used when playing a game. The user information acquisition unit 153 may acquire operation information by any method. In this embodiment, the user information acquisition unit 153 acquires the operations that user 5 inputs to the input unit 11 (controller, etc.) as operation information. Alternatively, for example, a sensor is provided to detect the position and orientation of user 5 in real space, and the user information acquisition unit 153 may acquire the position and orientation of user 5 detected by that sensor as operation information.

[0053] Furthermore, the facial expression information of user 5 referred to here is information about user 5's facial expression in the real world. User 5's facial expression information may be acquired by any method, and in this embodiment, the user information acquisition unit 153 acquires user 5's facial expression information (image including facial expression) using a camera (not shown) provided in the information terminal 10B.

[0054] Furthermore, the biometric information of user 5 referred to here may be any biometric information, such as user 5's brainwaves or pulse rate. The user information acquisition unit 153 according to this embodiment acquires user 5's brainwave information from a brainwave sensor (not shown) attached to user 5. The brainwave sensor is a sensor that detects brainwaves and may be an electroencephalograph (EEG), which uses electrodes to detect brainwaves that originate from the weak electric currents flowing through the brain's neural network. For example, the brainwave sensor may be implemented by placing a conductive gel between user 5's skin and the electrode to reduce the contact resistance between the electrode and the skin, or by providing a sponge between the skin and the electrode and soaking the sponge in an electrolyte solution. In this embodiment, the brainwave sensor is attached to user 5's head. However, the location of the brainwave sensor may be any location other than user 5's head.

[0055] (Information Processing Device) Figure 6 is a schematic block diagram of the information processing device according to this embodiment. As shown in Figure 6, the information processing device 30B includes an emotion estimation unit 337 in the control unit 33.

[0056] (Processing of the Information Processing Device) The processing contents of the information processing device 30B will be described below. However, detailed explanations of processing common to the first embodiment will be omitted. Note that the processing of the information processing device 30B described below may also be performed by the information terminal 10B that outputs images and sound to the user 5. That is, the information terminal 10B that outputs images and sound to the user 5 may function as an information processing device and perform the processing of the information processing device 30B described below.

[0057] (Information Acquisition Unit) The information acquisition unit 332 acquires user information from the user information acquisition unit 153 on the user 5's information terminal 10B. The acquired user information is used to estimate the emotions of the second user 5B, which will be described later.

[0058] (Emotion Estimation Unit) The emotion estimation unit 337 estimates the emotion of user 5. More specifically, the emotion estimation unit 337 according to this embodiment acquires user information from the user information acquisition unit 153 on the information terminal 10B of the second user 5B, and estimates the emotion of the second user 5B based on this user information. The method for estimating the emotion of the second user 5B can be any method; for example, the emotion estimation unit 337 according to this embodiment estimates the emotion based on the facial expression information of the second user 5B from the user information. Here, the method for estimating the emotion based on the facial expression information of the second user 5B can be any method; for example, the emotion estimation unit 337 according to this embodiment estimates the emotion using a facial expression recognition model that has been trained on an image (facial expression information) containing the user's face.

[0059] (Facial Expression Recognition Model) The facial expression recognition model is a model (program) created by analyzing the correspondence between input data and events using statistical analysis and machine learning. The input data here refers to parameters (variables) that affect the events. The event refers to an event that occurs in response to the input data. The event in this embodiment may be of any content, but it is preferably the user's emotion. The input data may be any parameters that affect the event, but the input data in this embodiment is preferably the facial expression information (image including facial expression) of the second user 5B acquired by the user information acquisition unit 153. The facial expression recognition model in this embodiment, when the facial expression information of the second user 5B is input, extracts the outline of the second user 5B's face and feature points for each part of the face that are captured in the image including the second user 5B's facial expression, tracks the movement of the feature points to recognize the user's facial expression, and outputs the emotion of the second user 5B associated with the recognized facial expression as an emotion estimated from the facial expression information.

[0060] In this embodiment, the emotion estimation unit 337 reads the facial expression recognition model stored in the memory unit 31 and inputs the facial expression information of the second user 5B that it has acquired into the facial expression recognition model, thereby estimating the emotion of the second user 5B.

[0061] Furthermore, the emotion estimation unit 337 according to this embodiment may estimate emotions based on user 5's operation information from the user information. The method for estimating emotions based on user 5's operation information can be any method, and the emotion estimation unit 337 according to this embodiment estimates emotions based on the behavioral characteristics (keystroke dynamics) of the second user 5B when inputting using a controller or keyboard, etc. For example, if the second user 5B is in a state of anger, and exhibits behavioral characteristics such as a decrease in the number of characters entered at once and an increase in the frequency of entering the Enter key, the emotion estimation unit 337 according to this embodiment may estimate whether the second user 5B is angry based on the number of characters entered at once and the frequency of entering the Enter key from the second user 5B's operation information.

[0062] Furthermore, the emotion estimation unit 337 may estimate the user 5's emotion as one of several levels in a predetermined index indicating the user 5's psychological state. Specifically, the emotion estimation unit 337 may estimate the user 5's emotional level using the aforementioned biometric information group or operation information. Alternatively, the emotion estimation unit 337 may simply determine positive emotions on a three-level scale, or negative emotions on a three-level scale.

[0063] Furthermore, the emotion estimation unit 337 according to this embodiment may estimate emotions based on the biometric information of user 5 from the user information. The method for estimating emotions based on the biometric information of user 5 can be any method, and the emotion estimation unit 337 according to this embodiment estimates emotions based on brain waves from the biometric information of second user 5B. It is generally known that there is a correlation between emotions and brain wave frequencies, and it is known that when a person is irritated or stressed, beta waves with frequencies between 14 and 23 Hz become dominant, and conversely, when a person is relaxed, alpha waves with frequencies between 9 and 11 Hz become dominant. For example, the emotion estimation unit 337 according to this embodiment may estimate the emotions of second user 5B based on the dominant frequencies of the acquired brain waves.

[0064] (Threshold Setting Unit) The threshold setting unit 334 according to this embodiment sets the intimacy threshold based on the emotion estimation result of the emotion estimation unit 337. For example, if user 5 has feelings of anger or sadness because they lost a game, it is expected that they will be more likely to utter abusive words towards the other party. Therefore, the threshold setting unit 334 according to this embodiment sets the intimacy threshold based on the emotions of the second user 5B. Specifically, during the period from when the outcome of the game is decided and the first game ends until the next game begins, when user 5 is not performing any game operations (there is no object to operate), the threshold setting unit 334 sets the intimacy threshold to a higher value than when it is estimated that second user 5B does not have feelings of anger or sadness, if the emotion estimation unit 337 estimates that second user 5B has feelings of anger or sadness. This increases the likelihood that the information processing device 30B will correct the voice of the second user 5B, thereby preventing the first user 5A from feeling disgusted by the abusive language of the second user 5B.

[0065] Furthermore, the threshold setting unit 334 may set the intimacy threshold based on the emotions of the first user 5A. For example, if user 5 is angry or sad because they lost a game, it is expected that they will be more likely to perceive the other person's remarks as abusive. Therefore, if the emotion estimation unit 337 estimates that the first user 5A is angry or sad, the threshold setting unit 334 may set the intimacy threshold to a higher value compared to when it is estimated that the first user 5A is not angry or sad. This increases the possibility that the information processing device 30B will correct the voice of the second user 5B, thereby suppressing the first user 5A from perceiving the voice data of the second user 5B as abusive and feeling disgusted. The intimacy threshold may also be set according to the level of emotion. For example, if the emotion of the second user 5B is level "2" on a three-level scale for anger, the intimacy threshold is set higher than the default value (the value when the anger level is "1"). If the anger level is "3", the threshold is set higher than when the anger level is "2". For example, the threshold setting unit 334 may set the intimacy threshold by referring to a database that stores the emotional levels of the first user 5A and the intimacy thresholds linked to them. Specifically, the threshold setting unit 334 refers to the database stored in the storage unit 31 and sets the intimacy threshold linked to the emotional level of the first user 5A estimated by the emotion estimation unit 337.

[0066] Furthermore, the threshold setting unit 334 may set the intimacy threshold based on both the intimacy level and the emotions of the second user 5B. For example, the threshold setting unit 334 may set the intimacy threshold by referring to a database that stores the intimacy level, the emotional level of the second user 5B, and the intimacy threshold in association with each other. Specifically, the threshold setting unit 334 refers to the database stored in the storage unit 31 and sets an intimacy threshold linked to the intimacy level calculated by the intimacy calculation unit 333 and the emotional level of the second user 5B estimated by the emotion estimation unit 337.

[0067] (Processing Flow) Next, the processing flow according to this embodiment will be described. Figure 7 is a flowchart illustrating the processing flow of the information processing device according to the first embodiment. As shown in Figure 7, the information acquisition unit 332 acquires relationship information acquired by the relationship information acquisition unit 151 at the information terminal 10B of the first user 5A and user information acquired by the user information acquisition unit 153 at the information terminal 10B of the second user 5B (step S30). The intimacy calculation unit 333 calculates the intimacy of the first user 5A towards the second user 5B based on the relationship information (step S31). The emotion estimation unit 337 estimates the emotion of the second user 5B based on the user information (step S32). The threshold setting unit 334 sets an intimacy threshold based on the emotion estimation result of the second user 5B (step S33). Specifically, if it is estimated that the second user 5B is angry, the system determines that the second user 5B is dissatisfied with the game's outcome and, for example, refers to a database that stores the calculated intimacy level, the estimated emotion level, and the intimacy threshold, and sets an intimacy threshold according to the relationship between the intimacy level and the estimated emotion level. If the calculated intimacy level is greater than or equal to the set intimacy threshold (step S34; YES), the correction content determination unit 335 does not correct the second user 5B's voice data, and the output control unit 336 outputs the second user 5B's voice data as is to the information terminal 10B (step S35). If the calculated intimacy level is less than the set intimacy threshold (step S34; No), the correction content determination unit 335 corrects the second user 5B's voice data, and the output control unit 336 outputs the second user 5B's voice data corrected by the correction content determination unit 335 to the information terminal 10B (step S36), and the process ends.

[0068] As described above, the information processing device 30B according to this embodiment calculates the level of intimacy between the first user 5A and the second user 5B based on relationship information. In addition, the emotion estimation unit 337 estimates the emotion of the second user 5B based on user information. The information processing device 30B then sets an intimacy threshold based on the emotion estimation result of the second user 5B. If the calculated level of intimacy between the first user 5A and the second user 5B is equal to or greater than the set intimacy threshold, the information processing device 30B outputs the unaltered voice data of the second user 5B to the information terminal 10B. If the calculated level of intimacy between the first user 5A and the second user 5B is less than the set intimacy threshold, the information processing device 30B outputs the corrected voice data of the second user 5B to the information terminal 10B. According to this disclosure, the information processing device 30B can appropriately correct the audio data of statements made by the second user 5B, which is output to the information terminal 10B of the first user 5A, based on a comparison result between the first user 5A's level of intimacy with the second user 5B and an intimacy threshold set based on the emotion estimation result of the second user 5B.

[0069] In addition, in this disclosure, the threshold setting unit 334 sets the intimacy threshold based on the emotion estimation result of the emotion estimation unit 337. As a result, the threshold setting unit 334 sets the intimacy threshold to a high value when the second user 5B is experiencing anger or sadness and is likely to utter abusive language. Therefore, even if the second user 5B utters abusive language, the information processing device 30B can appropriately correct the audio data of the second user 5B's statements output to the information terminal 10B of the first user 5A.

[0070] (Third Embodiment) Next, a third embodiment will be described. In the second embodiment, the threshold setting unit 334 set the intimacy threshold based on the emotion estimation result of the second user 5B, but in this embodiment, the threshold setting unit 334 sets the intimacy threshold based on the behavior of the avatar Ub corresponding to the second user 5B. In addition, the information processing device 30C in the third embodiment differs from the second embodiment in that the control unit 33 further includes a motion detection unit 338. Note that other configurations are the same as in the second embodiment, so their description will be omitted.

[0071] (Information Processing Device) Figure 8 is a schematic block diagram of the information processing device according to this embodiment. As shown in Figure 8, the information processing device 30C includes an operation detection unit 338 in the control unit 33.

[0072] (Processing of the Information Processing Device) The processing content of the information processing device 30C will be described below. However, detailed explanations of processing common to the second embodiment will be omitted. Note that the processing of the information processing device 30C described below may also be performed by the information terminal 10B that outputs images and sound to the user 5. That is, the information terminal 10B that outputs images and sound to the user 5 may act as an information processing device and perform the processing of the information processing device 30C described below.

[0073] (Motion Detection Unit) The motion detection unit 338 detects the actions of avatar U corresponding to user 5. More specifically, the motion detection unit 338 according to this embodiment determines whether avatar Ub corresponding to second user 5B has performed a predetermined action. The method for determining whether avatar Ub has performed a predetermined action is arbitrary, and the motion detection unit 338 according to this embodiment determines whether avatar Ub has performed a predetermined action using a motion detection model that has been trained on an image of avatar U.

[0074] (Action Detection Model) The action detection model is a model (program) created by analyzing the correspondence between input data and events using statistical analysis and machine learning. The input data here refers to parameters (variables) that affect the events. The event refers to an event that occurs in response to the input data. The event in this embodiment may be of any content, but it is preferably a predetermined action. The input data may be any parameters that affect the event, but the input data in this embodiment is preferably image information of avatar Ub corresponding to the second user 5B, created by the spatial generation unit 331. The action detection model in this embodiment is a model (program) that, when image information of avatar Ub corresponding to the second user 5B is input, extracts the outline of avatar Ub's body and feature points for each body part that are visible in the image of avatar Ub, and tracks the movement of the feature points to output the action being performed by avatar Ub as an estimated action being performed.

[0075] In this embodiment, the motion detection unit 338 reads the motion detection model stored in the storage unit 31 and inputs the acquired image information of the avatar Ub corresponding to the second user 5B into the motion detection model to determine whether or not the avatar Ub has performed a predetermined action.

[0076] The motion detection unit 338 may also determine whether or not the avatar Ub has performed a predetermined action based on the operation information of the second user 5B acquired by the user information acquisition unit 153 in the information terminal 10B.

[0077] Here, the predetermined actions of avatar Ub can be set arbitrarily, but in this embodiment, for example, when first user 5A and second user 5B are playing a game together in virtual space V, it is preferable that the predetermined actions are those that second user 5B considers to be taboo in the game, or so-called NG actions that first user 5A finds unpleasant. Figure 9 is a schematic diagram showing an example of an NG action. In Figure 9, an example of an NG action is shown in which avatar Ub, corresponding to second user 5B, persistently attacks avatar Ua, corresponding to first user 5A, who has lost the game and is unable to move.

[0078] In this embodiment, the threshold setting unit 334 sets an intimacy threshold based on the actions of the avatar Ub corresponding to the second user 5B. More specifically, in a situation where the first user 5A and the second user 5B are playing a game together in the virtual space V by operating their respective avatars U, the threshold setting unit 334 in this embodiment sets the intimacy threshold to a higher value when the avatar Ub corresponding to the second user 5B performs a predetermined action compared to when the predetermined action is not performed. Specifically, when the action detection unit 338 determines that the avatar Ub has performed a predetermined NG action, the threshold setting unit 334 sets an intimacy threshold to a higher value compared to when the avatar Ub does not perform the NG action.

[0079] (Processing Flow) Next, the processing flow according to this embodiment will be described. Figure 10 is a flowchart illustrating the processing flow of the information processing device according to the third embodiment. As shown in Figure 10, the information acquisition unit 332 acquires relationship information acquired by the relationship information acquisition unit 151 at the information terminal 10B of the first user 5A (step S50). The intimacy calculation unit 333 calculates the intimacy of the first user 5A towards the second user 5B based on the relationship information (step S51). The action detection unit 338 determines the behavior of the avatar Ub based on the image information of the avatar Ub corresponding to the second user 5B (step S52). The threshold setting unit 334 sets an intimacy threshold based on the determined behavior of the avatar Ub (step S53). Specifically, if it is determined that the avatar Ub corresponding to the second user 5B is performing an NG action, it is determined that the second user 5B is dissatisfied with the first user 5A, and an intimacy threshold is set that is higher than when it is determined that the avatar Ub is not performing an NG action. If the calculated intimacy level is greater than or equal to the set intimacy threshold (step S54; YES), the correction content determination unit 335 does not correct the voice data of the second user 5B, and the output control unit 336 outputs the voice data of the second user 5B as is to the information terminal 10B (step S55). If the calculated intimacy level is less than the set intimacy threshold (step S54; No), the correction content determination unit 335 corrects the voice data of the second user 5B, and the output control unit 336 outputs the voice data of the second user 5B corrected by the correction content determination unit 335 to the information terminal 10B (step S56), and this process ends.

[0080] For example, in a virtual space V, when a first user 5A and a second user 5B are playing a game together, if the second user 5B performs a so-called "NG behavior," there is a high probability that the second user 5B will also make remarks that the first user 5A perceives as abusive. Therefore, by setting the intimacy threshold of the avatar Ub corresponding to the second user 5B to a higher value when the avatar performs an NG behavior in the game compared to when it does not, the information processing device 30A is more likely to correct the voice of the second user 5B. This suppresses the first user 5A from perceiving the voice data of the second user 5B as abusive and feeling disgusted.

[0081] (Effects) As described above, the information processing device 30A of this disclosure sets an intimacy threshold based on the actions of the avatar Ub corresponding to the second user 5B. As a result, the threshold setting unit 334 sets the intimacy threshold to a high value when the avatar Ub corresponding to the second user 5B performs a predetermined action (an action that is considered taboo when a user plays a game in the virtual space V, or a so-called NG action that the person on the receiving end finds unpleasant). Therefore, even if the avatar Ub corresponding to the second user 5B performs an NG action and the second user 5B also utters abusive language, the information processing device 30A can appropriately correct the audio data of the second user 5B's statements output to the information terminal 10A of the first user 5A.

[0082] (Effects of the Disclosure) As described above, the information processing device 30A of the Disclosure is an information processing device that displays a virtual space V and an avatar U on an information terminal 10A, and includes an information acquisition unit 332 that acquires relationship information, which is information relating to the relationship between avatar Us corresponding to a first user 5A and a second user 5B, and an output control unit 336 that outputs information relating to the correction content of the voice data of the second user 5B to the information terminal 10A of the first user 5A based on the relationship information. According to the Disclosure, the information processing device 30A can appropriately correct the voice data of speech by the second user 5B that is output to the information terminal 10A of the first user 5A based on the relationship information.

[0083] Furthermore, according to this disclosure, the information processing device 30B further comprises an emotion estimation unit 337 that estimates the emotions of the second user 5B, and the output control unit 336 outputs information regarding the correction content of the second user 5B's voice data to the first user 5A's information terminal 10B based on the estimation results of the emotion estimation unit 337. According to this disclosure, the information processing device 30B can appropriately correct the voice data of statements made by the second user 5B that is output to the first user 5A's information terminal 10B, based on a comparison result between the first user 5A's level of intimacy with the second user 5B and an intimacy threshold set based on the emotion estimation result of the second user 5B.

[0084] Furthermore, according to this disclosure, the output control unit 336 outputs information regarding the correction of the second user 5B's voice data based on the actions of the avatar Ub corresponding to the second user 5B to the information terminal 10A of the first user 5A. According to this disclosure, the threshold setting unit 334 sets the intimacy threshold to a high value when the avatar Ub corresponding to the second user 5B performs a predetermined action (an action that is considered taboo when a user plays a game in the virtual space V, or a so-called NG action that the person on the receiving end finds unpleasant). Therefore, even if the avatar Ub corresponding to the second user 5B performs an NG action and the second user 5B utters abusive language, the information processing device 30A can appropriately correct the voice data of the second user 5B's statements output to the information terminal 10A of the first user 5A.

[0085] Furthermore, according to this disclosure, the system further comprises: a closeness calculation unit 333 that calculates the closeness of the first user 5A to the second user 5B based on relationship information; a threshold setting unit 334 that sets a closeness threshold, which is a threshold for determining whether or not to perform correction on the voice data of the second user 5B; and a correction content determination unit 335 that determines the correction content of the voice data of the second user 5B based on the closeness and the closeness threshold. The correction content determination unit 335 performs correction on the voice data of the second user 5B when the closeness is less than the closeness threshold, and does not perform correction on the voice data of the second user 5B when the closeness is equal to or greater than the closeness threshold. According to this disclosure, the correction content determination unit 335 can appropriately determine the correction content of the voice data of the second user 5B output to the information terminal 10A of the first user 5A based on the closeness of the first user 5A to the second user 5B.

[0086] Furthermore, according to this disclosure, the correction content determination unit 335 changes the degree to which it corrects the voice data of the second user 5B output to the information terminal 10A of the first user 5A, based on the level of familiarity and the level of familiarity threshold. According to this disclosure, the correction content determination unit 335 can appropriately determine the correction content (degree of correction) for the voice data of the second user 5B output to the information terminal 10A of the first user 5A, based on the level of familiarity and the level of familiarity threshold.

[0087] Furthermore, according to this disclosure, the correction content determination unit 335 corrects only predetermined words from the voice data of the second user 5B. According to this disclosure, the correction content determination unit 335 corrects only predetermined words (for example, words that the first user 5A finds extremely offensive) from the voice data of the second user 5B output to the information terminal 10A of the first user 5A, thereby preventing the first user 5A from feeling offensive and suppressing the correction of voice output by the information processing device 30A more than necessary.

[0088] Furthermore, according to this disclosure, the threshold setting unit 334 sets the intimacy threshold based on the emotion estimation result of the emotion estimation unit 337. According to this disclosure, since the threshold setting unit 334 can appropriately set the intimacy threshold based on the emotion estimation result of the emotion estimation unit 337, the correction content determination unit 335 can also appropriately determine whether or not to correct the voice of the second user 5B. This makes it possible to suppress the first user 5A from feeling aversion due to the abusive language of the second user 5B.

[0089] The information processing method relating to this disclosure is a processing method for an information processing device 30A that displays a virtual space V and an avatar U on an information terminal 10A, and comprises the steps of: acquiring relationship information, which is information relating to the relationship between avatar Us corresponding to a first user 5A and a second user 5B; and outputting information relating to the correction content of the second user 5B's voice data to the first user 5A's information terminal 10A based on the relationship information. According to this disclosure, the voice data of speech by the second user 5B output to the first user 5A's information terminal 10A can be appropriately corrected based on the relationship information.

[0090] The program relating to this disclosure is a program executed by an information processing device 30A that displays a virtual space V and an avatar U on an information terminal 10A, and causes a computer to perform the following steps: acquiring relationship information, which is information relating to the relationship between avatar Us corresponding to a first user 5A and a second user 5B; and outputting information regarding the correction content of the second user 5B's voice data to the first user 5A's information terminal 10A based on the relationship information. According to this disclosure, the voice data of speech by the second user 5B output to the first user 5A's information terminal 10A can be appropriately corrected based on the relationship information.

[0091] The information processing device, information processing method, and program of this embodiment can be used, for example, in a communication tool.

[0092] 10A Information terminal 30A Information processing device 5 User U Avatar V Virtual space 151 Relationship information acquisition unit 152 Display control unit 153 User information acquisition unit 331 Space generation unit 332 Information acquisition unit 333 Intimacy calculation unit 334 Threshold setting unit 335 Correction content determination unit 336 Output control unit 337 Emotion estimation unit 338 Action detection unit

Claims

1. An information processing device for displaying a virtual space and avatars on an information terminal, comprising: an information acquisition unit that acquires relationship information which is information relating to the relationship between avatars corresponding to a first user and a second user; and an output control unit that outputs information relating to the correction content of the second user's voice data to the first user's information terminal based on the relationship information.

2. The information processing apparatus according to claim 1, further comprising an emotion estimation unit for estimating the emotions of a second user, wherein the output control unit outputs information regarding the correction content of the second user's voice data to the first user's information terminal based on the estimation result of the emotion estimation unit.

3. The information processing apparatus according to claim 1 or 2, wherein the output control unit outputs information regarding the correction content of the second user's voice data to the first user's information terminal based on the actions of the avatar corresponding to the second user.

4. An information processing device according to claim 1, further comprising: a closeness calculation unit that calculates the closeness of the first user to the second user based on the relationship information; a threshold setting unit that sets a closeness threshold which is a threshold for determining whether or not to perform correction on the voice data of the second user; and a correction content determination unit that determines the content of correction on the voice data of the second user based on the closeness and the closeness threshold, wherein the correction content determination unit performs correction on the voice data of the second user when the closeness is less than the closeness threshold, and does not perform correction on the voice data of the second user when the closeness is equal to or greater than the closeness threshold.

5. The information processing apparatus according to claim 4, wherein the correction content determination unit changes the degree to which it corrects the voice data of the second user output to the information terminal of the first user, based on the intimacy level and the intimacy level threshold.

6. The information processing apparatus according to claim 4, wherein the correction content determination unit corrects only predetermined words from the second user's voice data.

7. The information processing apparatus according to claim 2, further comprising a threshold setting unit for setting an intimacy threshold, which is a threshold for determining whether or not to perform correction on the voice data of the second user, wherein the threshold setting unit sets the intimacy threshold based on the emotion estimation result of the emotion estimation unit.

8. A processing method for an information processing device executed by an information processing system that displays a virtual space and avatars on an information terminal, comprising: acquiring relationship information which is information relating to the relationship between avatars corresponding to a first user and a second user; and outputting information relating to the correction content of the second user's voice data to the first user's information terminal based on the relationship information.

9. A program executed by an information processing device that displays a virtual space and avatars on an information terminal, comprising: a step of acquiring relationship information which is information relating to the relationship between avatars corresponding to a first user and a second user; and a step of outputting information relating to the correction content of the second user's voice data to the first user's information terminal based on the relationship information.

Images