Information processing device and information processing method

Abstract

An information processing device 1 according to an embodiment of the present invention enables communication between users in a virtual space by means of communication with other terminals connected via a network, said information processing device comprising: a personal distance measurement unit 11 that determines whether an operator and another user are moving together in the virtual space; a speech measurement unit 13 that acquires conversational speech for a predetermined time between the operator and a user determined to be moving together with the operator; a movement speed identification unit 12 that acquires a parameter of the user's movement speed in the virtual space; a movement speed adjustment unit 14 that adjusts the value of the user's movement speed on the basis of the acquired volume of the conversational speech for the predetermined time; and a movement speed updating unit 15 that updates the parameter of the user's movement speed.

Description

Information processing device and information processing method

[0001] One aspect of the present invention relates to an information processing device and an information processing method.

[0002] CVE (Collaborative Virtual Environment) is a communication environment that combines the real and cyber worlds. In CVE, multiple users can gather in the same virtual space using AR (Augmented Reality) glasses or HMD (Head-mounted display) to work and interact.

[0003] The key to realizing CVE is to share the location information and actions of users in the virtual space. Methods for acquiring the location information of users in the virtual space include technologies that utilize the Global Positioning System (GPS) and the Visual Positioning System (VPS). In addition, methods for reflecting user actions often involve estimating and reflecting user actions based on information from sensors.

[0004] “What is a Collaborative Virtual Environment?”, [online], [Retrieved August 26, 2024], Internet, <URL:https: / / www.xrtoday.com / virtual-reality / what-is-a-collaborative-virtual-environment / >

[0005] When multiple users move together in the metaverse, such as when users converse with each other while moving on a CVE, the users' movement speeds may differ between the metaverse space and the real space. For example, if a user in the metaverse space (another user using the metaverse space) moves quickly from the perspective of a user in the real space (the viewpoint or position coordinates in the metaverse space of a given user using the metaverse space) and the user in the real space cannot keep up with the user in the metaverse space, the user in the metaverse space will need to finely adjust their movement speed to match the user in the real space, which is cumbersome and causes problems in communication.

[0006] To address this issue, there is a method for controlling the user's movement speed using the above-mentioned location information. However, if the user's movement speed is controlled by measuring the user's location information every five seconds, the user's position in the metaverse space as seen from the user's perspective will be forcibly corrected every five seconds. This can cause users to feel uncomfortable with each other's movements, hindering communication. Another possible method is to increase the frequency of measuring location information and finely correct the user's position, but this method increases the device's communication and processing, resulting in increased battery consumption.

[0007] The present invention has been made in light of the above circumstances, and aims to provide an information processing device and an information processing method that are capable of adjusting the movement speed of a user on the metaverse without impeding communication.

[0008] An information processing device according to one embodiment of the present invention is an information processing device that enables communication between users in a virtual space by communicating with other terminals connected via a network, and includes a person distance measurement unit that determines whether an operator and another user are moving together in the virtual space, an audio measurement unit that acquires conversational audio between the operator and a user that is determined to be moving with the operator for a predetermined period of time, a movement speed confirmation unit that acquires parameters of the user's movement speed in the virtual space, a movement speed adjustment unit that adjusts the value of the user's movement speed based on the volume of the conversational audio acquired for the predetermined period of time, and a movement speed update unit that updates the parameters of the user's movement speed.

[0009] According to one aspect of the present invention, it is possible to provide an information processing device and an information processing method that are capable of adjusting the movement speed of a user on the metaverse without impeding communication.

[0010] Fig. 1 is a diagram schematically showing an example of the configuration of an information processing device according to an embodiment. Fig. 2 is a flowchart showing an example of processing executed by the information processing device according to an embodiment. Fig. 3 is a graph for explaining an example of a movement speed adjustment process executed by the information processing device according to an embodiment. Fig. 4 is a diagram for explaining the effects achieved by the information processing device according to an embodiment. Fig. 5 is a diagram for explaining the effects achieved by the information processing device according to an embodiment.

[0011] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS An embodiment of the present invention will now be described with reference to the accompanying drawings. Fig. 1 is a diagram illustrating an example of the configuration of an information processing apparatus according to an embodiment of the present invention.

[0012] The information processing device 1 is a computer capable of constructing a metaverse space that enables communication between users in a virtual space by communicating with other terminals connected via a network such as the Internet. The information processing device 1 is connected, for example, via the network to AR glasses, an HMD, or the like used by an operator of the information processing device 1, and outputs an image of the metaverse space to the operator of the information processing device 1. Alternatively, the information processing device 1 may be, for example, a wearable device itself, such as AR glasses or an HMD. The information processing device 1 may be configured to be able to communicate with other information processing devices 1 via the network, or may be configured to enable communication between users in a virtual space shared with other information processing devices 1 by a server device connected via the network.

[0013] The information processing device 1 includes a control unit 10, a storage unit 20, an input unit 30, an output unit 40, and a communication unit 50. In the following description, a user in the real space refers to the operator of the information processing device 1. A user in the metaverse space refers to another user in the virtual space as seen from the operator of the information processing device 1.

[0014] The input unit 30 acquires various pieces of information supplied from outside the information processing device 1. The input unit 30 is operated by an operator of the information processing device 1 and may include, for example, a user interface such as a mouse or keyboard for inputting various pieces of information (for example, information for changing the numerical values ​​of various settings of the information processing device 1, information indicating the operation of the operator (operation information), etc.), various sensors such as a microphone or a touch panel, etc.

[0015] The output unit 40 outputs, for example, visual or auditory information to be presented to the operator of the information processing device 1. The output unit 40 may include, for example, a monitor or the like that visually supplies information to the operator of the information processing device 1. The output unit 40 may also include an audio output means, such as a speaker, that auditorily supplies information to the operator of the information processing device 1.

[0016] The communication unit 50 can transmit information received from outside to components within the information processing device 1, and can also transmit information received from components within the information processing device 1 to the outside. For example, the communication unit 50 acquires various information used in the processing described below from a server device connected via a network such as the Internet. The communication unit 50 is also configured to be able to communicate with wearable devices such as AR glasses (not shown) and a VR terminal equipped with an HMD (head-mounted display) used by the operator of the information processing device 1, for example, via a network such as the Internet.

[0017] The storage unit 20 includes, for example, a main storage unit and an auxiliary storage unit. The main storage unit may include, for example, a read-only memory (ROM) and a random-access memory (RAM). The ROM is a non-volatile memory used exclusively for reading data, and can store data and various setting values ​​used by the control unit 10 when performing various processes. The RAM may be used as a so-called work area for temporarily storing data when the control unit 10 performs various processes. The main storage unit in the embodiment is, for example, a RAM, and is used as a memory.

[0018] The auxiliary storage unit is a non-transitory computer-readable storage medium of a computer centered around the control unit 10. The auxiliary storage unit is, for example, an EEPROM (registered trademark) (electric erasable programmable read-only memory), a hard disk drive (HDD), or a solid state drive (SSD). The auxiliary storage unit can store data used by the control unit 10 when performing various processes, data generated by the processes in the control unit 10, various setting values, etc.

[0019] In the embodiment, the memory unit 20 stores a real space position information memory unit 21, a metaverse space position information memory unit 22, a real space audio data memory unit 23, a metaverse space audio data memory unit 24, and a metaverse space parameter memory unit 25.

[0020] The real space position information storage unit 21 stores position information of the user in real space. That is, the real space position information storage unit 21 stores position information of the operator of the information processing device 1. The operator's position information is, for example, GPS-based position information acquired via the communication unit 50, or VPS-based position information identified from a video of the operator's surroundings acquired via the input unit 30, etc. The operator's position information is measured at predetermined intervals (for example, every 5 seconds) and stored in the real space position information storage unit 21. Note that the operator's position information may also be position information in the metaverse space input by the operator operating a motion sensor such as a controller or stick, for example.

[0021] The metaverse space position information storage unit 22 stores position information of users in the metaverse space. The position information of users in the metaverse space is, for example, coordinate information in the virtual space of other users who have accessed the metaverse space constructed or accessed by the information processing device 1. The position information of users in the metaverse space is acquired from other user terminals connected via the communication unit 50 and stored in an identifiable manner for each user.

[0022] The real-space audio data storage unit 23 stores audio data of a user in real space. That is, the real-space audio data storage unit 23 stores audio data of the operator of the information processing device 1. The audio data of the user in real space is acquired and stored, for example, via a microphone or the like of the input unit 30.

[0023] The metaverse space voice data storage unit 24 stores voice data of users of the metaverse space. The voice data of users of the metaverse space is acquired, for example, from other user terminals connected via the communication unit 50, and is stored in an identifiable manner for each user.

[0024] The metaverse space parameter storage unit 25 stores various parameters of users in the metaverse space. In an embodiment, the metaverse space parameter storage unit 25 stores parameters of the movement speed of any user in the virtual space in a manner that allows the user to identify the parameter. The metaverse space parameter storage unit 25 also stores a default movement speed of the user in the metaverse space that is set in advance.

[0025] The control unit 10 is typically a central processing unit (CPU) and / or a graphics processing unit (GPU), but may also include a processor such as a microcomputer, a field programmable gate array (FPGA), or a digital signal processor (DSP). The control unit 10 can implement various functions of the information processing device 1 by executing programs such as system software, application software, or firmware stored in the storage unit 20.

[0026] The control unit 10 includes a person distance measurement unit 11 , a movement speed confirmation unit 12 , a voice measurement unit 13 , a movement speed adjustment unit 14 , and a movement speed update unit 15 .

[0027] The person distance measurement unit 11 determines whether the operator and another user are moving together in the metaverse space (for example, side by side and moving (or about to move) in the same direction). The person distance measurement unit 11 determines whether the user in the real space and the user in the metaverse space are moving together, based on the position information of the operator stored in the real space position information storage unit 21 and the position information of the user in the metaverse space stored in the metaverse space position information storage unit 22.

[0028] The movement speed confirmation unit 12 determines whether the movement speed of another user determined by the person distance measurement unit 11 to be moving together with the operator is stored in the metaverse space parameter storage unit 25. Based on the result of the determination, the movement speed confirmation unit 12 reads out the movement speed of the other user in the metaverse space.

[0029] The voice measurement unit 13 acquires a conversation voice between the operator and another user determined to be moving with the operator for a predetermined time. The voice measurement unit 13 acquires the conversation voice of the operator via, for example, the input unit 30 and stores it in the real space voice data storage unit 23. The voice measurement unit 13 also acquires the conversation voice of another user in the metaverse space via, for example, the communication unit 50 and stores it in the metaverse space voice data storage unit 24. The voice measurement unit 13 acquires the conversation voice between the operator and another user in the metaverse space for a predetermined time.

[0030] The movement speed adjustment unit 14 adjusts the value of the movement speed of other users in the metaverse space based on the change in volume of the acquired conversational voice over a predetermined period of time. The movement speed adjustment unit 14 adjusts the movement speed of the user in the metaverse space based on the conversational voice of the operator stored in the real space audio data storage unit 23 and the conversational voice of other users in the metaverse space stored in the metaverse space audio data storage unit 24. Detailed processing by the movement speed adjustment unit 14 will be described later.

[0031] The movement speed update unit 15 updates the parameters of the movement speeds of other users. For other users in the metaverse space who are determined by the person distance measurement unit 11 to be moving together with the operator, the movement speed update unit 15 updates the parameters of the movement speeds of the users stored in the metaverse space parameter storage unit 25 to the values ​​of the movement speeds adjusted by the movement speed adjustment unit 14.

[0032] Next, an example of a process for adjusting the movement speed of another user in the metaverse space executed by the information processing device 1 according to the embodiment will be described. Fig. 2 is a flowchart showing an example of a process executed by the information processing device according to the embodiment.

[0033] The control unit 10 of the information processing device 1 starts the following process, for example, when another user exists in the metaverse space accessed by the operator of the information processing device 1.

[0034] The person distance measurement unit 11 determines whether the operator and another user are moving together in the metaverse space (step S101). First, the person distance measurement unit 11 calculates coordinate information of the operator in the metaverse space based on the position information of the operator stored in the real space position information storage unit 21. Then, the person distance measurement unit 11 calculates the distance in the virtual space between the operator and another user based on the calculated coordinate information of the operator and the coordinate information of the other user stored in the metaverse space position information storage unit 22.

[0035] When the operator and the other user are moving, the distance between the operator and the other user in the virtual space changes based on updates to location information, etc. Therefore, the person distance measurement unit 11 repeatedly calculates the distance and determines whether the calculated distance value is equal to or less than a predetermined threshold for a predetermined time. If the calculated distance value is not equal to or less than the predetermined threshold for a predetermined time, the person distance measurement unit 11 determines that the operator and the other user are not moving together in the metaverse space (step S101: NO) and repeats the processing of step S101. That is, the person distance measurement unit 11 repeats the processing of step S101 until it determines that the operator and the other user are moving together in the metaverse space.

[0036] If the person distance measurement unit 11 determines that the operator and another user are moving together in the metaverse space (step S101: YES), the voice measurement unit 13 determines whether or not a conversation between the operator and a user in the metaverse space has been acquired (step S102). The voice measurement unit 13 acquires the operator's conversational voice, for example, via the input unit 30, and acquires the conversational voice of the other user via the communication unit 50. The voice measurement unit 13 determines, for example, whether a state in which either the operator or the other user, or both, are emitting conversational voices has continued for a predetermined time (e.g., several seconds). If the voice measurement unit 13 cannot acquire conversational voices for the predetermined time, it determines that the operator and the other user are not conversing (step S102: NO) and returns to step S101.

[0037] On the other hand, if the conversational voice has been acquired for a predetermined time, the voice measurement unit 13 determines that the operator and the other user are having a conversation (step S102: YES), and stores the acquired conversational voice in the storage unit 20. If the voice measurement unit 13 determines that the conversation has been acquired, it stores the acquired conversational voices of the operator and the other user in the real space voice data storage unit 23 and the metaverse space voice data storage unit 24, respectively.

[0038] The movement speed confirmation unit 12 determines whether the operator and another user have moved together in the metaverse space in the past (step S103). The movement speed confirmation unit 12 determines whether the movement speed of the other user determined to be moving with the operator is stored in the metaverse space parameter storage unit 25. If it determines that the movement speed is stored (step S103: YES), it reads the stored past movement speed of the other user (step S104). On the other hand, if it determines that the movement speed of the other user is not stored (step S103: NO), it reads the default movement speed stored in the metaverse space parameter storage unit 25 as the movement speed of the other user (step S105).

[0039] After executing the process of step S104 or step S105, the movement speed adjustment unit 14 adjusts the value of the movement speed of other users in the metaverse space based on the volume of the acquired conversational voice (step S106).

[0040] 3 is a graph for explaining an example of a movement speed adjustment process executed by the information processing device according to the embodiment. The y-axis (vertical axis) of the graph in FIG. 3 represents volume, and the x-axis (horizontal axis) of the graph in FIG. 3 represents time.

[0041] In this embodiment, it is assumed that the voice measurement unit 13 acquires a conversation between the operator and another user for T seconds in step S105. In Fig. 3, the solid line CV in the graph indicates the relationship between the volume of the acquired conversation voice and time (hereinafter referred to as conversation voice CV).

[0042] The moving speed adjustment unit 14 checks the volume of the acquired conversational voice at predetermined intervals and performs linear regression analysis. In the embodiment, the moving speed adjustment unit 14 extracts volume values ​​at intervals of N seconds from the acquired conversational voice CV. Here, the acquired volume values ​​for every N seconds are expressed as y Ｎ , y ２Ｎ , ..., y ａＮ (a is a natural number and aN<T).

[0043] The moving speed adjusting unit 14 performs a linear regression analysis based on the acquired volume value every N seconds to obtain a regression equation y=Ax+B that indicates the relationship between volume y and time x. The moving speed adjusting unit 14 also uses a threshold value y stored in the storage unit 20 to determine whether the volume is large or small. ｍｉｎ and y ｍａｘ Read out.

[0044] The moving speed adjusting unit 14 calculates the regression equation y=Ax+B and the read y ｍｉｎ , y ｍａｘ Based on this, the moving speed of other users in the metaverse space is adjusted. ｍｉｎ ≦B≦y ｍａｘ And y ｍｉｎ ≦AT+B≦y ｍａｘIf so, the movement speed adjustment unit 14 determines that the volume of the conversation for T seconds did not change beyond the appropriate range (i.e., the distance between the operator and the other user is appropriate), and does not adjust the movement speed of the other user in the metaverse space. The movement speed adjustment unit 14 determines the movement speed read by the movement speed confirmation unit 12 as the movement speed of the other user.

[0045] On the other hand, AT+B<y ｍｉｎ If so, the movement speed adjustment unit 14 determines that the volume of the conversation voice is lower than the appropriate range (i.e., the distance between the operator and the other user is too far or has suddenly increased), and increases the movement speed of the other user in the metaverse space. For example, the movement speed adjustment unit 14 increases the value of the movement speed read by the movement speed confirmation unit 12 by U % and determines it as the movement speed of the other user.

[0046] Also, AT+B>y ｍａｘ If so, the movement speed adjustment unit 14 determines that the volume of the conversational voice is greater than the appropriate range (i.e., the distance between the operator and the other user is too close or has suddenly become closer), and slows down the movement speed of the other user in the metaverse space. For example, the movement speed adjustment unit 14 determines the value of the movement speed read by the movement speed confirmation unit 12 to be reduced by D % as the movement speed of the other user.

[0047] The movement speed update unit 15 updates the parameter of the movement speed of the user in the metaverse space (step S107). The movement speed update unit 15 updates the parameter of the movement speed of the other user stored in the metaverse space parameter storage unit 25 to the value of the movement speed determined by the movement speed adjustment unit 14. If the movement speed of the other user is not stored in the metaverse space parameter storage unit 25, the movement speed update unit 15 stores the value of the movement speed of the other user determined by the movement speed adjustment unit 14 in the metaverse space parameter storage unit 25 as the parameter of the movement speed of the other user.

[0048] The movement speed update unit 15 determines whether to continue the movement speed adjustment process (step S108). The movement speed update unit 15 checks whether other users exist in the metaverse space accessed by the operator. If other users exist in the metaverse space, the movement speed update unit 15 determines to continue the movement speed adjustment process (step S108: YES) and returns to step S101. On the other hand, if other users do not exist in the metaverse space, the movement speed update unit 15 determines not to continue the movement speed adjustment process (step S108: NO) and ends the operation.

[0049] Note that, in the above-described movement speed adjustment process executed by the information processing device 1, a case where the operator and other users exist in the metaverse space has been described, but multiple other users other than the operator may exist in the metaverse space. For example, in step S101, the information processing device 1 determines, for each of the other users other than the operator who exist in the metaverse space, whether the distance in the virtual space from the operator is equal to or less than a threshold, and executes the processes of steps S101 to S107 for each user whose distance from the operator is determined to be equal to or less than the threshold.

[0050] Next, the effects achieved by the information processing device 1 according to the embodiment will be described. Figures 4 and 5 are diagrams for explaining the effects achieved by the information processing device according to the embodiment.

[0051] When a real-space user and a metaverse space user move together while talking on the metaverse, the movement speed of the real-space user and the movement speed of the metaverse space user may differ. This is because the movement speed of a real-space user is subject to physical limitations, whereas the movement speed of a metaverse space user is determined in advance as a parameter. For example, if the movement speed of a metaverse space user is faster than the movement speed of a real-space user, the metaverse space user must make detailed operations to match the movement speed of the real-space user. This creates the problem that the operation of the metaverse space user is cumbersome and prevents the user from concentrating on communication.

[0052] There is also a method of reflecting a user's movement in the metaverse space using the user's location information. For example, the user's device measures the user's location information at regular intervals, and estimates the user's movement direction and speed based on two consecutive pieces of location information, thereby reflecting the user's movement in the metaverse space.

[0053] However, as described above, when controlling a user's movement based on location information, the user's position in the metaverse space is corrected at regular intervals when the location information is measured, as seen from the user in the real space. For example, as shown in Figure 4, when a user (a real space user) is moving and talking with a user in the metaverse space connected via a network, if a discrepancy occurs between the actual movement speed of the user in the metaverse space and the movement speed estimated from the location information, the user's position in the metaverse space will be suddenly corrected when the user's location information in the metaverse space is measured again. This can cause the users to feel uncomfortable with each other's movements, hindering communication.

[0054] Another possible method is to increase the frequency of location information measurement, which would allow users' movements to be reflected more accurately in the metaverse space and reduce the amount of location correction required. However, this method increases the communication and processing load on user devices.

[0055] In contrast, the information processing device 1 according to the embodiment corrects the moving speed of a user in the metaverse space as seen from a user in real space based on the volume of the conversation between the users. Based on the fact that conversations between people who are close to each other continue to be heard at the same volume, the information processing device 1 adjusts the moving speed of the user in the metaverse space so as to maintain the volume of the conversation between the user and the operator who is moving and talking with the operator in the metaverse space.

[0056] As a result, if the user's movement speed in the real world differs from the user's movement speed in the metaverse space, the user's movement speed in the metaverse space is automatically adjusted, eliminating the need for the user to perform complicated operations and preventing communication from being impeded. Furthermore, even when the user's movement speed is controlled based on location information, as shown in Figure 5, the amount of position correction required for positioning information measurement is reduced, preventing users from feeling uncomfortable with each other's movements.

[0057] Furthermore, the information processing device 1 according to the embodiment adjusts the movement speed by utilizing conversational voice data that can be naturally acquired, particularly in a metaverse space involving communication such as CVE. Therefore, with the information processing device 1 according to the embodiment, there is no need to take the trouble of acquiring data for position adjustment.

[0058] Therefore, according to the information processing device 1 according to the embodiment, it is possible to provide an information processing device and an information processing method that are capable of adjusting the movement speed of a user on the metaverse without impeding communication.

[0059] The present invention is not limited to the above-described embodiments, and various modifications can be made in the implementation stage without departing from the spirit of the invention. Furthermore, the embodiments may be implemented in appropriate combinations, in which case the combined effects can be obtained. Furthermore, the above-described embodiments include various inventions, and various inventions can be extracted by combining selected elements from the disclosed elements. For example, if the problem can be solved and the desired effect can be obtained even if some elements are deleted from all elements shown in the embodiments, the configuration from which these elements are deleted can be extracted as an invention.

[0060] DESCRIPTION OF SYMBOLS 1... Information processing device 10... Control unit 11... Person distance measurement unit 12... Movement speed confirmation unit 13... Sound measurement unit 14... Movement speed adjustment unit 15... Movement speed update unit 20... Storage unit 21... Real space position information storage unit 22... Metaverse space position information storage unit 23... Real space audio data storage unit 24... Metaverse space audio data storage unit 25... Metaverse space parameter storage unit 30... Input unit 40... Output unit 50... Communication unit

Claims

1. An information processing device that enables communication between users in a virtual space by communicating with other terminals connected via a network, comprising: a person distance measurement unit that determines whether an operator and another user are moving together in the virtual space; an audio measurement unit that acquires conversational audio between the operator and a user that is determined to be moving together with the operator for a predetermined period of time; a movement speed confirmation unit that acquires parameters of the user's movement speed in the virtual space; a movement speed adjustment unit that adjusts the value of the user's movement speed based on the volume of the conversational audio acquired for the predetermined period of time; and a movement speed update unit that updates the parameters of the user's movement speed.

2. The information processing device described in claim 1, wherein the person distance measurement unit determines that the operator and the user are moving together in the virtual space when the value of the distance in the virtual space between the operator and the user is below a predetermined threshold for a predetermined period of time.

3. The information processing device of claim 1, wherein the movement speed adjustment unit adjusts the value of the movement speed so that the user's movement speed increases if it determines that the volume of the conversation voice for the acquired predetermined time is smaller than a range of appropriate values, and adjusts the value of the movement speed so that the user's movement speed decreases if it determines that the volume of the conversation voice for the acquired predetermined time is larger than a range of appropriate values.

4. An information processing device as described in claim 1, further comprising a parameter storage unit that stores the movement speed parameters of any user in the virtual space in a manner that allows them to be identified for each user, wherein the movement speed confirmation unit acquires the movement speed parameters of the user determined to be moving together with the operator, which are stored in the parameter storage unit, and the movement speed update unit updates the movement speed parameters of the user stored in the parameter storage unit to the movement speed value adjusted by the movement speed adjustment unit.

5. A method executed by an information processing device that enables communication between users in a virtual space by communicating with other terminals connected via a network, the method comprising: determining whether an operator and another user are moving together in the virtual space; acquiring a predetermined period of conversational audio between the operator and the user determined to be moving together with the operator; acquiring parameters of the user's movement speed in the virtual space; adjusting the value of the user's movement speed based on the volume of the conversational audio for the predetermined period of time; and updating the parameters of the user's movement speed.

Images