Communication system, display control method of communication system, and program

Abstract

To enable remote users to communicate with each other while exchanging opinions and impressions with each other by inviting a specific user to the same real space and linking both the real space and a virtual space in digital twin service provided by a server device.SOLUTION: The server device 1 includes an avatar control unit 16-3 that performs control to display, on the second data terminals 4-1 to 4-N and 5-1 to 5-N, the second avatar at the virtual-space coordinate position subjected to coordinate transformation to the digital reality space based on the real-space coordinate position designated by the first user and acquired from the first data terminals 3-1 to 3-N via the predetermined communication media. 1A.SELECTED DRAWING: Figure 2

Description

[Technical Field]

[0001] The present invention relates to a communication system in which a virtual space and a real space are linked to communicate with each other, and a display control method and program for the communication system. [Background technology]

[0002] In recent years, virtual events using virtual reality have been held frequently.

[0003] This type of virtual event is a general term for events hosted by server devices on the Internet, and is characterized by the fact that participants do not need to physically gather in the same place and that they take place in an online space that can be accessed from anywhere in the world.

[0004] In recent years, due to the spread of the new coronavirus disease (COVID-19), virtual events have rapidly gained attention as a means for people to communicate, exchange information, and enjoy entertainment while being restricted from going out.

[0005] The following Patent Document 1 discloses that "in order to enable real-space participants in an event that uses both real and virtual spaces to easily enjoy the event, the image generation system comprises a virtual space image acquisition unit that acquires in real time a virtual space image, which is a screen that visualizes the virtual space; a photographing unit that photographs the user and generates the photographed image; an image processing unit that combines the virtual space image with the photographed image to generate a composite image; and a display device that displays the composite image." [Prior art documents] [Patent documents]

[0006] [Patent Document 1] Japanese Patent Publication No. 2023-65073 Summary of the Invention [Problem to be solved by the invention]

[0007] The virtual event system disclosed in Patent Document 1 allows pre-registered users to operate data terminals via a network to connect to a server device hosting the virtual event, allowing the users to operate avatars associated with the data terminals and move around the virtual event venue. Recently, digital twins have been attracting attention as a technology for reproducing information about physical spaces in virtual spaces and performing simulations and predictions. In particular, the advantage of using technologies such as IoT, AI, and AR to copy real-world environments into virtual spaces and reproduce physical spaces in real time is that the virtual space is reflected in the real space, leading to the active proposal of various uses.

[0008] On the other hand, when a real building appears in a digital twin system, one user can view digital content that is replicated from the real building displayed in the virtual space, but it has been pointed out that specific users cannot be invited into the same virtual space to exchange opinions and impressions about the same digital content, and this means that the scalability and characteristics of digital twins cannot be fully utilized.

[0009] The present invention has been made to solve the above-mentioned problems, and aims to provide a communication system, a display control method for a communication system, and a program for the communication system, which allows remote users to communicate with each other by inviting specific users to the same real space and linking both the real space and the virtual space in a digital twin service provided by a server device, and exchanging opinions and impressions with each other. [Means for solving the problem]

[0010] The communication system of the present invention that achieves the above object has the following configuration.

[0011] The communication system of the present invention is a communication system in which a server device that provides a digital twin service that reproduces real space as a digital real space via a virtual space platform, a first data terminal operated by a first user connected to the virtual space platform, and a second data terminal operated by a second user connected to the virtual space platform communicate via a predetermined communication medium, and the server device is equipped with an avatar control means that, when providing a digital twin on the virtual space platform to the second data terminal in which the real space coordinates where the first user exists are replaced with virtual space coordinates, controls avatar display so that a second avatar corresponding to the second user invited by the first user appears on the virtual space platform at a virtual space coordinate position that corresponds to the real space coordinate position specified by the first user. [Effects of the Invention]

[0012] According to the present invention, in a digital twin service provided by a server device, specific users can be invited to the same real space, and both the real space and the virtual space can be linked, allowing remote users to communicate with each other and exchange opinions and impressions. [Brief explanation of the drawings]

[0013] The drawings illustrate particular embodiments of the present invention, including essential features of the invention as well as alternative and preferred embodiments. [Figure 1] 1 is a block diagram illustrating the configuration of a communication system according to an embodiment of the present invention. [Figure 2] FIG. 2 is a block diagram illustrating the configuration of the server device shown in FIG. [Figure 3] 2A is a block diagram illustrating the configuration of the first data terminal shown in FIG. 1, and FIG. 2B is a diagram illustrating a program loaded in the RAM shown in FIG. 2A. [Figure 4]2A is a block diagram illustrating the configuration of the second data terminal shown in FIG. 1, and FIG. 2B is a diagram illustrating a program loaded in the RAM shown in FIG. 2A. [Figure 5] FIG. 1 is a schematic diagram showing a display mode of a digital real space provided by a communication system according to an embodiment of the present invention. [Figure 6] 6 is a flowchart showing an example of data processing by a first data terminal in the communication system according to the present embodiment. [Figure 7] 10 is a flowchart showing an example of data processing by the server device in the communication system according to the embodiment. [Figure 8] 10 is a flowchart showing an example of data processing by a second data terminal in the communication system according to the present embodiment. [Figure 9] 10 is a diagram illustrating an example in which a second user is invited and displayed as a second avatar on a first data terminal in the communication system according to the present embodiment. FIG. [Figure 10] 10 is a diagram illustrating an example in which a second user is invited and displayed as a second avatar on a first data terminal in the communication system according to the present embodiment. FIG. [Figure 11] 10 is a diagram illustrating an example in which a second user is invited and displayed as a second avatar on a first data terminal in the communication system according to the present embodiment. FIG. DETAILED DESCRIPTION OF THE INVENTION

[0014] Next, the best mode for carrying out the present invention will be described with reference to the drawings.

[0015] <System configuration description> [First embodiment] 1 is a block diagram illustrating the configuration of a communication system according to this embodiment. This embodiment exemplifies a communication system in which a server device 1 provides a digital twin service that reproduces a real space as a digital real space (digital twin) via a metaverse platform 1A over a predetermined communication medium, first data terminals 3-1 to 3-N operated by a first user who connects to the metaverse platform 1A, and second data terminals 4-1 to 4-N, 5-1 to 5-N operated by a second user who connects to the metaverse platform 1A communicate with each other via a predetermined communication medium. Note that multiple data terminals can be connected to the metaverse platform 1A simultaneously.

[0016] 1, first data terminals 3-1 to 3-N connect to a metaverse platform 1A made public by a server device 1 and instruct the server device 1 to make a digital real space appear that represents a real space corresponding to the current location in the real space (coordinate values ​​identified by GPS (current location of the first user)). In order to make the digital real space appear, the server device 1 cooperates with, for example, a server device 100 that provides predetermined limited provision data, and executes a process to make a digital real space appear on the metaverse platform 1A as a copy of the real space viewed from the current location of the first user in response to a request from the first user operating the first data terminals 3-1 to 3-N.

[0017] Here, when the second user receives an invitation to the digital real space from the first user as described below, the second user can operate the second data terminals 4-1 to 4-N, 5-1 to 5-N to connect to the server device 1, and can then view the digital real space reproduced on the metaverse platform 1A via a display device that communicates with the second data terminals 4-1 to 4-N, 5-1 to 5-N.

[0018] In this embodiment, an HMD is used as an example of a display device, but the present invention can also be applied to display units provided in the first and second data terminals.

[0019] In addition, when a first user operating a first data terminal 3-1 to 3-N specifies a specific URL and a second data terminal 4-1 to 4-N, 5-1 to 5-N connects to the metaverse platform 1A, the server device 1 performs a process of inviting the second user to a digital real space (a copy of the real space with the same coordinate values ​​as the first user is located) reproduced on the metaverse platform 1A.

[0020] The first data terminals 3-1 to 3-N are equipped with hardware resources as shown in FIG. 3, and similarly, the second data terminals 4-1 to 4-N, 5-1 to 5-N are equipped with hardware resources as shown in FIG. 4, and are connected to the metaverse platform 1A via the network 21 to receive communication services from the server device 1.

[0021] In the communication system shown in this embodiment, the server device 100 functions as a server device for reproducing a digital twin, but the server device 1 may also have the functions of the server device 100.

[0022] By making the server device 100 independent from the server device 1 of this system, the server device 1 has the advantage of not having to bear the burden of the massive data update processing (data update processing when the landscape changes due to reconstruction, urban redevelopment, etc.) and data maintenance work processing that the server device 100 originally functions to perform.

[0023] FIG. 2 is a block diagram for explaining the configuration of the server device 1 shown in FIG. 1, and the same components as those in FIG. 1 are assigned the same reference numerals.

[0024] In FIG. 2, reference numeral 11 denotes a communication unit that controls communication processing for connecting first data terminals 3-1 to 3-N and second data terminals 4-1 to 4-N, 5-1 to 5-N connected to a network 21 as a communication medium with a server device 100 that acquires image information for reproducing a predetermined digital real space.

[0025] Reference numeral 13 denotes a CPU, which starts an operating system (OS) stored in external memory 18 connected to internal bus 12 and launches various applications installed via API. It also executes the processing of various programs deployed on RAM 16. Reference numeral 14 denotes a database, which stores prototype models of avatars used to create a virtual space and various items to be placed in the virtual space, as well as account information for a first user operating first data terminals 3-1 to 3-N and a second user operating second data terminals 4-1 to 4-N, 5-1 to 5-N to connect to metaverse platform 1A.

[0026] 15 is a keyboard, which is used to input numbers, text, and icon commands for running applications.

[0027] Reference numeral 16 denotes an expandable RAM, which expands and stores the basic program of the metaverse platform 1A installed in the external memory 18. Reference numeral 17 denotes a display, which displays the program's running status via various UI screens (not shown).

[0028] The basic program in this embodiment is deployed on RAM 16 and executed by CPU 13: an appearance unit 16-1 that acquires the current location information of the first user and makes the real space appear as a digital real space on the virtual space platform; an invitation unit 16-2 that invites a second user designated by the first user as a second avatar into the digital real space that has appeared on metaverse platform 1A by appearance unit 16-1; and an avatar control unit 16-3 that controls the display of the second avatar on second data terminals 4-1 to 4-N, 5-1 to 5-N at a virtual space coordinate position that is coordinate-converted to digital real space based on the real space coordinate position specified by the first user acquired from first data terminals 3-1 to 3-N.

[0029] As shown in the flowchart below, the avatar control unit 16-3 is configured to control the display of the second avatar on the first data terminals 3-1 to 3-N at a real space coordinate position where the virtual space coordinate position at which the second avatar is displayed is converted into real space coordinates.

[0030] 3(a), the first HMD 322 includes a first audio processor 322A and a display controller 322B that converts audio information uttered by a first user into an audio signal and transmits it to the server device 1, and converts audio information uttered by a second user received from the server device 1 into an audio signal and outputs it from a speaker. In this embodiment, the first audio processor 322A performs functional processing as a microphone and functional processing as a speaker.

[0031] The first HMD 322 is configured to be able to display various types of information on a display device (not shown) under the control of a display control unit 322B.

[0032] In addition, the avatar control unit 16-3 controls the display of the second avatar on the first HMD 322 operated by the first user by associating the virtual space coordinates at which the second avatar is displayed with the real space coordinate positions converted into the real space coordinates.

[0033] Similarly, the avatar control unit 16-3 controls the display of the first avatar on the second HMD 323 operated by the second user by associating the real space coordinates where the first user exists with virtual space coordinate positions converted into virtual space coordinates.

[0034] In addition, the appearance unit 16-1 acquires GPS information indicating the movement state of the first user in the real space from the first data terminal 3-1 to 3-N, and performs a process of continuously making a digital real space that is adapted to the movement state appear on the metaverse platform 1A.

[0035] As a result, the display of the second avatar is controlled so that the avatar of the second user also moves in the digital real space in accordance with the movement of the first user shown in FIG. 11, which will be described later.

[0036] Furthermore, in the server device 1, the appearance unit 16-1 is configured to make a real space corresponding to the first user's current location appear on the metaverse platform 1A by using predetermined limited provision data linked to the GPS information provided by the server device 100.

[0037] Here, the specified limited provision data is three-dimensional high-precision map data, and is composed of limited provision data that accumulates a three-dimensional still image of the surrounding area including a specific building by identifying GPS information.

[0038] In addition, image information including buildings, roads, etc. that represent real space is managed by GPS mapping, and is configured to be able to be linked with map information obtained from a server device 100 that provides map information connected to the network 21.

[0039] 19 is the AI ​​support unit, which supports the massive spatial modeling process to support the process of making the virtual space appear so that the virtual space moves seamlessly, even when a virtual space movement occurs such as leaving the first virtual space and entering a second virtual space.

[0040] Fig. 3 is a block diagram illustrating the configuration of first data terminals 3-1 to 3-N connected to the server device 1 shown in Fig. 1. Note that the first data terminals 3-1 to 3-N are assumed to be pad devices, PC devices, smartphones, etc. Note that the HMD (Head Mounted Display (HMD)) is assumed to be configured to perform communication processing with the first data terminals 3-1 to 3-N via WIFI or Bluetooth (registered trademark), but if the HMD has a wireless communication function for a data terminal, it is also possible for the HMD alone to constitute a data terminal.

[0041] Furthermore, this system may be configured to require the use of an HMD, or may be configured as a system that links with the screen of a data terminal instead of an HMD.

[0042] 3(a), 301 denotes a CPU, which executes various applications by loading an OS and control programs stored in ROM 302 into RAM 303 and executing them. 304 denotes a communication unit, which controls communication for connecting to a server device 1 connected to a network 21. 310 denotes a keyboard, which is configured to be able to input text onto a screen displayed on a display device (not shown) in cooperation with a pointing device (not shown) and to select tabs and icons on the displayed screen.

[0043] 3(a), the first HMD 322 includes a first audio processor 322A and a display controller 322B that converts audio information uttered by a first user into an audio signal and transmits it to the server device 1, and also converts audio information uttered by a second user received from the server device 1 into an audio signal and outputs it from a speaker. In this embodiment, the first audio processor 322A performs functional processing as a microphone and functional processing as a speaker.

[0044] The first HMD 322 is configured to be able to display various types of information on a display device (not shown) under the control of a display control unit 322B.

[0045] In RAM 303 shown in (b) of Figure 3, 303-1 is a UI control unit that controls the display of a UI screen presented from the metaverse platform 1A provided by the server device 1 shown in Figure 1 on the display unit provided in the first HMD 322 or the first data terminals 3-1 to 3-N.

[0046] The CPU 301 manages account information (user ID, password) registered by a user in the metaverse platform 1A that the server device 1 makes public on the web, using an external storage device such as an SSD (not shown).

[0047] Fig. 4 is a block diagram illustrating the configuration of second data terminals 4-1 to 4-N and 5-1 to 5-N connected to server device 1 shown in Fig. 1. Note that the second data terminals 4-1 to 4-N and 5-1 to 5-N are assumed to be pad devices, PC devices, smartphones, etc.

[0048] The second HMD 323 is assumed to be configured to perform communication processing with the second data terminals 4-1 to 4-N, 5-1 to 5-N via WIFI or Bluetooth (registered trademark), but if the second HMD 323 has wireless communication functionality with the second data terminals 4-1 to 4-N, 5-1 to 5-N, it is also possible for the HMD alone to function as a data terminal.

[0049] 4(a), 301 denotes a CPU, which executes various applications by loading an OS and control programs stored in ROM 302 into RAM 303 and executing them. Various information is input and icons displayed on a user interface screen (UI screen) are clicked directly on the display unit provided in the second data terminals 4-1 to 4-N, 5-1 to 5-N. 304 denotes a communication unit, which controls communication for connecting to a server device 1 connected to the network 21.

[0050] 4(a), the second HMD 323 includes a second audio processor 323A and a display controller 323B that converts audio information uttered by the second user into an audio signal and transmits it to the server device 1, and also converts audio information uttered by the first user received from the server device 1 into an audio signal and outputs it from a speaker. In this embodiment, the second audio processor 323A performs functional processing as a microphone and functional processing as a speaker.

[0051] The second HMD 323 is configured to be able to display various types of information on a display device (not shown) under the control of a display control unit 323B.

[0052] This allows the first user and the second user to have simultaneous two-way voice communication with each other on the metaverse platform 1A via the first HMD 322 and the second HMD 323.

[0053] In RAM 303 shown in (b) of Figure 4, 303-1 is a UI control unit that controls the display of a UI screen presented from the metaverse platform 1A provided by the server device 1 shown in Figure 1 on the display unit provided in the second HMD 323 or the second data terminals 4-1 to 4-N, 5-1 to 5-N.

[0054] The CPU 301 manages account information (user ID, password) registered by a user in the metaverse platform 1A that the server device 1 makes public on the web, using an external storage device such as an SSD (not shown).

[0055] Figure 5 is a schematic diagram showing the display mode of the digital real space provided by the communication system showing this embodiment. In this example, a first user 51B is located in a real space 51, for example, in the middle of a city, wearing a first HMD 322 and looking at buildings. The first user invites a second avatar 52C of a second user to the digital real space 52, and by transmitting GPS information corresponding to the current location to the server device 1, specific limited provision data that reproduces the digital real space 52 of the real space is obtained from the server device 100. This allows the first user 51B and the second avatar 52C of the second user to be close to each other in the single digital real space 52, as if accompanying the first user 51B to his / her current location, and provides the second user with the experience of sharing the real space 51 that the first user is focusing on. Furthermore, the first user operating the first HMD 322 in the real space 51 can visually recognize the second avatar 51C of the second user on the screen of the first HMD 322. Furthermore, in digital reality space 52, the second user can view first avatar 52B corresponding to first user 51B and second avatar 52C representing the second user on the screen of second HMD 323. Specific display control of first avatar 52B and second avatar 52C will be described with reference to a flowchart described later. In Figure 5, 51A is the real building viewed by the first user 51B, and the second user can view object 52A (a digital copy of building 51A) displayed in digital real space 52 as a digital twin of building 51A on the screen of the second HMD 323.

[0056] As a result, as shown in FIG. 5, the second HMD 323 displays a first avatar 52B corresponding to the first user 51B and a second avatar 52C corresponding to the second user.

[0057] In addition, the first HMD 322 controls the manner of avatar display (including combinations of color, shape, display, lighting, and blinking) so that a second avatar 51C corresponding to a second user can be displayed and a first avatar 52B corresponding to a first user can be identified.

[0058] 6 is a flowchart showing an example of data processing by the first data terminals 3-1 to 3-N in the communication system according to this embodiment. Note that (1) to (6) indicate steps, which are implemented by the CPU 301 loading a program stored in an external memory (not shown) onto the RAM 303 and executing it.

[0059] After the first user turns on the power of the first data terminal 3-1 to 3-N, the CPU 301 confirms that the first HMD 322 is running (1). The first HMD 322 is equipped with a GPS chip (not shown), which is a chip incorporating a circuit for receiving signals from GPS satellites and calculating the current location. The GPS chip acquires location information corresponding to the first user's current location and transmits it to the server device 1 to transfer the location information (2).

[0060] Next, the first user selects the user ID corresponding to the second user whom he / she wishes to invite into the real space while referring to a list of second users registered in the internal memory of the first data terminals 3-1 to 3-N on a screen that is not displayed on the first data terminals 3-1 to 3-N (3).

[0061] Next, the CPU 301 transmits the user ID of the second user selected by the first user to invite and the location information of the first user (current location (identified from the GPS signal)) to the server device 1 (4).

[0062] Thereafter, when the first data terminal 3-1 to 3-N receives digital real-space location information in which the second user's virtual space coordinate values ​​have been converted into latitude and longitude in real space from the server device 1 via the network 21 (5), the CPU 301 displays a second avatar corresponding to the second user on the screen of the first HMD 322 at the longitude and latitude position identified in the received real space of the second user (6), and ends the processing.

[0063] In this way, a first user operating a first data terminal 3-1 to 3-N and a second user operating a second data terminal 4-1 to 4-N, 5-1 to 5-N can view the digital real space displayed on the metaverse platform 1A provided by the server device 1, which is the real space viewed from the first user's current location.

[0064] In this case, a second avatar representing the second user is displayed in an invited manner on the screen of the first HMD 322 operated by the first user, and on the screen of the second HMD 323 operated by the second user, the real space corresponding to the first user's current location can be experienced in a virtual space.

[0065] From then on, the CPU 301 allows the first user to experience walking around the real space or accompanying the second avatar as part of a sightseeing trip while facing the second avatar in the real space until the first user logs out of the metaverse platform 1A.

[0066] 7 is a flowchart showing an example of data processing by the server device 1 in the communication system according to this embodiment. Note that (11) to (16) indicate steps, which are implemented by the CPU 13 loading a program stored in the external memory 18 onto the RAM 16 and executing it.

[0067] First, CPU 13 acquires user information (user name, virtual space avatar, current location information) that identifies the first user transmitted from the first data terminal 3-1 to 3-N operated by the first user via network 21 (11). Then, CPU 13 references the user ID corresponding to the second user invited from the second data terminal 4-1 to 4-N, 5-1 to 5-N (12), and executes a process of converting the current location information (indicating longitude and latitude) of the first user into virtual space coordinates that are visible to the second user (13).

[0068] Next, the CPU 13 transmits a notification (including the virtual space coordinates and the platform coordinates) to the second data terminals 4-1 to 4-N, 5-1 to 5-N, based on the user ID that identifies the second user, that the second user has been invited to the digital real space (14).

[0069] Next, CPU 13 executes a process of converting the virtual space coordinates where the second avatar corresponding to the second user operating second data terminal 4-1 to 4-N, 5-1 to 5-N is located into latitude and longitude indicating the real space coordinates where the first user operating first data terminal 3-1 to 3-N is located (15). Next, CPU 13 transmits the second avatar data and information indicating the converted latitude and longitude to first data terminal 3-1 to 3-N (16), and ends the process.

[0070] 8 is a flowchart showing an example of data processing by the second data terminals 4-1 to 4-N and 5-1 to 5-N in the communication system according to this embodiment. Note that (21) to (23) indicate steps, which are implemented by the CPU 301 loading a program stored in an external memory (not shown) onto the RAM 303 and executing it.

[0071] First, when the CPU 301 of the second data terminal 4-1 to 4-N, 5-1 to 5-N receives a notification of an invitation to the real space obtained from the first user from the server device 1 (21), the CPU 301 of the second data terminal 4-1 to 4-N, 5-1 to 5-N generates a second avatar representing the second user at coordinates indicating the real space where the first user is located (22).

[0072] Next, the CPU 301 of the second data terminals 4-1 to 4-N, 5-1 to 5-N transmits to the server device 1 second avatar data for visualizing a second avatar corresponding to the second user and the coordinates of the second avatar to be displayed in the virtual space (23), and ends the process.

[0073] 9 to 11 are diagrams illustrating how a second user is invited as a second avatar on the first data terminals 3-1 to 3-N in the communication system according to this embodiment. ST1 to ST3 indicate the timing on the first user's side, and ST11 to ST13 indicate the timing on the second user's side. The star displayed on the first data terminals 3-1 to 3-N shown in Fig. 10 indicates the invitation position designated by the first user to invite the second avatar corresponding to the second user.

[0074] [Timing ST1 (real space) vs. Timing ST11 (digital real space)] As shown in Figure 9, a first HMD 322 is prepared on the first user side and is connectable to first data terminals 3-1 to 3-N, and a second HMD 323 is prepared on the second user side and is connectable to second data terminals 4-1 to 4-N, 5-1 to 5-N.

[0075] [Timing ST2 (real space) vs. Timing ST12 (digital real space)] 9, a first user U1 is walking around town wearing the first HMD 322 after the first HMD 322 is prepared and connected to the first data terminals 3-1 to 3-N. The first user U1 then wants to invite a second user U2 into the real space.

[0076] [Timing ST3 (real space) vs. Timing ST13 (digital real space)] Therefore, as shown in FIG. 10, a first user U1 designates coordinates (star shapes in the figure) for inviting a second avatar corresponding to a second user U2 into real space.

[0077] The specified coordinates of the real space (position information (latitude, longitude) acquired from the GPS) are transmitted from the first data terminals 3-1 to 3-N to the server device 1, and are converted into virtual space coordinates by the CPU 13 of the server device 1 in step (13) shown in FIG. 7.

[0078] This converts the coordinate values ​​into a virtual reality space (digital reality space) as a digital twin of the real space, and the second user U2 sees a second avatar AV2 displayed at the invitation position specified by the first user U1, as shown in Figure 10.

[0079] Furthermore, when the first user U1 moves on the road in the direction of the arrow (timing ST4), the second avatar AV2 (timing ST14) corresponding to the second user in the digital virtual space also moves in the direction of the arrow, following the movement, as shown in Figure 11.

[0080] [Effects of the first embodiment] According to this embodiment, in the digital twin service provided by the server device 1, specific users can be invited to the same real space, and while linking both the real space and the virtual space, remote users can communicate with each other, exchanging opinions and impressions. This makes it possible to significantly increase the possibilities and scalability of communication services that use digital twin technology.

[0081] Second Embodiment In the above embodiment, a system in which a first user and a second user communicate one-to-one was described, but the present invention can also be applied to a system in which a first user invites multiple second users and communicates one-to-N simultaneously.

[0082] The present invention can also be realized by supplying a program that realizes one or more functions of the above-described embodiments to a system or device via a network or a storage medium, and having one or more processors in the computer of the system or device read and execute the program. It can also be realized by a circuit (e.g., ASIC) that realizes one or more functions.

[0083] Specifically, the present invention includes a server computer in a communication system in which a server device that provides a digital twin service that reproduces a real space as a digital real space via a virtual space platform, a first data terminal operated by a first user that connects to the virtual space platform, and a second data terminal operated by a second user that connects to the virtual space platform communicate via a predetermined communication medium, and a step of acquiring current location information of the first user and making the real space appear as a digital real space on the virtual space platform; and an avatar control step of controlling the display of the second avatar on the first data terminal at a virtual space coordinate position that is converted into the digital real space based on the real space coordinate position specified by the first user and obtained from the first data terminal. In this system, we have explained the case where a virtual reality space is displayed on the screen using an HMD, but the same effect can be expected with a configuration that uses intelligent glasses (XR type).

[0084] The disclosure of the present invention described above can be summarized at least as follows.

[0085] (1) A communication system in which a server device that provides a digital twin service that reproduces real space as a digital real space via a virtual space platform communicates with a first data terminal operated by a first user connected to the virtual space platform and a second data terminal operated by a second user connected to the virtual space platform via a predetermined communication medium, wherein the server device is characterized by having an avatar control means that, when providing a digital twin on the virtual space platform to the second data terminal in which the real space coordinates where the first user exists are replaced with virtual space coordinates, controls avatar display so that a second avatar corresponding to the second user invited by the first user appears on the virtual space platform at a virtual space coordinate position that corresponds to the real space coordinate position specified by the first user.

[0086] (2) The avatar control means is characterized in that it controls the display of the second avatar on a first HMD operated by the first user by correlating the virtual space coordinates at which the second avatar is displayed with the real space coordinate positions converted into the real space coordinates.

[0087] (3) The avatar control means is characterized in that it controls the display of the first avatar on a second HMD operated by the second user by correlating the real-space coordinates where the first user exists with the virtual-space coordinate positions converted into the virtual-space coordinates.

[0088] (4) The first HMD is characterized in that it has a first audio processing unit that converts the audio information spoken by the first user into an audio signal and transmits it to the server device, and that converts the audio information spoken by the second user received from the server device into an audio signal and outputs it from a speaker; the second HMD is characterized in that it has a second audio processing unit that converts the audio information spoken by the second user into an audio signal and transmits it to the server device, and that converts the audio information spoken by the first user received from the server device into an audio signal and outputs it from a speaker; and the first user and the second user are configured to be able to communicate with each other on the virtual space platform.

[0089] (5) The avatar control means is characterized in that it is configured to acquire GPS information indicating the current location of the first user in the real space coordinates and make a digital real space appear on the virtual space platform.

[0090] (6) The avatar control means is characterized in that it is configured to acquire GPS information indicating the movement state of the first user in the real space and continuously cause a digital real space adapted to the movement state to appear on the virtual space platform.

[0091] (7) A display control method for a communication system in which a server device that provides a digital twin service that reproduces real space as a digital real space via a virtual space platform via a predetermined communication medium, a first data terminal operated by a first user connected to the virtual space platform, and a second data terminal operated by a second user connected to the virtual space platform communicate, characterized in that the server device comprises: an appearance step of acquiring current location information of the first user and making the real space appear on the virtual space platform as a digital real space; an invitation step of inviting the second user designated by the first user as a second avatar into the digital real space that appeared on the virtual space platform in the appearance step; and an avatar control step of controlling the display of the second avatar on the second data terminal at a virtual space coordinate position that is coordinate-converted to the digital real space based on the real space coordinate position designated by the first user acquired from the first data terminal.

[0092] (8) The avatar control step is characterized by controlling the display of the second avatar on the first data terminal at a real space coordinate position where the virtual space coordinate position at which the second avatar is displayed is converted into real space coordinates.

[0093] (9) The avatar control step is characterized in that a second avatar corresponding to the second user is displayed on a first HMD operated by the first user.

[0094] (10) A program that causes a server computer in a communication system in which a server device that provides a digital twin service that reproduces real space as a digital real space via a virtual space platform, a first data terminal operated by a first user connected to the virtual space platform, and a second data terminal operated by a second user connected to the virtual space platform communicate via a predetermined communication medium, to execute the following steps: an appearance step of acquiring current location information of the first user and making the real space appear on the virtual space platform as a digital real space; an invitation step of inviting the second user designated by the first user as a second avatar into the digital real space that appeared on the virtual space platform in the appearance step; and an avatar control step of controlling the second avatar to be displayed on the second data terminal at a virtual space coordinate position that is coordinate-converted into the digital real space based on the real space coordinate position designated by the first user acquired from the first data terminal, and controlling the second avatar to be displayed on the first data terminal at a real space coordinate position that is converted from the virtual space coordinate position at which the second avatar is displayed to real space coordinates. [Industrial Applicability]

[0095] In the above embodiment, an example was described in which the first user and the second user are displayed as avatars, but by inputting the physical feature data of the first user and the second user, it is also possible to configure the system to display a 3D modeled object in place of the avatar, allowing the user to appear in the virtual space as a My Figure with physical features, creating a system that is easy to relate to.

[0096] Furthermore, by controlling the movements of the My Figure so that its parts, such as its arms, wrists, and fingers, move in three dimensions, it is also possible to create a system in which each user can wear sensors that measure body pressure and warmth and hold hands in a virtual space, even reproducing the feel of their hands. [Explanation of symbols]

[0097] 1. Server device 1A Metaverse Platform 3-1~3-N First data terminal 4-1~4-N Second data terminal 5-1~5-N Second data terminal 100 Server device

Claims

1. A communication system in which a server device that provides a digital twin service that reproduces a real space as a digital real space via a virtual space platform, a first data terminal operated by a first user that connects to the virtual space platform, and a second data terminal operated by a second user that connects to the virtual space platform communicate with each other via a predetermined communication medium, The server device A communication system characterized by comprising an avatar control means for controlling avatar display so that, when a digital twin in which the real-space coordinates where the first user exists are replaced with virtual-space coordinates is provided to the second data terminal on the virtual-space platform, a second avatar corresponding to the second user invited by the first user appears on the virtual-space platform at a virtual-space coordinate position that corresponds to the real-space coordinate position specified by the first user.

2. The avatar control means The communication system described in claim 1, characterized in that the virtual space coordinates at which the second avatar is displayed are associated with real space coordinate positions converted into real space coordinates, and the second avatar is controlled to be displayed on a first HMD operated by the first user.

3. The avatar control means The communication system described in claim 1, characterized in that the real space coordinates where the first user exists are associated with virtual space coordinate positions converted into the virtual space coordinates, and the first avatar is displayed on a second HMD operated by the second user.

4. The first HMD includes: a first audio processing unit that converts audio information uttered by the first user into an audio signal and transmits the converted audio signal to the server device, and converts audio information uttered by the second user received from the server device into an audio signal and outputs the audio signal from a speaker; The second HMD includes: a second audio processing unit that converts audio information uttered by the second user into an audio signal and transmits the converted audio signal to a server device, and converts audio information uttered by the first user received from the server device into an audio signal and outputs the audio signal from a speaker; 4. The communication system according to claim 2, wherein the first user and the second user are configured to be able to communicate with each other on the virtual space platform.

5. The communication system described in claim 1, characterized in that the avatar control means is configured to acquire GPS information indicating the current location of the first user in the real space coordinates and make a digital real space appear on the virtual space platform.

6. The communication system described in claim 1, characterized in that the avatar control means is configured to acquire GPS information indicating the movement state of the first user in the real space and continuously cause a digital real space adapted to the movement state to appear on the virtual space platform.

7. A display control method for a communication system in which a server device that provides a digital twin service that reproduces a real space as a digital real space via a virtual space platform, a first data terminal operated by a first user that connects to the virtual space platform, and a second data terminal operated by a second user that connects to the virtual space platform communicate with each other via a predetermined communication medium, comprising: The server device a step of acquiring current location information of the first user and making a real space appear as a digital real space on the virtual space platform; an inviting step of inviting the second user designated by the first user as a second avatar into the digital real space that has appeared on the virtual space platform in the appearing step; an avatar control step of controlling the second avatar to be displayed on the second data terminal at a virtual space coordinate position that is converted into the digital real space based on a real space coordinate position designated by the first user and acquired from the first data terminal; A display control method for a communication system, comprising:

8. The avatar control step includes: A display control method for a communication system as described in claim 7, characterized in that the second avatar is displayed on the first data terminal at a real space coordinate position where the virtual space coordinate position at which the second avatar is displayed is converted to a real space coordinate position.

9. The display control method for a communication system according to claim 7 or 8, characterized in that the avatar control step displays the second avatar corresponding to the second user on a first HMD operated by the first user.

10. A server computer in a communication system in which a server device that provides a digital twin service that reproduces a real space as a digital real space via a virtual space platform, a first data terminal operated by a first user that connects to the virtual space platform, and a second data terminal operated by a second user that connects to the virtual space platform communicate via a predetermined communication medium, a step of acquiring current location information of the first user and making a real space appear as a digital real space on the virtual space platform; an inviting step of inviting the second user designated by the first user as a second avatar into the digital real space that has appeared on the virtual space platform in the appearing step; a control step of controlling the display of the second avatar on the second data terminal at a virtual space coordinate position that is converted into the digital real space based on the real space coordinate position specified by the first user obtained from the first data terminal, and of controlling the display of the second avatar on the first data terminal at a real space coordinate position that is converted into real space coordinates from the virtual space coordinate position at which the second avatar is displayed.

Images