COMMUNICATION SYSTEM, DISPLAY CONTROL METHOD AND PROGRAM FOR COMMUNICATION SYSTEM

Abstract

In a digital twin service provided by a server device, specific users are invited to the same real space, and both the real space and the virtual space are linked, allowing remote users to communicate with each other by exchanging opinions and impressions. [Solution] A server device 1 that provides a digital twin service that reproduces real space as a digital real space via a metaverse platform 1A via a specified communication medium is equipped with an avatar control unit 16-3 that controls the display of a second avatar on a second data terminal 4-1 to 4-N, 5-1 to 5-N at a virtual space coordinate position that is coordinate-converted into digital real space based on the real space coordinate position specified by a first user obtained from a first data terminal 3-1 to 3-N.

Description

[Technical field]

[0001] The present invention relates to a communication system in which a virtual space and a real space are linked to each other to perform communication, and a display control method and program for the communication system. [Background technology]

[0002] In recent years, virtual events using virtual reality have become more and more popular.

[0003] This type of virtual event is a general term for events provided by server devices on the Internet, and is characterized by the fact that participants do not need to physically gather in the same place, but rather take place in an online space that can be accessed from anywhere in the world.

[0004] In recent years, due to the spread of the new coronavirus disease (COVID-19), virtual events have rapidly gained attention as a way for people to communicate, exchange information, and enjoy entertainment while being restricted from going out.

[0005] The following Patent Document 1 discloses that "in order to enable real-space participants in an event which uses both real and virtual spaces to easily enjoy the event, the image generation system comprises a virtual space image acquisition unit which acquires in real time a virtual space image which is a screen which visualizes the virtual space, a photographing unit which photographs the user and generates a photographed image, an image processing unit which combines the virtual space image with the photographed image to generate a composite image, and a display device which displays the composite image." [Prior art documents] [Patent documents]

[0006] [Patent Document 1] JP 2023-65073 A Summary of the Invention [Problem to be solved by the invention]

[0007] In the virtual event system disclosed in the above Patent Document 1, a pre-registered user operates a data terminal via a network to connect to a server device that hosts the virtual event, and can operate avatars associated with multiple users who operate the data terminal to move around the venue of the virtual event. Recently, digital twins have been attracting attention as a technology for reproducing information about a physical space in a virtual space and performing simulations and predictions. In particular, the advantage of being able to reproduce a physical space in real time by copying the environment of the real world into a virtual space using technologies such as IoT, AI, and AR is that the virtual space is reflected in the real space, and various uses have been actively proposed.

[0008] On the other hand, when a real building appears in a digital twin system, it is possible for one user to view digital content that is replicated from the real building displayed in the virtual space, but it has been pointed out that it is not possible to invite specific users into the same virtual space to exchange opinions and impressions about the same digital content, meaning that the scalability and characteristics of digital twins cannot be fully utilized.

[0009] The present invention has been made to solve the above-mentioned problems, and aims to provide a communication system, a display control method and a program for a communication system, in which specific users can be invited to the same real space in a digital twin service provided by a server device, and remote users can communicate with each other by exchanging opinions and impressions while linking both the real space and the virtual space. [Means for solving the problem]

[0010] A communication system according to the present invention for achieving the above object comprises the following configuration.

[0011] The communication system according to the present invention includes a server device that provides a digital twin service that reproduces a real space as a digital real space via a virtual space platform through a predetermined communication medium, a first data terminal operated by a first user that is connected to the virtual space platform, and Another server device that provides predetermined limited provision data and cooperates with the server device in order to create a digital reality space; A communication system for communicating with a second data terminal operated by a second user connected to the virtual space platform, The predetermined limited provision data is three-dimensional high-precision map data, and is composed of a plurality of still images that allow a specific building and its surrounding area to be viewed stereoscopically by identifying GPS information; The server device includes an avatar control means for controlling avatar display so that a second avatar corresponding to the second user invited by the first user appears on the virtual space platform at a virtual space coordinate position corresponding to the real space coordinate position designated by the first user when providing a digital twin in which real space coordinates where the first user exists are replaced with virtual space coordinates to the second data terminal on the virtual space platform. The avatar control means is configured to acquire the GPS information indicating the first user's current position in the real space coordinates and cause a digital real space to appear on the virtual space platform, and is also configured to acquire the GPS information indicating the first user's movement state in the real space and cause a digital real space adapted to the movement state to continuously appear on the virtual space platform. Effect of the Invention

[0012] According to the present invention, in a digital twin service provided by a server device, specific users can be invited to the same real space, and both the real space and the virtual space can be linked, allowing remote users to communicate with each other by exchanging opinions and impressions. [Brief description of the drawings]

[0013] The drawings show specific embodiments of the present invention, including essential features of the invention as well as alternative and preferred embodiments. [Figure 1] FIG. 1 is a block diagram illustrating a configuration of a communication system according to an embodiment of the present invention. [Diagram 2] FIG. 2 is a block diagram illustrating the configuration of a server device shown in FIG. [Diagram 3] FIG. 2A is a block diagram illustrating the configuration of the first data terminal shown in FIG. 1, and FIG. 2B is a diagram illustrating a program expanded in a RAM shown in FIG. [Figure 4]2A is a block diagram illustrating the configuration of the second data terminal shown in FIG. 1, and FIG. 2B is a diagram illustrating a program expanded in a RAM shown in FIG. [Diagram 5] FIG. 2 is a schematic diagram showing the display mode of a digital real space provided by the communication system according to the present embodiment. [Figure 6] 6 is a flowchart showing an example of data processing in a first data terminal in the communication system according to the present embodiment. [Figure 7] 6 is a flowchart showing an example of data processing by the server device in the communication system according to the embodiment. [Figure 8] 6 is a flowchart showing an example of data processing by a second data terminal in the communication system according to the present embodiment. [Figure 9] 10 is a diagram for explaining a mode in which a second user is invited and displayed as a second avatar on a first data terminal in the communication system according to the embodiment. FIG. [Figure 10] 10 is a diagram for explaining a mode in which a second user is invited and displayed as a second avatar on a first data terminal in the communication system according to the embodiment. FIG. [Figure 11] 10 is a diagram for explaining a mode in which a second user is invited and displayed as a second avatar on a first data terminal in the communication system according to the embodiment. FIG. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0014] Next, the best mode for carrying out the present invention will be described with reference to the drawings.

[0015] <System configuration> [First embodiment] 1 is a block diagram for explaining the configuration of a communication system according to the present embodiment. In the present embodiment, a communication system is exemplified in which a server device 1 that provides a digital twin service that reproduces a real space as a digital real space (digital twin) via a metaverse platform 1A, a first data terminal 3-1 to 3-N operated by a first user who connects to the metaverse platform 1A, and a second data terminal 4-1 to 4-N, 5-1 to 5-N operated by a second user who connects to the metaverse platform 1A communicate with each other via a predetermined communication medium. Note that a plurality of data terminals can be connected to the metaverse platform 1A at the same time.

[0016] 1, first data terminals 3-1 to 3-N connect to a metaverse platform 1A made public by a server device 1, and instruct the server device 1 to make a digital real space appear that represents a real space corresponding to the current position in the real space (coordinate values ​​specified by GPS (current position of the first user)). In order to make the digital real space appear, the server device 1 cooperates with, for example, a server device 100 that provides predetermined limited provision data, and executes a process of making a digital real space appear on the metaverse platform 1A as a copy of the real space viewed from the current position of the first user in response to a request from the first user operating the first data terminals 3-1 to 3-N.

[0017] Here, when the second user receives an invitation to the digital real space from the first user as described below, the second user can operate the second data terminals 4-1 to 4-N, 5-1 to 5-N to connect to the server device 1 and view the digital real space reproduced on the metaverse platform 1A via a display device that communicates with the second data terminals 4-1 to 4-N, 5-1 to 5-N.

[0018] In this embodiment, an HMD is used as an example of a display device, but the present invention can also be applied to display units provided in the first and second data terminals.

[0019] In addition, when a specific URL is specified by a first user operating a first data terminal 3-1 to 3-N and a second data terminal 4-1 to 4-N, 5-1 to 5-N is connected to the metaverse platform 1A, the server device 1 performs a process of inviting the second user to a digital real space (a copy of the real space with the same coordinate values ​​as the first user is located) reproduced on the metaverse platform 1A.

[0020] The first data terminals 3-1 to 3-N are equipped with hardware resources as shown in FIG. 3, and similarly, the second data terminals 4-1 to 4-N, 5-1 to 5-N are equipped with hardware resources as shown in FIG. 4, and are connected to the metaverse platform 1A via the network 21 to receive communication services from the server device 1.

[0021] In the communication system shown in this embodiment, the server device 100 functions as a server device for reproducing a digital twin, but the server device 1 may also be configured to have the functions of the server device 100.

[0022] By making the server device 100 independent from the server device 1 of this system, the server device 1 has the advantage of not having to bear the burden of massive data update processing (data update processing when the landscape changes due to reconstruction, urban redevelopment, etc.) and data maintenance work processing that the server device 100 originally performs.

[0023] FIG. 2 is a block diagram for explaining the configuration of the server device 1 shown in FIG. 1, and the same components as those in FIG. 1 are given the same reference numerals.

[0024] In FIG. 2, reference numeral 11 denotes a communication unit which controls communication processing for connecting first data terminals 3-1 to 3-N, second data terminals 4-1 to 4-N, 5-1 to 5-N connected to a network 21 as a communication medium, to a server device 100 which acquires image information for reproducing a specified digital real space.

[0025] Reference numeral 13 denotes a CPU, which starts an operating system (OS) stored in an external memory 18 connected to the internal bus 12, and starts various applications installed through the API. It also executes the processing of various programs deployed on the RAM 16. Reference numeral 14 denotes a database, which stores prototype models of avatars used to make a virtual space appear, various items to be placed in the virtual space, and also stores account information for a first user who operates the first data terminals 3-1 to 3-N and a second user who operates the second data terminals 4-1 to 4-N, 5-1 to 5-N to connect to the metaverse platform 1A.

[0026] Reference numeral 15 denotes a keyboard, which is used to input numbers, text, and icon commands for running applications.

[0027] Reference numeral 16 denotes an expandable RAM, which expands and stores the basic program of the metaverse platform 1A installed in the external memory 18. Reference numeral 17 denotes a display, which displays the program's running status via various UI screens (not shown).

[0028] The basic program in this embodiment includes an appearance unit 16-1 that acquires current location information of the first user and makes the real space appear as a digital real space on the virtual space platform, an invitation unit 16-2 that invites a second user designated by the first user as a second avatar into the digital real space that has appeared on the metaverse platform 1A by the appearance unit 16-1, and an avatar control unit 16-3 that controls the display of the second avatar on the second data terminals 4-1 to 4-N, 5-1 to 5-N at a virtual space coordinate position that is coordinate-converted into the digital real space based on the real space coordinate position specified by the first user acquired from the first data terminals 3-1 to 3-N. These programs are deployed on RAM 16 and executed by CPU 13.

[0029] As shown in the flowchart described below, the avatar control unit 16-3 is configured to control the display of the second avatar on the first data terminals 3-1 to 3-N at a real space coordinate position where the virtual space coordinate position at which the second avatar is displayed is converted into real space coordinates.

[0030] 3(a), the first HMD 322 includes a first audio processor 322A and a display controller 322B that converts audio information uttered by a first user into an audio signal and transmits it to the server device 1, and also converts audio information uttered by a second user received from the server device 1 into an audio signal and outputs it from a speaker. In this embodiment, the first audio processor 322A performs functional processing as a microphone and functional processing as a speaker.

[0031] The first HMD 322 is configured to be capable of displaying various types of information on a display device (not shown) under the control of a display control unit 322B.

[0032] In addition, the avatar control unit 16-3 controls the display of the second avatar on the first HMD 322 operated by the first user by corresponding the virtual space coordinates at which the second avatar is displayed to a real space coordinate position converted into the real space coordinates.

[0033] Similarly, the avatar control unit 16-3 controls the display of the first avatar on the second HMD 323 operated by the second user by corresponding the real space coordinates in which the first user exists to virtual space coordinate positions converted into virtual space coordinates.

[0034] In addition, the appearance unit 16-1 acquires GPS information indicating the movement state of the first user in the real space from the first data terminals 3-1 to 3-N, and performs a process of continuously making a digital real space that is adapted to the movement state appear on the metaverse platform 1A.

[0035] As a result, the display of the second avatar is controlled so that the avatar of the second user also moves in the digital real space in accordance with the movement of the first user shown in FIG. 11, which will be described later.

[0036] Furthermore, in the server device 1, the appearance unit 16-1 is configured to make a real space corresponding to the current location of the first user appear on the metaverse platform 1A by utilizing predetermined limited provision data linked to the GPS information provided by the server device 100.

[0037] Here, the specified limited provision data is three-dimensional high-precision map data, and is composed of limited provision data that has been accumulated as a three-dimensional still image of a specific building and its surrounding area by identifying GPS information.

[0038] In addition, image information including buildings, roads, etc. that represent real space is managed by GPS mapping, and is configured to be able to be linked with map information obtained from a server device 100 that provides map information connected to the network 21.

[0039] 19 is the AI ​​support unit, which supports the massive spatial modeling process to support the process of making the virtual space appear so that the virtual space moves seamlessly, even when a virtual space movement occurs in which the first virtual space is left and a second virtual space is entered.

[0040] Fig. 3 is a block diagram for explaining the configuration of the first data terminals 3-1 to 3-N connected to the server device 1 shown in Fig. 1. Note that the first data terminals 3-1 to 3-N are assumed to be pad devices, PC devices, smartphones, etc. Note that the HMD (Head Mounted Display (HMD)) is assumed to be configured to perform communication processing with the first data terminals 3-1 to 3-N via WIFI or Bluetooth (registered trademark), but if the HMD has a wireless communication function for a data terminal, it is also possible for the HMD alone to constitute a data terminal.

[0041] Furthermore, this system may be configured so that an HMD is essential, or may be configured as a system that links with the screen of a data terminal instead of an HMD.

[0042] 3(a), reference numeral 301 denotes a CPU, which executes various applications by loading an OS and a control program stored in a ROM 302 into a RAM 303 and executing them. Reference numeral 304 denotes a communication unit, which controls communication for connecting to a server device 1 connected to a network 21. Reference numeral 310 denotes a keyboard, which is configured to be able to input text onto a screen displayed on a display device (not shown) in cooperation with a pointing device (not shown) and to be able to select tabs and icons on the displayed screen.

[0043] 3(a), the first HMD 322 includes a first audio processor 322A and a display controller 322B that converts audio information uttered by a first user into an audio signal and transmits it to the server device 1, and also converts audio information uttered by a second user received from the server device 1 into an audio signal and outputs it from a speaker. In this embodiment, the first audio processor 322A performs functional processing as a microphone and functional processing as a speaker.

[0044] The first HMD 322 is configured to be capable of displaying various types of information on a display device (not shown) under the control of a display control unit 322B.

[0045] In the RAM 303 shown in (b) of Figure 3, 303-1 is a UI control unit that executes control to display a UI screen presented from the metaverse platform 1A provided by the server device 1 shown in Figure 1 on a display unit equipped in the first HMD 322 or the first data terminals 3-1 to 3-N.

[0046] The CPU 301 manages account information (user ID, password) registered by a user in the metaverse platform 1A that the server device 1 makes public on the Web, by using an external storage device such as an SSD (not shown).

[0047] Fig. 4 is a block diagram for explaining the configuration of second data terminals 4-1 to 4-N and 5-1 to 5-N connected to server device 1 shown in Fig. 1. Note that the second data terminals 4-1 to 4-N and 5-1 to 5-N are assumed to be pad devices, PC devices, smartphones, etc.

[0048] The second HMD 323 is assumed to be configured to perform communication processing with the second data terminals 4-1 to 4-N, 5-1 to 5-N via WIFI or Bluetooth (registered trademark). However, if the second HMD 323 has wireless communication capabilities with the second data terminals 4-1 to 4-N, 5-1 to 5-N, it is also possible for the HMD alone to function as a data terminal.

[0049] 4(a), 301 denotes a CPU, which executes various applications by loading an OS and a control program stored in a ROM 302 into a RAM 303 and executing them. Input of various information and clicking of icons displayed on a user interface screen (UI screen) are performed directly on the display units provided in the second data terminals 4-1 to 4-N and 5-1 to 5-N. 304 denotes a communication unit, which controls communication for connecting to a server device 1 connected to a network 21.

[0050] 4(a), the second HMD 323 includes a second audio processor 323A and a display controller 323B that converts audio information uttered by a second user into an audio signal and transmits it to the server device 1, and also converts audio information uttered by a first user received from the server device 1 into an audio signal and outputs it from a speaker. In this embodiment, the second audio processor 323A performs functional processing as a microphone and functional processing as a speaker.

[0051] The second HMD 323 is configured to be capable of displaying various types of information on a display device (not shown) under the control of a display control unit 323B.

[0052] This enables the first user and the second user to have simultaneous two-way voice communication with each other on the metaverse platform 1A via the first HMD 322 and the second HMD 323.

[0053] In the RAM 303 shown in (b) of Figure 4, 303-1 is a UI control unit that executes control to display a UI screen presented from the metaverse platform 1A provided by the server device 1 shown in Figure 1 on the display unit equipped in the second HMD 323 or the second data terminals 4-1 to 4-N, 5-1 to 5-N.

[0054] The CPU 301 manages account information (user ID, password) registered by a user in the metaverse platform 1A that the server device 1 makes public on the Web, by using an external storage device such as an SSD (not shown).

[0055] FIG. 5 is a schematic diagram showing the display mode of the digital real space provided by the communication system showing this embodiment. In this example, a first user 51B is located in a real space 51, for example in the middle of a city, wearing a first HMD 322 and looking at buildings. The first user 51B invites a second avatar 52C of a second user to the digital real space 52, and transmits GPS information corresponding to the current location to the server device 1 to obtain from the server device 100 specific limited provision data that reproduces the digital real space 52 in the real space. This allows the first user 51B and the second avatar 52C of the second user to be close to each other in a single digital real space 52 as if accompanying the first user 51B to his / her current location, providing the second user with the experience of sharing the real space 51 that the first user is paying attention to. Furthermore, the first user operating the first HMD 322 in the real space 51 can visually recognize the second avatar 51C of the second user on the screen of the first HMD 322. Furthermore, in the digital reality space 52, the second user can visually recognize a first avatar 52B corresponding to the first user 51B and a second avatar 52C representing the second user on the screen of the second HMD 323. Note that specific display control of the first avatar 52B and the second avatar 52C will be described with reference to a flowchart described later. In FIG. 5, 51A is the real building viewed by the first user 51B, and the second user can view object 52A (a digital copy of building 51A) displayed in digital real space 52 as a digital twin of building 51A on the screen of the second HMD 323.

[0056] As a result, as shown in FIG. 5, the second HMD 323 displays a first avatar 52B corresponding to the first user 51B and a second avatar 52C corresponding to the second user.

[0057] In addition, the first HMD 322 controls the manner of avatar display (including combinations of color, shape, display, lighting, and blinking) so as to display a second avatar 51C corresponding to a second user and to identify a first avatar 52B corresponding to a first user.

[0058] 6 is a flowchart showing an example of data processing by the first data terminals 3-1 to 3-N in the communication system according to the present embodiment. Note that (1) to (6) indicate steps, and each step is realized by the CPU 301 loading a program stored in an external memory (not shown) onto the RAM 303 and executing the program.

[0059] After the first user turns on the power of the first data terminal 3-1 to 3-N, the CPU 301 confirms that the first HMD 322 is running (1). The first HMD 322 is equipped with a GPS chip (not shown). The GPS chip is a chip incorporating a circuit for receiving signals from GPS satellites and calculating the current location. The CPU 301 obtains location information corresponding to the first user's current location and transmits it to the server device 1 to transfer the location information (2).

[0060] Next, the first user selects a user ID corresponding to the second user whom he / she wishes to invite into real space while referring to a list of second users registered in the internal memory of the first data terminals 3-1 to 3-N on a screen that is not displayed on the first data terminals 3-1 to 3-N (3).

[0061] Next, the CPU 301 transmits the user ID of the second user selected by the first user to be invited and the location information of the first user (current location (specified from a GPS signal)) to the server device 1 (4).

[0062] Thereafter, the first data terminals 3-1 to 3-N receive digital real-space position information in which the virtual-space coordinate values ​​of the second user are converted into latitude and longitude in the real space from the server device 1 via the network 21 (5). The CPU 301 then displays a second avatar corresponding to the second user on the screen of the first HMD 322 at the latitude and longitude position identified in the received real space of the second user (6), and terminates the processing.

[0063] In this way, a first user operating a first data terminal 3-1 to 3-N and a second user operating a second data terminal 4-1 to 4-N, 5-1 to 5-N can view the digital real space displayed on the metaverse platform 1A provided by the server device 1, which represents the real space viewed at the first user's current location.

[0064] At this time, a second avatar representing the second user is displayed in an invited manner on the screen of the first HMD 322 operated by the first user, and on the screen of the second HMD 323 operated by the second user, the real space corresponding to the first user's current location can be experienced in a virtual space.

[0065] Thereafter, CPU 301 enables the first user to experience walking around the real space or accompanying the second avatar as part of a sightseeing trip while facing the second avatar in the real space until the first user logs out from metaverse platform 1A.

[0066] 7 is a flowchart showing an example of data processing by the server device 1 in the communication system according to the present embodiment. Note that (11) to (16) indicate each step, and each step is realized by the CPU 13 loading a program stored in the external memory 18 onto the RAM 16 and executing the program.

[0067] First, CPU 13 acquires user information (user name, virtual space avatar, current location information) identifying the first user transmitted from a first data terminal 3-1 to 3-N operated by the first user via network 21 (11). Then, CPU 13 refers to a user ID corresponding to a second user invited from a second data terminal 4-1 to 4-N, 5-1 to 5-N (12), and executes a process of converting the current location information (indicating longitude and latitude) of the first user into virtual space coordinates viewed by the second user (13).

[0068] Next, the CPU 13 transmits a notification (including the virtual space coordinates and the platform coordinates) to the second data terminals 4-1 to 4-N and 5-1 to 5-N that the second user has been invited to the digital real space, based on the user ID that identifies the second user (14).

[0069] Next, CPU 13 executes a process of converting the virtual space coordinates where the second avatar corresponding to the second user operating second data terminal 4-1-4-N, 5-1-5-N is located into latitude and longitude indicating the real space coordinates where the first user operating first data terminal 3-1-3-N is located (15). Next, CPU 13 transmits the second avatar data and information indicating the converted latitude and longitude to first data terminal 3-1-3-N (16), and ends the process.

[0070] 8 is a flowchart showing an example of data processing by the second data terminals 4-1 to 4-N and 5-1 to 5-N in the communication system according to the present embodiment. Note that (21) to (23) indicate each step, which is realized by the CPU 301 loading a program stored in an external memory (not shown) onto the RAM 303 and executing it.

[0071] First, when the CPU 301 of the second data terminal 4-1 to 4-N, 5-1 to 5-N receives a notification of an invitation to real space from the first user from the server device 1 (21), the CPU 301 of the second data terminal 4-1 to 4-N, 5-1 to 5-N generates a second avatar representing the second user at coordinates indicating the real space where the first user is located (22).

[0072] Next, CPU 301 of second data terminals 4-1 to 4-N, 5-1 to 5-N transmits second avatar data for visualizing a second avatar corresponding to the second user and the coordinates of the second avatar to be displayed in the virtual space to server device 1 (23), and ends the process.

[0073] 9 to 11 are diagrams for explaining a manner in which a second user is invited and displayed as a second avatar on the first data terminals 3-1 to 3-N in the communication system according to the present embodiment. ST1 to ST3 indicate the timing on the first user's side, and ST11 to ST13 indicate the timing on the second user's side. The star displayed on the first data terminals 3-1 to 3-N shown in FIG. 10 indicates the invitation position designated by the first user to invite the second avatar corresponding to the second user.

[0074] [Timing ST1 (real space) vs. Timing ST11 (digital real space)] As shown in FIG. 9, a first HMD 322 is prepared on the first user side and is connectable to first data terminals 3-1 to 3-N, and a second HMD 323 is prepared on the second user side and is connectable to second data terminals 4-1 to 4-N, 5-1 to 5-N.

[0075] [Timing ST2 (real world) vs. Timing ST12 (digital real world)] 9, a first user U1 is walking around town wearing a first HMD 322 after a first HMD 322 is prepared and connected to first data terminals 3-1 to 3-N on the first user side. Here, the first user U1 wants to invite a second user U2 into real space.

[0076] [Timing ST3 (real world) vs. Timing ST13 (digital real world)] Therefore, as shown in FIG. 10, a first user U1 has designated coordinates (star shape in the figure) for inviting a second avatar corresponding to a second user U2 into real space.

[0077] The specified coordinates of the real space (position information (latitude, longitude) acquired from the GPS) are transmitted from the first data terminals 3-1 to 3-N to the server device 1, and are converted into virtual space coordinates by the CPU 13 of the server device 1 in step (13) shown in FIG. 7.

[0078] This converts the coordinate values ​​into virtual reality space (digital reality space) as a digital twin of the real space, and the second user U2 is displayed with a second avatar AV2 aligned with the invitation position specified by the first user U1, as shown in Figure 10.

[0079] Furthermore, when the first user U1 moves on the road in the direction of the arrow (timing ST4), the second avatar AV2 corresponding to the second user in the digital virtual space (timing ST14) also moves in the direction of the arrow in response to the movement, as shown in FIG.

[0080] [Effects of the first embodiment] According to this embodiment, in the digital twin service provided by the server device 1, specific users can be invited to the same real space, and while linking both the real space and the virtual space, remote users can communicate with each other by exchanging opinions and impressions. This makes it possible to significantly increase the possibilities and scalability of communication services that use digital twin technology.

[0081] Second Embodiment In the above embodiment, a system in which a first user and a second user communicate one-to-one was described, but the present invention can also be applied to a system in which a first user invites multiple second users and simultaneously communicates one-to-N.

[0082] The present invention can also be realized by a process in which a program for implementing one or more of the functions of the above-described embodiments is supplied to a system or device via a network or a storage medium, and one or more processors in a computer of the system or device read and execute the program. The present invention can also be realized by a circuit (e.g., ASIC) for implementing one or more of the functions.

[0083] Specifically, the present invention includes a server device providing a digital twin service that reproduces a real space as a digital real space via a virtual space platform, a first data terminal operated by a first user who connects to the virtual space platform, and a second data terminal operated by a second user who connects to the virtual space platform, and a server computer in the communication system in which the server device and the first data terminal communicate with each other through a predetermined communication medium, the server computer acquiring current location information of the first user and making the real space appear as a digital real space on the virtual space platform, and a display step of displaying the digital real space that has appeared on the virtual space platform in the display step. and an avatar control step of controlling the display of the second avatar on the first data terminal at a virtual space coordinate position that is coordinate converted into the digital real space based on a real space coordinate position specified by the first user obtained from the first data terminal. The program is configured to execute the steps of: inviting the second user designated by the first user as a second avatar; and an avatar control step of controlling the display of the second avatar on the second data terminal at a virtual space coordinate position that is coordinate converted into the digital real space based on a real space coordinate position specified by the first user obtained from the first data terminal. In this system, we have explained the case where the virtual reality space is displayed on a screen using an HMD, but the same effect can be expected with a configuration that uses intelligent glasses (XR type).

[0084] The disclosure regarding the present invention described above can be summarized at least as follows.

[0085] (1) A communication system in which a server device providing a digital twin service that reproduces a real space as a digital real space via a virtual space platform via a predetermined communication medium, a first data terminal operated by a first user connected to the virtual space platform, and a second data terminal operated by a second user connected to the virtual space platform communicate with each other via a predetermined communication medium, wherein the server device is characterized in that when providing a digital twin in which the real space coordinates where the first user exists are replaced with virtual space coordinates to the second data terminal on the virtual space platform, the server device is equipped with an avatar control means that controls avatar display so that a second avatar corresponding to the second user invited by the first user appears on the virtual space platform at a virtual space coordinate position that corresponds to the real space coordinate position specified by the first user.

[0086] (2) The avatar control means is characterized in that it controls the display of the second avatar on a first HMD operated by the first user by corresponding the virtual space coordinates at which the second avatar is displayed to a real space coordinate position converted into the real space coordinates.

[0087] (3) The avatar control means is characterized in that it controls the display of a first avatar on a second HMD operated by the second user by corresponding real space coordinates in which the first user exists to virtual space coordinate positions converted into the virtual space coordinates.

[0088] (4) The first HMD is characterized in that it is configured so that the first user and the second user can communicate with each other on the virtual space platform, and is equipped with a first audio processing unit that converts the audio information spoken by the first user into an audio signal and transmits it to the server device, and converts the audio information spoken by the second user received from the server device into an audio signal and outputs it from a speaker; and the second HMD is equipped with a second audio processing unit that converts the audio information spoken by the second user into an audio signal and transmits it to the server device, and converts the audio information spoken by the first user received from the server device into an audio signal and outputs it from a speaker.

[0089] (5) The avatar control means is configured to acquire GPS information indicating the first user's current location in the real space coordinates and to make a digital real space appear on the virtual space platform.

[0090] (6) The avatar control means is configured to acquire GPS information indicating the movement state of the first user in the real space, and continuously cause a digital real space adapted to the movement state to appear on the virtual space platform.

[0091] (7) A display control method for a communication system in which a server device providing a digital twin service that reproduces a real space as a digital real space via a virtual space platform via a predetermined communication medium, a first data terminal operated by a first user connected to the virtual space platform, and a second data terminal operated by a second user connected to the virtual space platform communicate with each other, the server device comprising: an appearance step of acquiring current location information of the first user and making the real space appear on the virtual space platform as a digital real space; an invitation step of inviting the second user designated by the first user as a second avatar into the digital real space that appeared on the virtual space platform in the appearance step; and an avatar control step of controlling the display of the second avatar on the second data terminal at a virtual space coordinate position that is coordinate converted to the digital real space based on the real space coordinate position designated by the first user acquired from the first data terminal.

[0092] (8) The avatar control step is characterized in that it controls the display of the second avatar on the first data terminal at a real space coordinate position where the virtual space coordinate position at which the second avatar is displayed is converted into real space coordinates.

[0093] (9) The avatar control step is characterized in that a second avatar corresponding to the second user is displayed on a first HMD operated by the first user.

[0094] (10) A program that causes a server computer in a communication system in which a server device that provides a digital twin service that reproduces a real space as a digital real space via a virtual space platform via a predetermined communication medium, a first data terminal operated by a first user that connects to the virtual space platform, and a second data terminal operated by a second user that connects to the virtual space platform communicate with each other, to execute the following steps: an appearance step of acquiring current location information of the first user and making the real space appear on the virtual space platform as a digital real space; an invitation step of inviting the second user designated by the first user as a second avatar into the digital real space that appeared on the virtual space platform in the appearance step; and an avatar control step of controlling the second avatar to be displayed on the second data terminal at a virtual space coordinate position that is coordinate-converted into the digital real space based on a real space coordinate position designated by the first user acquired from the first data terminal, and controlling the second avatar to be displayed on the first data terminal at a real space coordinate position that is coordinate-converted from the virtual space coordinate position at which the second avatar is displayed. [Industrial Applicability]

[0095] In the above embodiment, an example was described in which the first user and the second user were displayed as avatars. However, by inputting the physical characteristic data of the first user and the second user, it is also possible to configure the system to display a 3D modeled object in place of the avatar, allowing the user to appear in the virtual space as a My Figure with physical characteristics, creating a system that gives a sense of familiarity.

[0096] Furthermore, by controlling the movements of the My Figure so that parts of the figure, such as the arms, wrists, and fingers, move in three dimensions, it is also possible to create a system in which each user can wear sensors that detect body pressure and warmth and hold hands in the virtual space, even recreating the feel of their hands. [Explanation of symbols]

[0097] 1. Server device 1A Metaverse Platform 3-1~3-N 1st data terminal 4-1~4-N Second Data Terminal 5-1~5-N Second Data Terminal 100 Server device

Claims

1. A communication system in which a server device providing a digital twin service that reproduces a real space as a digital real space via a virtual space platform, a first data terminal operated by a first user who connects to the virtual space platform, another server device that provides predetermined limited-provision data to make a digital real space appear and cooperates with the server device, and a second data terminal operated by a second user who connects to the virtual space platform communicate with each other via a predetermined communication medium, The predetermined limited provision data is three-dimensional high-precision map data, and is composed of a plurality of still images that show a specific building and its surrounding area in three dimensions by specifying GPS information; The server device includes: an avatar control means for controlling avatar display so that a second avatar corresponding to the second user invited by the first user appears on the virtual space platform at a virtual space coordinate position corresponding to the real space coordinate position designated by the first user when a digital twin in which real space coordinates where the first user exists are replaced with virtual space coordinates is provided to the second data terminal on the virtual space platform; The avatar control means includes: The GPS information indicating the current position of the first user in the real space coordinates is acquired, and a digital real space is generated on the virtual space platform; and A communication system characterized by being configured to acquire GPS information indicating the movement state of the first user in the real space, and continuously cause a digital real space adapted to the movement state to appear on the virtual space platform.

2. The avatar control means includes: The communication system described in claim 1, characterized in that the virtual space coordinates at which the second avatar is displayed are matched to real space coordinate positions converted into real space coordinates, and the second avatar is controlled to be displayed on a first HMD operated by the first user.

3. The avatar control means includes: The communication system described in claim 2, characterized in that the real space coordinates in which the first user exists are associated with a virtual space coordinate position converted into the virtual space coordinates, and a first avatar is displayed on a second HMD operated by the second user.

4. The first HMD includes: a first voice processing unit that converts voice information uttered by the first user into a voice signal and transmits the voice signal to the server device, and converts voice information uttered by the second user received from the server device into a voice signal and outputs the voice signal from a speaker; The second HMD includes: a second voice processing unit that converts voice information uttered by the second user into a voice signal and transmits the voice signal to a server device, and converts voice information uttered by the first user received from the server device into a voice signal and outputs the voice signal from a speaker; 4. The communication system according to claim 3, wherein the first user and the second user are configured to be able to communicate with each other on the virtual space platform.

5. A communication system in which a server device providing a digital twin service that reproduces a real space as a digital real space via a virtual space platform via a specified communication medium, a first data terminal operated by a first user connected to the virtual space platform, another server device that provides specified limited-provision data to make a digital real space appear and cooperates with the server device, and a second data terminal operated by a second user connected to the virtual space platform communicate with each other via a specified communication medium, The predetermined limited provision data is three-dimensional high-precision map data, and is composed of a plurality of still images that show a specific building and its surrounding area in three dimensions by specifying GPS information; The server device includes: an avatar control means for controlling avatar display so that a second avatar corresponding to the second user invited by the first user appears on the virtual space platform at a virtual space coordinate position corresponding to the real space coordinate position designated by the first user when a digital twin in which real space coordinates where the first user exists are replaced with virtual space coordinates is provided to the second data terminal on the virtual space platform; The avatar control means includes: The GPS information indicating the current position of the first user in the real space coordinates is acquired, and a digital real space is generated on the virtual space platform; and A communication system configured to acquire the GPS information indicating a moving state of the first user in the real space and continuously generate a digital real space adapted to the moving state on the virtual space platform, A display control method for a communication system in which a server device that provides a digital twin service that reproduces a real space as a digital real space via a virtual space platform, a first data terminal operated by a first user that connects to the virtual space platform, and a second data terminal operated by a second user that connects to the virtual space platform communicate with each other via a predetermined communication medium, comprising: The server device includes: a step of acquiring current location information of the first user and making a real space appear as a digital real space on the virtual space platform; an inviting step of inviting the second user designated by the first user as a second avatar into the digital reality space that has appeared on the virtual space platform in the appearing step; an avatar control step of controlling the second avatar to be displayed on the second data terminal at a virtual space coordinate position that is converted into the digital real space based on a real space coordinate position designated by the first user and acquired from the first data terminal; A display control method for a communication system comprising:

6. The avatar control step includes: A display control method for a communication system as described in claim 5, characterized in that the second avatar is displayed on the first data terminal at a real space coordinate position where the virtual space coordinate position at which the second avatar is displayed is converted into real space coordinates.

7. The display control method for a communication system according to claim 5 or 6, characterized in that the avatar control step comprises displaying the second avatar corresponding to the second user on a first HMD operated by the first user.

8. A communication system in which a server device providing a digital twin service that reproduces a real space as a digital real space via a virtual space platform via a specified communication medium, a first data terminal operated by a first user connected to the virtual space platform, another server device that provides specified limited-provision data to make a digital real space appear and cooperates with the server device, and a second data terminal operated by a second user connected to the virtual space platform communicate with each other via a specified communication medium, The predetermined limited provision data is three-dimensional high-precision map data, and is composed of a plurality of still images that show a specific building and its surrounding area in three dimensions by specifying GPS information; The server device includes: an avatar control means for controlling avatar display so that a second avatar corresponding to the second user invited by the first user appears on the virtual space platform at a virtual space coordinate position corresponding to the real space coordinate position designated by the first user when a digital twin in which real space coordinates where the first user exists are replaced with virtual space coordinates is provided to the second data terminal on the virtual space platform; The avatar control means includes: The GPS information indicating the current position of the first user in the real space coordinates is acquired, and a digital real space is generated on the virtual space platform; and A communication system configured to acquire the GPS information indicating a moving state of the first user in the real space and continuously generate a digital real space adapted to the moving state on the virtual space platform, A server device that provides a digital twin service that reproduces a real space as a digital real space via a virtual space platform, a first data terminal operated by a first user that connects to the virtual space platform, and a second data terminal operated by a second user that connects to the virtual space platform communicate with a server computer in a communication system in which the server device and the second data terminal communicate with each other via a predetermined communication medium, a step of acquiring current location information of the first user and making a real space appear as a digital real space on the virtual space platform; an inviting step of inviting the second user designated by the first user as a second avatar into the digital reality space that has appeared on the virtual space platform in the appearing step; a control step of controlling the display of the second avatar on the second data terminal at a virtual space coordinate position that is coordinate converted into the digital real space based on a real space coordinate position specified by the first user obtained from the first data terminal, and of controlling the display of the second avatar on the first data terminal at a real space coordinate position that is converted from the virtual space coordinate position at which the second avatar is displayed to real space coordinates.

Images