Information processing device, program, and information processing method

Abstract

We provide information processing devices that allow the first user to experience the real-world space they inhabit within a digital twin space. [Solution] An information processing device 2 used by a second user communicating with a terminal 1 used by a first user, comprising: a location information acquisition unit 21 that acquires location information 11a transmitted by terminal 1, including the current time of terminal 1, the latitude and longitude of terminal 1 at the current time, and the altitude; a space generation unit 22 that generates a digital twin space of a point that reproduces the surrounding environment of the real space of the point at the current time's latitude, longitude, and altitude; and a display processing unit 23 that displays the digital twin space generated by the space generation unit 22 on a display unit.

Description

Technical Field

[0001] The present disclosure relates to a technique for displaying a digital twin spatial image.

Background Art

[0002] A digital twin is a technology that digitally reproduces something in the real world as if it were a twin, or a technology that uses data collected from the real-world environment to reproduce the same environment on a virtual space as if it were a twin. Hereinafter, a 3D spatial image that reproduces the real space is referred to as a digital twin spatial image.

[0003] For example, Patent Document 1 discloses a technique that enables a user to more easily grasp or experience an event in the real space by displaying a reproduction image that visually reproduces the past situation of a certain location.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0005] However, the technique disclosed in Patent Document 1 merely grasps or experiences the past situation with a reproduction image.

[0006] In one aspect, an object of the present disclosure is to provide an information processing apparatus and the like that enables a first user to experience the real space on a digital twin space.

Means for Solving the Problems

[0007] (1) The information processing device disclosed herein is an information processing device used by a second user communicating with a terminal used by a first user, and is characterized by comprising: a location information acquisition unit that acquires location information transmitted by the terminal, including the current time of the terminal, the latitude and longitude of the terminal at the current time, and the altitude; a space generation unit that generates a digital twin spatial image that reproduces the real space at the latitude and longitude and altitude included in the location information, and the surrounding environment of the real space at the current time included in the location information; and a display control unit that displays the digital twin spatial image generated by the space generation unit on a display unit.

[0008] • "First user" refers to the communication partner of the second user who uses the information processing device.

[0009] "Surrounding environment" refers to the surrounding environment that changes over time, such as sound, weather, brightness, and the position of the sun or moon.

[0010] A "digital twin spatial image" is a 3D spatial image that reproduces real space, and is generated based on 3D model data, which is data that creates a three-dimensional model of real space including objects.

[0011] (2) In addition, such an information processing device may include: a voice information transmitting / receiving unit that receives first voice information including the voice of the first user transmitted by the terminal and transmits second voice information including the voice of the second user; a voice acquisition unit that acquires the voice of the second user and generates second voice information; and a voice output unit that outputs the voice of the first user.

[0012] "Voice" refers to sounds such as the voices of the first user and the second user during a phone call. The "sound acquisition unit" is, for example, a microphone. The "audio output unit" is, for example, a speaker.

[0013] (3) Here, the surrounding environment includes meteorological conditions including at least one of cloud cover, snow, thunderstorms, and rain, and rain may include heavy rainfall or light rainfall.

[0014] (4) Furthermore, if the surrounding environment includes rain or thunder, the digital twin spatial image may include the sound of rain or thunder.

[0015] (5) The surrounding environment may also include brightness, the position of the sun, or the position of the moon.

[0016] (6) In addition, in such an information processing device, the location information acquisition unit acquires location information transmitted by the terminal while the first user is moving during a predetermined time, the space generation unit generates digital twin space images for each current time that reproduce the real space at each of the latitude, longitude, and altitude included in each location information acquired during the predetermined time, and the surrounding environment of each of the real spaces at each current time included in each location information, and the display control unit may cause the display unit to display the digital twin space images for each current time generated by the space generation unit.

[0017] (7) In addition, in such an information processing device, the space generation unit may further generate a digital twin spatial image that reproduces the real space of an overhead viewpoint, which is at an altitude higher than the latitude and longitude at the current time and the altitude, and at an altitude from which the destination to which the first user is moving can be viewed from above, and the display control unit may cause the display unit to display the digital twin spatial image of the overhead viewpoint generated by the space generation unit.

[0018] (8) The program of the present disclosure is a program for an information processing apparatus used by a second user communicating with a terminal used by a first user, the program causing a computer to execute: a position information acquisition step of acquiring position information transmitted by the terminal, the position information including the current time of the terminal, the latitude and longitude of the terminal at the current time, and the altitude; a space generation step of generating a digital twin space image that reproduces the real space at the latitude and longitude and altitude included in the position information and the surrounding environment of the real space at the current time included in the position information; and a display control step of causing a display unit to display the digital twin space image generated by the space generation unit.

[0019] (9) The information processing method of the present disclosure is an information processing method for an information processing apparatus used by a second user communicating with a terminal used by a first user, the method including: a position information acquisition step of acquiring position information transmitted by the terminal, the position information including the current time of the terminal, the latitude and longitude of the terminal at the current time, and the altitude; a space generation step of generating a digital twin space image that reproduces the real space at the latitude and longitude and altitude included in the position information and the surrounding environment of the real space at the current time included in the position information; and a display control step of causing a display unit to display the digital twin space image generated by the space generation unit.

Advantages of the Invention

[0020] The information processing apparatus and the like of the present disclosure can enable a first user to experience the real space on a digital twin space.

Brief Description of the Drawings

[0021] [Figure 1] It is a schematic diagram showing an outline of an information processing system according to an embodiment. [Figure 2] It is a functional block diagram showing a terminal and an information processing apparatus according to an embodiment. [Figure 3] It is a schematic diagram showing an example of a screen displayed on a display unit of an information processing apparatus. [Figure 4]It is a schematic diagram showing an example of a screen displayed on a display unit of an information processing apparatus. [Figure 5] It is a functional block diagram showing a server of an embodiment. [Figure 6] It is a schematic diagram showing an example of the hardware configuration of an information processing apparatus. [Figure 7] It is a schematic diagram showing an example of the hardware configuration of a terminal. [Figure 8] It is a schematic flow showing an example of the operation of an information processing apparatus.

Embodiments of the Invention

[0022] Hereinafter, an information processing apparatus and the like according to an aspect of the present disclosure will be specifically described with reference to the drawings. Note that each of the embodiments described below shows a specific example of the present disclosure. Numerical values, shapes, components, arrangement positions of components, etc. shown in the following embodiments are merely examples and are not intended to limit the present disclosure. Also, among the components in the following embodiments, components not described in the independent claims indicating the most general concept are described as optional components. In all embodiments, the respective contents can also be combined.

[0023] Also, the following drawings are schematic, and for convenience of explanation, some configurations may be omitted. Also, the same reference numerals are given to common parts in one or more embodiments, and the description may be omitted.

[0024] [1. General Explanation] (Information Processing System) FIG. 1 is a schematic diagram showing an overview of an information processing system according to an embodiment.

[0025] First, an overview of the information processing system of the embodiment will be explained using Figure 1. The information processing system shown in Figure 1 allows a second user 51 of one or more information processing devices 2, which are communicating with a terminal 1 used by a first user 50, to experience the real space in which the first user 50 currently exists, in a digital twin space. The information processing system shown in Figure 1 comprises a terminal 1, one or more information processing devices 2, and a server 4. The terminal 1, the one or more information processing devices 2, and the server 4 are connected to each other via a communication network 3.

[0026] (Terminal 1) Terminal 1 is a terminal used by the first user 50 and communicates with the information processing device 2. Terminal 1 can obtain its current latitude, longitude, and altitude using GPS (Global Positioning System) signals, GNSS (Global Navigation Satellite System) signals, etc. Terminal 1 can transmit the obtained latitude, longitude, altitude, and current time as location information to the information processing device 2. In addition, Terminal 1 can support conversations between the first user and the second user by transmitting the first user's voice (voice information) and receiving and outputting the second user's voice (voice information).

[0027] (Information Processing Device 2) Each of the one or more information processing devices 2 communicates with the terminal 1 used by the first user 50 and acquires location information from the terminal 1. Each information processing device 2 can generate a digital twin spatial image that reproduces the surrounding environment of the real space in which the first user 50 is currently located, based on the acquired location information, and display it on the display unit 201. Alternatively, one or more information processing devices 2 may not generate a digital twin spatial image, but instead acquire a digital twin spatial image generated by the server 4 and display it on the display unit 201. Furthermore, one or more information processing devices 2 can support communication between the first user and the second user by outputting the first user's voice and transmitting the second user's voice (voice information) to the terminal 1.

[0028] (Communication Network 3) Communication network 3 is, for example, the internet. Communication network 3 may also be an intranet, or a wired or wireless communication network. Wireless communication may include, for example, Bluetooth®, Wi-Fi®, or LPWA (Low Power Wide Area). As for LPWA, LTE® or frequency bands that do not require a license may be used.

[0029] (Server 4) Server 4 can perform some of the processing performed by Information Processing Unit 2. For example, Server 4 can acquire location information from Terminal 1, generate a digital twin spatial image from the acquired location information, and transmit it to Information Processing Unit 2. Alternatively, Server 4 may acquire location information from Terminal 1 and transmit it to Information Processing Unit 2. Furthermore, Server 4 may transmit parts necessary for Information Processing Unit 2 to generate a digital twin spatial image in response to a request from Information Processing Unit 2.

[0030] [2. Detailed explanation] Figure 2 is a functional block diagram showing the terminal 1 and information processing device 2 of the embodiment. Figures 3 and 4 are schematic diagrams showing an example of a screen displayed on the display unit 201 of the information processing device 2. (Terminal 1) Terminal 1 is a device capable of communicating with the information processing device 2, and is, for example, a smartphone. Alternatively, terminal 1 may be a tablet PC, a personal computer, or the like.

[0031] In this embodiment, as shown in Figure 2, terminal 1 includes a sensor unit 11, a voice acquisition unit 12, a location information transmission unit 13, a storage unit 14, a voice output unit 15, and a voice information transmission / reception unit 16. However, terminal 1 does not necessarily have to include the voice acquisition unit 12, the voice output unit 15, and the voice information transmission / reception unit 16.

[0032] (Sensor unit 11) The sensor unit 11 detects the latitude, longitude, and altitude of terminal 1, for example, using GNSS signals. When using GNSS signals, the sensor unit 11 can detect the latitude, longitude, and altitude of terminal 1 based on radio waves arriving from GNSS satellites such as GPS satellites. Alternatively, the sensor unit 11 may detect the latitude and longitude of terminal 1 using GPS signals and detect the altitude of terminal 1 using means other than GPS signals. Furthermore, the sensor unit 11 may store the detected latitude, longitude, and altitude of terminal 1 at the current time, including the current time, as location information in the storage unit 14.

[0033] (Speech acquisition unit 12) The voice acquisition unit 12 is, for example, a microphone, and acquires the voice of the first user 50 and generates first voice information. In this embodiment, the voice acquisition unit 12 acquires the voice of the first user 50, who is in a call with the second user 51 of the information processing device 2, and converts it into voice information, thereby generating voice information that includes the voice of the first user.

[0034] (Location information transmission unit 13) The location information transmission unit 13 transmits location information, including the current time, along with the latitude, longitude, and altitude of the terminal 1 detected by the sensor unit 11, to the communication network 3. In this embodiment, the location information transmission unit 13 transmits location information 11a, including the latitude, longitude, altitude, and current time of the terminal 1, to the information processing device 2.

[0035] (Storage unit 14) The storage unit 14 is composed of an HDD (Hard Disk Drive), an SDD (Solid State Drive), or memory. In this embodiment, for example, the storage unit 14 stores the latitude, longitude, and altitude of the terminal 1 detected by the sensor unit 11, as well as second voice information including the voice of a second user received by the voice information transmission / reception unit 16.

[0036] (Audio output unit 15) The audio output unit 15 is, for example, a speaker, and outputs the audio information of the second user received by the audio information transmission / reception unit 16. In this embodiment, the audio output unit 15 can output the audio of the second user 51 who is in a call with the first user 50.

[0037] (Voice information transmission / reception unit 16) The voice information transmission / reception unit 16 receives second voice information including the voice of the second user 51 and transmits first voice information including the voice of the first user. In this embodiment, the voice information transmission / reception unit 16 transmits voice information 16a including the voice of the first user and receives voice information 25a including the voice of the second user 51. In this way, the first user and the second user can communicate in both directions.

[0038] (Information Processing Device 2) The information processing device 2 is a device capable of communicating with the terminal 1, such as a personal computer, tablet PC, or smartphone. In this embodiment, as shown in Figure 2, the information processing device 2 includes a location information acquisition unit 21, a space generation unit 22, a display processing unit 23, a storage unit 24, an audio information transmission / reception unit 25, an audio output unit 26, and an audio acquisition unit 27. However, the information processing device 2 does not necessarily have to include the audio information transmission / reception unit 25, the audio output unit 26, and the audio acquisition unit 27.

[0039] (Location information acquisition unit 21) The location information acquisition unit 21 acquires location information transmitted by terminal 1. The location information transmitted by terminal 1 includes the current time of terminal 1, the latitude and longitude of terminal 1 at that time, and its altitude. The current time may be the time when the latitude and longitude and altitude of terminal 1 are detected, or the time when terminal 1 transmits the location information. Since terminal 1 and the information processing device 2 are communicating, the time when the latitude and longitude and altitude of terminal 1 are detected and the time when terminal 1 transmits the location information are only different by a predetermined time (the processing time of terminal 1), and can be handled similarly.

[0040] In this embodiment, the location information acquisition unit 21 acquires location information 11a that includes the current time of terminal 1, which is communicating with the information processing device 2, and the latitude, longitude, and altitude of terminal 1 at that current time.

[0041] (Space generation unit 22) The spatial generation unit 22 generates a digital twin spatial image that reproduces the real space in terms of latitude, longitude, and altitude included in the location information, and the surrounding environment of the real space at the current time included in the location information. More specifically, the spatial generation unit 22 generates a digital twin spatial image of the real space based on 3D model data, which is data that 3D models the real space in terms of latitude, longitude, and altitude included in the location information 11a acquired by the location information acquisition unit 21, and reflects the surrounding environment of the real space at the current time included in the location information 11a into the generated digital twin spatial image. For example, as shown in Figure 3, the spatial generation unit 22 can identify the location 112 on the map 111 from the latitude and longitude included in the location information, for example via the internet, and therefore can generate a digital twin spatial image of the real space at location 112. Similarly, the spatial generation unit 22 can identify the surrounding environment of the real space at the current time included in the location information, for example via the internet, and therefore can reflect the surrounding environment of the real space at the current time into the generated digital twin spatial image.

[0042] The spatial generation unit 22 may obtain parts (components) for generating the digital twin spatial image from the storage unit 14, or from the server 4 via the communication network 3. Alternatively, the spatial generation unit 22 may not generate a digital twin spatial image at all, instead using a digital twin spatial image generated by the server 4. Furthermore, when the spatial generation unit 22 generates a digital twin spatial image for the current time, it may only reflect the difference between this image and the digital twin spatial image for the previous current time into the digital twin spatial image for the previous current time.

[0043] (Surrounding environment) The surrounding environment refers to the environment that changes over time, such as sound, weather, brightness, and the position of the sun or moon. In this embodiment, the surrounding environment includes meteorological conditions, including at least one of cloud cover, snow, thunderstorms, and rainy weather, with rainy weather including cases of heavy rainfall or light rainfall. Furthermore, if the surrounding environment includes rainy weather or thunderstorms, the digital twin spatial image may also include the sound of rain or thunder. In this case, the spatial generation unit 22 generates a digital twin spatial image that reproduces the sound of rainy weather or thunderstorms as the surrounding environment.

[0044] (Display processing unit 23) The display processing unit 23 displays information acquired by the information processing device 2 on the display unit 201. For example, the display processing unit 23 displays the digital twin spatial image generated by the spatial generation unit 22 on the display unit 201.

[0045] The display processing unit 23 may, for example as shown in Figure 3, display map 111 and location 112 on the display unit 201 based on latitude and longitude included in location information, such as via the Internet. The display processing unit 23 may, for example as shown in Figure 4, display the digital twin spatial image 202 generated by the spatial generation unit 22 on the display unit 201. When displaying the digital twin spatial image 202 on the display unit 201, the display processing unit 23 may also display operation buttons 203. The operation buttons 203 allow manipulation of the latitude, longitude, and altitude of the digital twin spatial image 202. By operating the left and right arrows on the operation buttons 203, the digital twin spatial image 202 with shifted latitude and longitude can be displayed, and by operating the up and down arrows on the operation buttons 203, the digital twin spatial image 202 with shifted altitude can be displayed.

[0046] (Display section 201) The display unit 201 may be a device that displays images, such as a liquid crystal display (LCD), a plasma display panel (PDP), or an organic electroluminescent (EL) display. Preferably, the display terminal has a touch panel function.

[0047] (Storage unit 24) The storage unit 24 is composed of an HDD, SSD, or memory. In this embodiment, for example, the storage unit 24 stores location information of terminal 1 acquired by the location information acquisition unit 21 and first voice information including the voice of the first user received by the voice information transmission / reception unit 25. The storage unit 24 may also store parts (components) for the space generation unit 22 to generate a digital twin spatial image.

[0048] (Voice information transmission / reception unit 25) The voice information transmission / reception unit 25 receives first voice information, which includes the voice of the first user 50 transmitted by terminal 1, and transmits second voice information, which includes the voice of the second user. In this embodiment, the voice information transmission / reception unit 25 receives voice information 16a, which includes the voice of the first user, and transmits voice information 25a, which includes the voice of the second user 51. In this way, the first user and the second user can communicate in both directions.

[0049] (Audio output unit 26) The audio output unit 26 is, for example, a speaker, and outputs the voice of the first user received by the audio information transmission / reception unit 25. In this embodiment, the audio output unit 26 can output the voice of the first user 50 while they are in a call with the second user 51. The audio output unit 26 can also output the sound of rain or thunder when the sound of rain or thunder is reproduced by the spatial generation unit 22.

[0050] (Speech acquisition unit 27) The voice acquisition unit 27 is, for example, a microphone, and acquires the voice of the second user 51 and generates second voice information. In this embodiment, the voice acquisition unit 27 acquires the voice of the second user 51, who is in a call with the first user 50, and converts it into voice information, thereby generating voice information that includes the voice of the second user.

[0051] (Server 4) Figure 5 is a functional block diagram showing server 4 of the embodiment. Server 4 is a device capable of communicating with terminal 1 and information processing device 2, such as a personal computer. Server 4 intervenes between terminal 1 and information processing device 2 to send and receive information when terminal 1 and information processing device 2 do not send and receive information directly. In this embodiment, as shown in Figure 5, server 4 includes a location information transmission / reception unit 41, a space generation processing unit 42, a voice information transmission / reception unit 43, and a storage unit 44. Note that server 4 does not necessarily have to include the voice information transmission / reception unit 43.

[0052] (Location information transmission / reception unit 41) The location information transmission / reception unit 41 can receive location information from terminal 1, including its latitude, longitude, altitude, and current time, and transmit it to the information processing device 2. The current time may be the time when terminal 1 transmitted the location information, or it may be the time when terminal 1 detected the latitude, longitude, and altitude.

[0053] For example, the location information transmission / reception unit 41 can receive location information 11a from terminal 1, which includes the current time while communicating with the information processing device 2, the latitude and longitude of terminal 1 at that time, and altitude, and transmit it to the information processing device 2.

[0054] (Space generation processing unit 42) The spatial generation processing unit 42 can generate a digital twin spatial image that reproduces the real space in terms of latitude, longitude, and altitude included in the location information received from terminal 1, and the surrounding environment of the real space at the current time included in the location information. More specifically, the spatial generation processing unit 42 generates a digital twin spatial image of the real space based on 3D model data, which is data that 3D models the real space in terms of latitude, longitude, and altitude included in the location information transmission / reception unit 41, and can reflect the surrounding environment of the real space at the current time included in the location information 11a in the generated digital twin spatial image. The spatial generation processing unit 42 can, for example, identify location 112 on the map 111 shown in Figure 3 from the latitude and longitude included in the location information via the internet, and can generate a digital twin spatial image of the real space at location 112. Similarly, the spatial generation processing unit 42 can identify the surrounding environment of the real space at the current time included in the location information via the internet, and can reflect the surrounding environment of the real space at the current time in the generated digital twin spatial image.

[0055] The spatial generation processing unit 42 can transmit the generated digital twin spatial image to the information processing device 2.

[0056] (Voice information transmission / reception unit 43) The voice information transmission / reception unit 43 receives first voice information, including the voice of the first user 50, transmitted by terminal 1, and transmits the first voice information to the information processing device 2. The voice information transmission / reception unit 43 also receives second voice information, including the voice of the second user 51, transmitted by the information processing device 2, and transmits the second voice information to terminal 1. In this embodiment, the voice information transmission / reception unit 43 can receive voice information 16a, including the voice of the first user, and transmit the voice information 16a to the information processing device 2. The voice information transmission / reception unit 43 can also receive voice information 25a, including the voice of the second user 51, transmitted by the information processing device 2, and transmit the voice information 25a to terminal 1.

[0057] (Storage unit 44) The storage unit 44 is composed of an HDD, SSD, or memory. In this embodiment, for example, the storage unit 44 is used for processing information transmission and reception by intervening between the terminal 1 and the information processing device 2. The storage unit 44 stores, for example, location information of the terminal 1 received by the location information transmission / reception unit 41, and first audio information including the voice of the first user and second audio information including the voice of the second user received by the audio information transmission / reception unit 43. The storage unit 44 may also store parts (components) for the spatial generation processing unit 42 to generate a digital twin spatial image, or it may store the digital twin spatial image generated by the spatial generation processing unit 42.

[0058] [3. Hardware Configuration] Figure 6 is a schematic diagram showing an example of the hardware configuration of the information processing device 2. Figure 7 is a schematic diagram showing an example of the hardware configuration of terminal 1.

[0059] Next, the hardware configurations of terminal 1 and information processing device 2 will be described using Figures 6 and 7.

[0060] (Hardware configuration of information processing device 2) The hardware configuration of the information processing device 2 is described below. As shown in Figure 6, the information processing device 2 of this embodiment uses, for example, a computer. The information processing device 2 is equipped with a CPU 170. The CPU 170 is connected to a memory 171, a connection port 173 for connecting / reading storage devices 172, etc., and a communication circuit 174 for communicating with the outside via a network, all via a bus line 175. The memory 171 stores a program 1713 for processing the information processing device 2. It may also store a browser program 1712 and an OS 1711 (operating system). Furthermore, the memory 171 may also store a location information database (hereinafter referred to as the location information DB) 1714 and a spatial generation processing database (hereinafter referred to as the spatial generation processing DB) 1715. Note that the memory 171 may be the storage unit 24, or it may be something different from the storage unit 24. If memory 171 and storage unit 24 are different, the location information DB 1714 and the spatial generation processing DB 1715 are stored in storage unit 24.

[0061] (Hardware configuration of terminal 1) The hardware configuration of terminal 1 is almost the same as that of the information processing device 2 described above, so the same parts are denoted by the same reference numerals, and their explanation is omitted. In this embodiment, terminal 1 uses, for example, a smartphone.

[0062] The computer's memory 171 stores a program 1716 for processing terminal 1. It may also store a browser program 1712 and an OS 1711 (operating system).

[0063] The memory 171 may also store the location information database 1717. Furthermore, the memory 171 may be the storage unit 14, or it may be a different memory unit. If the memory 171 and the storage unit 14 are different, the location information database 1717 is stored in the storage unit 14.

[0064] In this embodiment, programs 1713 and 1716 work together by utilizing the functions of OS 1711 and browser program 1712, respectively. Alternatively, programs 1713 and 1716 may operate independently without utilizing browser program 1712 and OS 1711, respectively.

[0065] In the hardware configuration of terminal 1 and information processing device 2 described above, the functions shown in Figure 2 are implemented, for example, using CPU 170, program 1713, and program 1716. However, some or all of these functions may be sequence-controlled using logic circuits such as a microcontroller or a PLC (programmable logic controller).

[0066] (location information DB1714, 1717) Location information databases 1714 and 1717 store location information of terminal 1, including its latitude, longitude, altitude, and current time. Location information databases 1714 and 1717 store, but are not limited to, the location information of terminal 1 at the current time when it is communicating with the information processing device 2. Location information databases 1714 and 1717 may store multiple locations of terminal 1 at predetermined time intervals.

[0067] (Space generation processing DB1715) The spatial generation processing DB1715 stores parts (components) for generating a digital twin spatial image. The spatial generation processing DB1715 may also store 3D model data, which is data that represents a three-dimensional model of the real space including objects.

[0068] The hardware configuration of server 4 is the same as that of the information processing device 2 described above, so its explanation will be omitted.

[0069] [4. Operation of Information Processing Device 2] (Operation of Information Processing Device 2) Next, the operation of the information processing device 2 will be explained using Figure 8. Figure 8 is a schematic flow chart showing an example of the operation of the information processing device 2. Figure 8 shows a flowchart illustrating one embodiment of the processing of program 1713 used in the information processing device 2.

[0070] (S11) The CPU 170 of the information processing device 2 acquires location information transmitted by the terminal 1 of the first user 50. In this embodiment, the information processing device 2 acquires location information 11a transmitted by the terminal 1 of the first user 50, which is being used by the second user 51 and communicating with the information processing device 2. The location information 11a includes the current time of the terminal 1, the latitude and longitude of the terminal 1 at that current time, and the altitude.

[0071] (S12) The CPU 170 of the information processing device 2 generates a digital twin spatial image that reproduces the real space in terms of latitude, longitude, and altitude included in the location information, and the surrounding environment of the real space at the current time included in the location information. In this embodiment, the information processing device 2 generates the real space in terms of latitude, longitude, and altitude included in the location information 11a acquired in step S11, and reflects the surrounding environment of the real space at the current time included in the location information 11a.

[0072] (S13) The CPU 170 of the information processing device 2 displays the digital twin spatial image generated in step S12 on the display unit. In this embodiment, the information processing device 2 displays the digital twin spatial image 202 generated in step S12 on the display unit 201, for example, as shown in Figure 4. When the information processing device 2 displays the digital twin spatial image 202 on the display unit 201, it may also display the operation buttons 203.

[0073] [5. Others] The above embodiments can be used in combination as appropriate.

[0074] For example, there may be multiple information processing devices 2. In that case, the first user 50 of terminal 1 and the second user 51 of the multiple information processing devices 2 are communicating with N parties. Furthermore, each of the multiple information processing devices 2 can acquire the location information of terminal 1 and use the acquired location information of terminal 1 to generate the same digital twin spatial image.

[0075] (Use Case 1) The above-described embodiment explains a case in which a digital twin spatial image is generated and displayed from the location information of a terminal 1 communicating with the information processing device 2, while acquiring the location information of the terminal 1. It also explains that the first user 50 of terminal 1 and the second user 51 of the information processing device 2 can communicate by phone. The above-described embodiment can be applied to use case 1, for example, as follows.

[0076] The above-described embodiment can be applied, for example, to use case 1 in which the first user 50 and the second user 51 are family members, and the first user makes a video call with the second user 51, who is in their home country, while the first user is traveling abroad. The terminal 1 may be equipped with a front camera and a rear camera. In this case, by having the information processing device 2 acquire the location information of the terminal 1, the second user 51 can see what is happening around the first user 50 using a digital twin spatial image generated from the location information of the terminal 1, in addition to the camera information captured by the front or rear camera of the terminal 1. The first user 50 and the second user 51 may be friends.

[0077] (Use Case 2) Alternatively, the system may acquire location information transmitted by terminal 1 while the first user 50 is in motion during a predetermined time period, and generate and display a digital twin spatial image from the acquired location information of terminal 1. Specifically, the location information acquisition unit 21 may acquire location information 11a transmitted by terminal 1 while the first user is in motion during a predetermined time period. In this case, the spatial generation unit 22 generates a digital twin spatial image for each current time that reproduces the real space at each latitude, longitude, and altitude included in the acquired location information during the predetermined time period, as well as the surrounding environment of the real space at each current time included in the location information. The display processing unit 23 can display the digital twin spatial image for each current time period generated by the spatial generation unit 22 on the display unit 201.

[0078] In this use case 2, the second user 51 of the information processing device 2 can virtually experience the real space where the first user 50 is moving through a digital twin spatial image. For example, if the first user 50 goes out (travels) for a trip or shopping, the second user 51 of the information processing device 2 can virtually experience what the area (real space) around the first user 50 is while they are out (traveling). The first user 50 and the second user 51 may be friends or family members. Furthermore, permission may be required for the information processing device 2 to acquire the location information transmitted by terminal 1.

[0079] (Use Case 3) Alternatively, a digital twin spatial image may be generated that reproduces the real space from an overhead viewpoint at an altitude from which the destination to which the first user 50 is traveling can be viewed, rather than at the altitude where the terminal 1 of the first user 50 is located. Specifically, the spatial generation unit 22 may generate a digital twin spatial image that reproduces the real space from an overhead viewpoint at an altitude higher than the current time's latitude, longitude, and altitude, and from which the destination to which the first user 50 is traveling can be viewed. In this case, the display processing unit 23 can display the overhead viewpoint digital twin spatial image generated by the spatial generation unit 22 on the display unit 201.

[0080] This allows the second user 51, who is viewing the overhead digital twin spatial image, to check the vicinity of the first user 50's current location and to get an overhead view of the area from the first user 50's current location to the destination. Therefore, the second user 51 can, for example, advise the first user 50 on the surrounding area that they cannot see themselves, how to get to the destination, and the route to the destination. Thus, in Use Case 3, the first user 50 can use the location information of terminal 1 to receive advice from the second user 51 regarding the surrounding area that the first user 50 cannot see, as well as directions and routes to the destination. The first user 50 and the second user 51 may be friends or family members.

[0081] [6. Summary] (1, 8, 9) The information processing device 2 is an information processing device 2 used by a second user 51 that is communicating with a terminal 1 used by a first user 50, and is characterized by comprising: a location information acquisition unit 21 that acquires location information 11a transmitted by the terminal 1, which includes the current time of the terminal 1, the latitude and longitude of the terminal 1 at the current time, and the altitude; a space generation unit 22 that generates a digital twin spatial image that reproduces the real space at the latitude and longitude and altitude included in the location information 11a and the surrounding environment of the real space at the current time included in the location information 11a; and a display processing unit 23 that displays the digital twin spatial image generated by the space generation unit 22 on a display unit 201.

[0082] As a result, the information processing device 2 can acquire the current location information of terminal 1, which is communicating with the information processing device 2, and generate and display a digital twin spatial image of the real space in which the first user 50 is currently located, based on the acquired location information of terminal 1.

[0083] Furthermore, since the digital twin spatial image displayed by the information processing device 2 reflects the surrounding environment of the real space in which the first user 50 currently resides, the second user 51 of the information processing device 2 can view or virtually experience the area near the real space in which the first user 50 currently resides through the digital twin spatial image.

[0084] Thus, according to the information processing device 2, the real-world space where the first user is located can be experienced in the digital twin space.

[0085] (2) Such an information processing device 2 may include an audio information transmission / reception unit 25 that receives first audio information including the voice of the first user 50 transmitted by terminal 1 and transmits second audio information including the voice of the second user 51; an audio acquisition unit 27 that acquires the voice of the second user 51 and generates second audio information; and an audio output unit 26 that outputs the voice of the first user 50. This allows the first user 50 of terminal 1 and the second user 51 of the information processing device 2 to communicate. Thus, while communicating with the first user 50, the second user 51 can view or virtually experience the surrounding area of ​​the real space where the first user 50 is currently located using a digital twin spatial image.

[0086] (3) Here, the surrounding environment includes weather conditions including at least one of cloud cover, snow, thunderstorms, and rain, and rain may include cases of heavy rainfall or light rainfall. In this way, the digital twin spatial image displayed by the information processing device 2 reflects the weather conditions as the surrounding environment of the real space in which the first user 50 currently exists. As a result, the second user 51 can view or virtually experience the weather conditions of the real space in which the first user 50 currently exists through the digital twin spatial image.

[0087] (4) Furthermore, if the surrounding environment includes rain or thunderstorms, the generated digital twin spatial image may also include the sound of rain or thunder. In this way, the digital twin spatial image displayed by the information processing device 2 can reflect the sound of rain or thunder as the surrounding environment of the real space in which the first user 50 is currently located. As a result, the second user 51 can not only see the weather conditions of the real space in which the first user 50 is currently located in the digital twin spatial image, but also virtually experience them by hearing the sounds of rain and thunder.

[0088] (5) The surrounding environment may also include brightness, the position of the sun, or the position of the moon. Thus, the digital twin spatial image displayed by the information processing device 2 reflects the surrounding environment of the real space in which the first user 50 currently resides, including brightness, the position of the sun, or the position of the moon. As a result, the second user 51 can not only view the conditions of the real space in which the first user 50 currently resides, such as day and night, evening conditions, and the position of the sun or moon, through the digital twin spatial image, but also experience them virtually.

[0089] (6) In such an information processing device 2, the location information acquisition unit 21 acquires location information 11a transmitted by the terminal 1 while the first user 50 is moving at a predetermined time, the space generation unit 22 generates digital twin space images for each current time that reproduce the real space at each latitude, longitude, and altitude included in each location information 11a acquired at the predetermined time, and the surrounding environment of the real space at each current time included in each location information, and the display processing unit 23 can display the digital twin space images for each current time generated by the space generation unit 22 on the display unit 201.

[0090] As a result, the second user 51 of the information processing device 2 can virtually experience the real-world environment in which the first user 50 is moving, using digital twin spatial images. For example, when the first user 50 goes out (travels) for a trip or shopping, the second user 51 of the information processing device 2 can virtually experience the surrounding area (real-world environment) in which the first user 50 is traveling.

[0091] (7) In addition, in such an information processing device 2, the space generation unit 22 further generates a digital twin spatial image that reproduces the real space from an overhead viewpoint, which is at an altitude higher than the current time's latitude and longitude, and an altitude from which the destination to which the first user 50 is traveling can be viewed from above. The display processing unit 23 can then display the digital twin spatial image from an overhead viewpoint generated by the space generation unit 22 on the display unit 201.

[0092] This allows the second user 51, who is viewing the overhead digital twin spatial image, to check the vicinity of the first user 50's current location and to get an overhead view of the area from the first user 50's current location to the destination. Therefore, the second user 51 can, for example, advise the first user 50 on the surrounding area that they cannot see themselves, how to get to the destination, and the route to the destination. [Explanation of Symbols]

[0093] 1 terminal 2. Information Processing Device 3. Communication Network 4 servers 11 Sensor section 11a Location information 12. Audio acquisition unit 13 Location Information Transmission Unit 14, 24, 44 storage section 15, 26 Audio output section 16, 25, 43 Voice Information Transmitting and Receiving Unit 16a Audio Information 21 Location information acquisition section 22 Space generation part 23 Display Processing Unit 41 Location information transmission / reception unit 42 Space Generation Processing Unit 50 First User 51 Second User 111 Map 112 positions 170 CPU 171 memory 172 Storage Devices 173 connection ports 174 Communication Circuits 175 Bus Line 201 Display section 202 Digital Twin Spatial Image 203 Operation Buttons 1711 OS 1712 Browser Program Programs 1713 and 1716 1714, 1717 Location information DB 1715 Space Generation Processing DB

Claims

1. An information processing device used by a second user that is communicating with a terminal used by a first user, A location information acquisition unit acquires location information transmitted by the terminal, including the current time of the terminal, the latitude and longitude of the terminal at the current time, and the altitude. A space generation unit generates a digital twin spatial image that reproduces the real space at the latitude, longitude, and altitude included in the location information, and the surrounding environment of the real space at the current time included in the location information. The system includes a display control unit that displays the digital twin spatial image generated by the spatial generation unit on a display unit. Information processing device.

2. A voice information transmitting and receiving unit that receives first voice information including the voice of the first user transmitted by the terminal and transmits second voice information including the voice of the second user, A voice acquisition unit that acquires the voice of the second user and generates second voice information, The system includes an audio output unit that outputs the voice of the first user, The information processing apparatus according to claim 1.

3. The aforementioned surrounding environment includes meteorological conditions including at least one of cloud cover, snow, thunderstorms, and rain. The aforementioned rainy weather includes cases with heavy rainfall or light rainfall. The information processing apparatus according to claim 1.

4. Furthermore, if the surrounding environment includes rain or thunder, the digital twin spatial image will include the sound of rain or thunder. The information processing apparatus according to claim 3.

5. The aforementioned surrounding environment includes brightness, the position of the sun or the position of the moon, The information processing apparatus according to claim 3 or 4.

6. The location information acquisition unit acquires location information transmitted by the terminal while the first user is moving within a predetermined time period. The space generation unit generates digital twin space images for each current time, which reproduce the real space at each of the latitude, longitude, and altitude included in the location information acquired during the predetermined time, and the surrounding environment of each of the real spaces at each current time included in the location information. The display control unit causes the display unit to display the digital twin spatial images of each current time generated by the spatial generation unit. The information processing apparatus according to claim 1.

7. The aforementioned space generation unit further, A digital twin spatial image is generated that reproduces the real space from an overhead viewpoint, which is at an altitude higher than the latitude and longitude at the current time and at an altitude from which the destination to which the first user is traveling can be viewed from above. The display control unit causes the display unit to display the overhead view digital twin spatial image generated by the spatial generation unit. The information processing apparatus according to claim 1.

8. A program for an information processing device used by a second user that is communicating with a terminal used by a first user, A location information acquisition step involves acquiring location information transmitted by the terminal, which includes the current time of the terminal, the latitude and longitude of the terminal at the current time, and the altitude. A spatial generation step that generates a digital twin spatial image that reproduces the real space at the latitude, longitude, and altitude included in the location information, and the surrounding environment of the real space at the current time included in the location information; A display control step which causes the digital twin spatial image generated in the spatial generation step to be displayed on the display unit, A program that causes a computer to execute something.

9. An information processing method for an information processing device used by a second user that is communicating with a terminal used by a first user, A location information acquisition step involves acquiring location information transmitted by the terminal, which includes the current time of the terminal, the latitude and longitude of the terminal at the current time, and the altitude. A spatial generation step that generates a digital twin spatial image that reproduces the real space at the latitude, longitude, and altitude included in the location information, and the surrounding environment of the real space at the current time included in the location information; The process includes a display control step that causes the digital twin spatial image generated in the spatial generation step to be displayed on a display unit, Information processing methods.

Images